[00:01:49] <wikibugs>	 Analytics-Cluster: Identify mobile apps as a separate agent_type - https://phabricator.wikimedia.org/T96324#1214354 (Yurik) NEW
[00:03:04] <wikibugs>	 Analytics-Cluster: Identify mobile apps as a separate agent_type - https://phabricator.wikimedia.org/T96324#1214363 (Yurik)
[00:15:31] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Compute pageviews aggregates daily and monthly from April {crow} - https://phabricator.wikimedia.org/T96067#1214402 (kevinator)
[00:15:33] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: {epic} Analyst runs query to get aggregated pageview counts {crow} - https://phabricator.wikimedia.org/T96314#1214401 (kevinator)
[00:31:09] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: setup 'testing' dataset on hive for Impala {crow} - https://phabricator.wikimedia.org/T96328#1214431 (kevinator) NEW
[00:33:41] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Install Impala on cluster {crow} - https://phabricator.wikimedia.org/T96329#1214443 (kevinator) NEW
[00:34:16] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: setup 'testing' dataset on hive for Impala {crow} [13 pts] - https://phabricator.wikimedia.org/T96328#1214450 (kevinator)
[00:35:28] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: test performance of Impala {crow} [8 pts] - https://phabricator.wikimedia.org/T96330#1214453 (kevinator) NEW
[00:36:49] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Test Impalla operationally {crow} - https://phabricator.wikimedia.org/T96331#1214459 (kevinator) NEW
[00:40:48] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: {epic} Analyst runs query to get aggregated pageview counts {crow} - https://phabricator.wikimedia.org/T96314#1214468 (kevinator)
[00:54:25] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Compute pageviews aggregates daily and monthly from April {wren} - https://phabricator.wikimedia.org/T96067#1214518 (kevinator)
[00:54:35] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: setup 'testing' dataset on hive for Impala {wren} [13 pts] - https://phabricator.wikimedia.org/T96328#1214526 (kevinator)
[00:54:47] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Install Impala on cluster {wren} - https://phabricator.wikimedia.org/T96329#1214527 (kevinator)
[00:55:03] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: test performance of Impala {wren} [8 pts] - https://phabricator.wikimedia.org/T96330#1214531 (kevinator)
[00:55:26] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Test Impalla operationally {wren} - https://phabricator.wikimedia.org/T96331#1214533 (kevinator)
[00:55:38] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: {epic} Analyst runs query to get aggregated pageview counts {wren} - https://phabricator.wikimedia.org/T96314#1214537 (kevinator)
[00:58:27] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Compute pageviews aggregates daily and monthly from April {wren} - https://phabricator.wikimedia.org/T96067#1214555 (kevinator) p:Triage>Normal
[01:16:52] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] "I want them at the very minimum in the workflow.xml file, so that the workflow.xml file could be used on its own without a coordinator or " [analytics/refinery] - https://gerrit.wikimedia.org/r/204621 (owner: Ottomata)
[01:19:01] <wikibugs>	 Analytics-Kanban, Analytics-Visualization: Create modular build system for Dashiki - https://phabricator.wikimedia.org/T96337#1214598 (kevinator) NEW a:Milimetric
[01:19:22] <ottomata>	 ahhhhh jobs all stuck again, this time my fault, too many fixes!
[01:19:25] <ottomata>	 going to try more things.
[01:19:27] <ottomata>	 yargghh
[01:19:41] <wikibugs>	 Analytics-Kanban, Analytics-Visualization: Create modular build system for Dashiki [21 pts] - https://phabricator.wikimedia.org/T96337#1214598 (kevinator)
[01:20:43] <wikibugs>	 Analytics-Kanban, Analytics-Visualization: Finish testing untested Dashiki code [8 pts] - https://phabricator.wikimedia.org/T96338#1214608 (kevinator) NEW a:Milimetric
[01:22:17] <wikibugs>	 Analytics-Kanban, Analytics-Visualization: Integrate Dygraphs into Vital Signs [13 pts] - https://phabricator.wikimedia.org/T96339#1214615 (kevinator) NEW a:Milimetric
[01:25:59] <madhuvishy>	 nuria: around?
[01:26:16] <madhuvishy>	 do you have special settings for gerrit on .ssh/config?
[07:55:11] <wikibugs>	 Analytics-EventLogging, Hovercards: Large number of popup events not validating - https://phabricator.wikimedia.org/T91272#1214830 (Prtksxna) Bump!
[11:14:55] <wikibugs>	 Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic, Google-Summer-of-Code-2015, Outreachy-Round-10: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#1215003 (Aklapper) In my humble opinion: Microtasks help students to...
[11:24:25] <wikibugs>	 Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic, Google-Summer-of-Code-2015, Outreachy-Round-10: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#1215021 (Sarvesh.onlyme) Thanks @Aklapper, I'm on it. I've created/st...
[14:07:05] <nuria>	 holaaaa
[14:12:32] <ottomata>	 howdyyy
[14:14:00] <wikibugs>	 Analytics-Kanban: Update pageview definition to take hoovercards into account  - https://phabricator.wikimedia.org/T96367#1215362 (Nuria) NEW
[14:15:27] <wikibugs>	 Analytics-Kanban: Update pageview definition to take hovercards into account  - https://phabricator.wikimedia.org/T96367#1215379 (Nuria)
[14:15:44] <wikibugs>	 Analytics-Kanban: Update pageview definition to take hovercards into account - https://phabricator.wikimedia.org/T96367#1215362 (Nuria)
[14:18:53] <wikibugs>	 Analytics-EventLogging, Hovercards: Large number of popup events not validating - https://phabricator.wikimedia.org/T91272#1215408 (Nuria) Once you deploy the new code we can close this ticket.
[14:21:41] <grrrit-wm>	 (CR) Nuria: "Sounds like oozie doesn't know how to specify these properties globally yet: http://mail-archives.apache.org/mod_mbox/oozie-dev/201501.mbo" [analytics/refinery] - https://gerrit.wikimedia.org/r/204621 (owner: Ottomata)
[14:24:23] <wikibugs>	 Analytics-Tech-community-metrics: Community Metrics for IRC channels not updated since 09/2013 - https://phabricator.wikimedia.org/T96371#1215416 (Aklapper) NEW
[14:29:36] <wikibugs>	 Analytics-Wikistats: Wikistats report on crawler stats broken - https://phabricator.wikimedia.org/T96372#1215429 (Aklapper) Please associate at least one [[ https://phabricator.wikimedia.org/project/query/active/ | project ]] with this task, otherwise nobody can find this task when searching in the correspon...
[14:40:12] <ottomata>	 joal|night: nm about the oozie coordinators as a bundle idea
[14:40:17] <ottomata>	 new version of hue will make this easier:
[14:40:18] <ottomata>	 http://gethue.com/oozie-dashboard-improvements/
[14:54:12] <wikibugs>	 Analytics-Cluster, Analytics-Kanban, Easy: Add better detection of wikipediaApp to user agent UDF - https://phabricator.wikimedia.org/T96376#1215480 (Nuria) NEW
[15:13:16] <milimetric>	 mforns: good work, man, I'm very curious to read your report
[15:13:24] <mforns>	 milimetric, xD
[15:13:33] <mforns>	 ok, writing that
[15:14:30] <madhuvishy>	 Gah I thought I'd make it to office for standup but missed the train by 5 seconds.
[15:24:41] <wikibugs>	 Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic, Google-Summer-of-Code-2015, Outreachy-Round-10: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#1215551 (QuimGil) It is unfortunate that now you have to deal with ex...
[15:26:12] <wikibugs>	 Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic, Google-Summer-of-Code-2015, Outreachy-Round-10: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#1215552 (Qgil) It is unfortunate that now you have to deal with exams...
[15:31:38] <milimetric>	 ottomata: exceptional standup
[15:31:52] <ottomata>	 OOop k
[15:31:59] <ottomata>	 thank you
[16:03:48] <wikibugs>	 Analytics-Kanban: Update pageview definition to take hovercards into account - https://phabricator.wikimedia.org/T96367#1215708 (leila) @Nuria As of now, Hovercard views should not be counted towards pageviews. We should revisit this task in some weeks from now, after we have results from Catalan and Greek, a...
[16:11:22] <wikibugs>	 Analytics-Kanban: Update pageview definition to take hovercards into account - https://phabricator.wikimedia.org/T96367#1215720 (Nuria) Hovercards are now api requests: http://ca.wikipedia.org/w/api.php?action=query&format=json&prop=extracts%7Cpageimages%7Crevisions%7Cinfo&redirects=true&exintro=true&exsenten...
[16:13:07] <wikibugs>	 Analytics-Kanban: Update pageview definition to take hovercards into account - https://phabricator.wikimedia.org/T96367#1215730 (Nuria) @leila: understood.  This is more to point out that -if we wanted to count them- we couldn't, given that those pageviews are not tagged.
[16:19:01] <wikibugs>	 Analytics-Kanban: Update pageview definition to take hovercards into account - https://phabricator.wikimedia.org/T96367#1215755 (leila) makes sense, @Nuria.
[16:24:05] <halfak>	 milimetric, when you have a minute, scope out this: https://meta.wikimedia.org/wiki/Research_talk:VisualEditor%27s_effect_on_newly_registered_editors/Work_log/2015-04-17
[16:24:26] <halfak>	 I looked at the ready --> saveSuccess rate for VE and wikitext and they look reasonable.
[16:24:55] <milimetric>	 when you say reasonable?
[16:25:03] <milimetric>	 I mean, what do you mean by reasonable
[16:25:27] <halfak>	 Well, for VE, it's 24% and for wikitext it's 20%
[16:25:31] <halfak>	 So, not that much different
[16:25:45] <halfak>	 And the copy-paste pattern of wikitext could easily explain the 4%
[16:26:33] <milimetric>	 I see, looking at your query now
[16:28:21] * halfak apologizes in advance for naming the subquery "foo"
[16:28:27] <milimetric>	 :)
[16:28:53] <milimetric>	 ok, halfak so I'm not sure it makes sense to look at the data without drawing the session boundaries
[16:29:06] <halfak>	 milimetric, I am drawing session boundaries
[16:29:10] <halfak>	 In the subquery
[16:29:19] <halfak>	 I'm getting counts of event_action per sessionId
[16:29:34] <halfak>	 Then counting the number of sessions with event_action counts > 0
[16:29:45] <milimetric>	 halfak: but in VE, that will get you duplicates, so you have  to group by clientIp
[16:30:33] <milimetric>	 oh ok, sorry
[16:30:50] <milimetric>	 i'm a little distracted now, let me get back to you after the quarterly
[16:31:16] <halfak>	 sounds good
[16:43:02] <joal|night>	 ottomata: got in quaterly meeting
[16:43:13] <joal|night>	 Will talk later ... But makes no sense
[16:53:49] <joal|night>	 ottomata: yarn-site.xml --> yarn.scheduler.maximum-allocation-mb
[16:56:34] <ottomata>	 hmm
[16:56:58] <ottomata>	 42080
[16:56:59] <ottomata>	 ?
[16:57:25] <ottomata>	 yarn-site.xml:    <name>yarn.scheduler.minimum-allocation-mb</name>
[16:57:25] <ottomata>	 yarn-site.xml-    <value>2104</value>
[16:57:25] <ottomata>	 yarn-site.xml-  </property>
[16:57:28] <ottomata>	 that could help
[16:57:30] <ottomata>	 ?
[16:57:31] <ottomata>	 maybe
[16:57:34] <joal|night>	 ottomata: Yeah, exacvtly
[16:57:40] <joal|night>	 mixed mini and maxi
[16:58:00] <ottomata>	 not quite 3072, but worth a try
[16:58:09] <joal|night>	 Nothing better on my side :(
[16:58:29] <ottomata>	 i'm getting lost in Hadoop source code
[16:58:38] <ottomata>	 too many abstract classes, can't find any real stuff :p
[16:58:47] <ottomata>	 I'm all like, all cool Resource getMemory()
[16:58:55] <ottomata>	 oh that is not implemented in Resource
[16:59:00] <ottomata>	 what the kind of resource is this...
[16:59:07] <ottomata>	 rabbit hole city...:p
[16:59:11] <joal|night>	 Yup
[16:59:21] <joal|night>	 Quite complex system as well
[17:00:29] <ottomata>	 haha, joal, omg
[17:00:30] <ottomata>	 https://github.com/apache/hadoop/blob/e1109fb65608a668cd53dc324dadc6f63a74eeb9/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/resource/DefaultResourceCalculator.java#L65
[17:01:03] <ottomata>	 hmm, oh that might be fore a different scheudler than fair?
[17:01:04] <ottomata>	 iunnoooo
[17:01:08] <ottomata>	 let's just try your setting :p
[17:01:39] <joal|night>	 :)
[17:02:52] <joal|night>	 ottomata: interesting !
[17:02:58] <joal|night>	 http://blog.cloudera.com/blog/2013/06/improvements-in-the-hadoop-yarn-fair-scheduler/
[17:03:15] <joal|night>	 What do you think about hierarchical queues ?
[17:06:25] <ottomata>	 could be useful, but it seems like it wouldn't really make a difference?  not sure.
[17:07:08] <joal|night>	 oozie might be a subqueue of production ?
[17:07:23] <joal|night>	 With higher prio
[17:07:55] <ottomata>	 weird, joal, that setting is
[17:07:56] <ottomata>	 740
[17:08:00] <joal|night>	 ottomata: can I share our spark conf with altiscale ?
[17:08:02] <ottomata>	 for the launcher container we were looking at
[17:08:10] <ottomata>	 joal, sure
[17:08:15] <ottomata>	 it is already public
[17:08:17] <joal|night>	 ottomata: WHAT ?
[17:08:19] <ottomata>	 sorty
[17:08:20] <ottomata>	 sorta
[17:08:23] <joal|night>	 :)
[17:08:25] <ottomata>	 everything in puppet is on github
[17:08:32] <joal|night>	 k
[17:08:36] <ottomata>	 i guess the rendered template isn't though
[17:08:37] <ottomata>	 but ja
[17:08:38] <ottomata>	 whatever
[17:08:42] <joal|night>	 setting is 740 ?
[17:08:52] <joal|night>	 What do you mean ? on 019 ?
[17:09:20] <ottomata>	 https://github.com/wikimedia/puppet-cdh/tree/master/templates/spark
[17:09:20] <ottomata>	 https://github.com/wikimedia/puppet-cdh/blob/master/manifests/spark.pp
[17:09:36] <ottomata>	 yes, in that oozie:launcher container we were looking at
[17:09:44] <ottomata>	 yarn.scheduler.minimum-allocation-mb
[17:09:44] <ottomata>	 740
[17:09:52] <ottomata>	 don't make no sense!
[17:09:55] <joal|night>	 in
[17:09:55] <joal|night>	 deed
[17:10:01] <joal|night>	 NO CLUE
[17:10:04] <joal|night>	  :(
[17:10:16] <ottomata>	 will try it anyway
[17:10:24] <joal|night>	 mouarf ...
[17:23:46] <ottomata>	 joal|night: i set both minimum and maxmimum scheduler mb
[17:23:48] <ottomata>	 to 256
[17:23:51] <ottomata>	 no effect
[17:24:24] <joal|night>	 weird ...
[17:27:14] <joal|night>	 ottomata: this setting is per node, right ?
[17:27:40] <joal|night>	 ottomata: I am looking at if we can change it without restart
[17:29:00] <joal|night>	 This change needs a recycle of the RM service
[17:29:04] <joal|night>	 ottomata: --^
[17:29:38] <joal|night>	 And by the way, this parameter can be set at job level
[17:31:16] <joal|night>	 Another stackoverflow reading says: <property> <name>oozie.launcher.mapreduce.map.memory.mb</name> <value>4096</value> </property> <property> <name>oozie.launcher.mapreduce.map.java.opts</name> <value>-Xmx3072m</value> </property>
[17:31:23] <joal|night>	 humf ...
[17:59:30] <joal|night>	 halfak: +1 for research collaborations ;)
[17:59:58] <halfak>	 Woo!  One thing I hope to bring up with Lila soon is my work you joal|night and milimetric.
[18:00:10] <joal|night>	 :D
[18:00:15] <halfak>	 If you want something to be productized, give me access to smart people who know how to scale big problems.
[18:00:26] <milimetric>	 aw, that sucks!
[18:00:28] <joal|night>	 I still did nothing, not even a working spark jar on altiscale ;)
[18:00:36] <halfak>	 joal|night, lame :P
[18:00:44] <joal|night>	 Muhahaha
[18:00:46] <milimetric>	 halfak: how about, "give me access to smart people who know how to scale big problems." and Dan
[18:00:51] <halfak>	 :P
[18:01:10] <milimetric>	 no, seriously, this is great.
[18:01:30] <halfak>	 +1  I dunno if I have felt so good at the end of quarterly review before
[18:01:57] <joal|night>	 \o/
[18:02:01] <milimetric>	 it was pretty awesome, I think Lila and Terry have a pretty great attitude
[18:11:11] <milimetric>	 https://www.irccloud.com/pastebin/FQshcNOU
[18:11:16] <milimetric>	 halfak: ^^
[18:11:36] <milimetric>	 that's almost exactly what I'm seeing in the funnel
[18:11:50] <halfak>	 Gotcha.
[18:12:00] <milimetric>	 the "successes_with_init_ready" I mean
[18:12:01] <halfak>	 So it looks like the init and/or ready is missing.
[18:12:06] <milimetric>	 yes
[18:12:19] <halfak>	 I've noticed in my tests that I sometimes don't get an "init" event.
[18:12:20] <milimetric>	 and I discounted sessions like that because I considered them degenerate
[18:12:27] <milimetric>	 as you can see they make almost no difference in VE
[18:12:40] <milimetric>	 I should've re-checked that assumption when I looked at Wikitext data
[18:12:46] <milimetric>	 but yeah, yet another reason to not trust the data
[18:13:07] <milimetric>	 like, it should be theoretically impossible to have a session without an init :)
[18:14:14] <milimetric>	 brb, fooding
[18:16:47] <halfak>	 milimetric, hopefully not wet flooding
[18:16:56] <halfak>	 oh.. misread food
[18:16:58] <halfak>	 derp
[18:17:03] * halfak goes for food too
[18:28:16] <ottomata>	 joal|night: i changed it in a job a submitted
[18:28:19] <ottomata>	 not on the resourcemanger default
[18:28:25] <ottomata>	 just like you say
[18:28:31] <ottomata>	 i did not try the java.ops one yet
[18:41:47] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: Troubleshoot EventLogging missing chunks of data - https://phabricator.wikimedia.org/T96082#1216401 (mforns) We found a very likely explanation to the "data gap" issues. The involved code is in: server/eventlogging/handlers.py  ``` events = co...
[18:42:50] <nuria>	 ottomata, joal: after testing it couple times i think kyro serialization has no "visible" effect on job
[18:43:02] <nuria>	 ottomata, joal: so I am not going to include it
[18:48:01] <ottomata>	 aye
[18:48:01] <ottomata>	 k
[18:51:29] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: Troubleshoot EventLogging missing chunks of data {oryx} - https://phabricator.wikimedia.org/T96082#1216489 (kevinator) p:Triage>High
[18:57:32] <joal|night>	 nuria: ok sounds g
[18:57:32] <joal|night>	 ood
[18:57:39] <joal|night>	 at least, no negative effect ;)
[18:57:49] <joal|night>	 ottomata: has it worked ?
[18:58:04] <joal|night>	 Ironholds: you there ?
[18:58:07] <ottomata>	 i haven't tried opts
[18:58:10] <ottomata>	 ok trying..
[18:58:34] <Ironholds>	 joal|night, yes?
[18:58:48] <joal|night>	 production queue seems to work great ottomata :)
[18:59:12] <joal|night>	 Ironholds: Still a few questions on pageview dewf
[18:59:24] <Ironholds>	 and they are..?
[19:00:06] <joal|night>	 I extracted a split of uri_host with dot as a separator
[19:00:41] <joal|night>	 and grouped by that (from portion 0 to 4), access_method etc
[19:00:49] <joal|night>	 as for the needed aggregation
[19:01:09] <joal|night>	 I have quite a few xx.mobile.xxxx.org
[19:01:19] <joal|night>	 that are marked as desktop
[19:01:26] <joal|night>	 same for xx.wap.xxxx.org
[19:01:38] <Ironholds>	 err
[19:01:44] <Ironholds>	 do you have some example lines?
[19:02:26] <joal|night>	 I aggregated and counted
[19:02:32] <ottomata>	 joal|night: no effect
[19:02:32] <ottomata>	 TotalMemoryNeeded	3072
[19:02:47] <joal|night>	 ottomata: Sheeeeeeeeeeeeeeeeiiiiiiiiiiit
[19:03:21] <Ironholds>	 joal|night, then I'm not sure what you want me to say, except "it should not be doing that" ;p
[19:04:06] <joal|night>	 Question is: I know that usually xx.mobile.xxxx.org (or wap) redirect to xx.m.xxxx.org
[19:04:13] <joal|night>	 It seems in some case not ...
[19:04:20] <ottomata>	 redirects get logged too, no?
[19:04:26] <joal|night>	 Yes
[19:04:56] <joal|night>	 and is_pageview removes redirect, so the examples I have are no redirects
[19:05:39] <Ironholds>	 https://en.wap.wikipedia.org/wiki/Main_Page
[19:05:42] <Ironholds>	 find me the redirect ;)
[19:05:43] <joal|night>	 or My question is: should we include mobile/wap in apgeview def ?
[19:05:58] <Ironholds>	 well, mobile and wap should be in the access method categoriser
[19:06:02] <Ironholds>	 if they're not I implemented it wrong
[19:06:16] <Ironholds>	 it should be \\.(m|zero|wap|mobile)\\.
[19:06:44] <joal|night>	 That's not what I have :(
[19:07:10] <Ironholds>	 then yeah, it was implemented wrong (almost certainly by me) and should be patched
[19:07:15] <joal|night>	 Ok, this explains that then :)
[19:07:47] <joal|night>	 Thx for the answer, I 'll continue my investigation and will ask if any other need !
[19:35:18] <ottomata>	 milimetric: yt?  batcave with me for  a sec about mondrian impala?
[19:42:36] <nuria>	 mforns: I will be back in an hour, let me know if you need help backfilling the 1 hour gaps, should be much easier that backfilling sparse events
[19:42:53] <mforns>	 nuria, ok
[19:43:13] <nuria>	 mforns: but we can also talk about it Monday
[19:43:28] <nuria>	 mforns: if it is getting too late for'ya
[19:43:29] <mforns>	 nuria, well I'll start today, see if I can finish
[19:43:37] <mforns>	 nuria, not for now
[19:44:20] <nuria>	 mforns: https://wikitech.wikimedia.org/wiki/EventLogging/Backfilling#Checkout_a_separate_clone_of_EventLogging
[19:44:55] <joal|night>	 End of day for me guys
[19:45:01] <mforns>	 nuria, ok will try that :]
[19:45:05] <joal|night>	 Have a good weekend !
[19:45:11] <mforns>	 night joal|night!
[19:45:18] <nuria>	 mforns: no local changes should be needed to the code as you can backfil using the current code
[19:45:24] <mforns>	 nuria, ok
[19:46:21] <nuria>	 mforns: and db can sustain a LOT more traffic (from different processs) than the one it currently has , about 1 order of magnitude more of staetements is no problem
[19:49:55] <mforns>	 nuria, aha
[20:13:08] <milimetric>	 ottomata: back
[20:13:12] <milimetric>	 batcave still?
[20:13:15] <ottomata>	 ja 1 min
[20:29:02] <milimetric>	 madhuvishy: how's it going, anything I can help with?
[20:32:57] <madhuvishy>	 milimetric: following the crumbs and reading validation code. do you want to pair on this may be? i want to see how you run tests etc. (i need 30 minutes though for lunch)
[20:34:36] <milimetric>	 madhuvishy: no problem, I'll be around for another hour and a half
[20:34:40] <milimetric>	 so ping me when you're done eating
[20:34:50] <madhuvishy>	 great.
[21:11:10] <madhuvishy>	 milimetric: back from lunch
[21:11:23] <milimetric>	 madhuvishy: cool, to the batcave!
[21:28:03] <nuria>	 madhuvishy: i will be here later on today but millimetric is probably wrapping up let me know if you need help
[21:33:32] <madhuvishy>	 nuria: sure!
[21:59:08] <grrrit-wm>	 (PS1) Milimetric: [WIP] Build process work, refactor in progress [analytics/dashiki] - https://gerrit.wikimedia.org/r/204951
[22:01:18] <milimetric>	 have a nice weekend everyone
[22:16:49] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: Troubleshoot EventLogging missing chunks of data {oryx} - https://phabricator.wikimedia.org/T96082#1217321 (Nuria) Such a good report!
[22:49:57] <madhuvishy>	 kevinator: ping
[23:01:11] <madhuvishy>	 nuria: do you know if i have to setup a wikimetrics user in the database? i tried to run tests and ended up with OperationalError: (_mysql_exceptions.OperationalError) (1044, "Access denied for user 'wikimetrics'@'localhost' to database 'blah_testing'")
[23:01:30] <nuria>	 "blah_testing"?
[23:01:37] <grrrit-wm>	 (CR) QChris: "Since this commit, 'mvn package' fails for me on stat1002 with:" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/204614 (https://phabricator.wikimedia.org/T93952) (owner: Ottomata)
[23:01:42] <madhuvishy>	 nuria: ha ha that's what it's called I presume
[23:02:01] <nuria>	 madhuvishy: you must have created that one right?
[23:02:10] <nuria>	 madhuvishy: cause testing databases are
[23:02:15] <madhuvishy>	 nuria: i din't..
[23:02:24] <madhuvishy>	 nuria: may be i should?
[23:02:43] <nuria>	 https://www.irccloud.com/pastebin/wafCLOHC
[23:03:03] <nuria>	 the dbs i have on vagrant:
[23:03:03] <nuria>	 https://www.irccloud.com/pastebin/ok3brkWY
[23:03:21] <nuria>	 "blah_testing" sounds like a db that should not exists
[23:04:43] <madhuvishy>	 this is what i also have https://www.irccloud.com/pastebin/9yTQw63t
[23:05:02] <madhuvishy>	 nuria: this is funny. let me look
[23:05:56] <nuria>	 madhuvishy: teh wikimetrics usrr will not have access cause grant access statements are on puppet for the 'real' _testing dbs only
[23:06:05] <nuria>	 *the wikimetrics user
[23:07:53] <madhuvishy>	 nuria: hmmm. i don't know why it's looking for blah_testing
[23:08:07] <nuria>	 madhuvishy: when is it happening?
[23:08:34] <madhuvishy>	 nuria: when i try to run scripts/test tests/test_models/test_validate_cohort.py
[23:08:50] <madhuvishy>	 grep -R "blah_testing" . gives nothing
[23:10:16] <nuria>	 madhuvishy: ahhh, but that is a "fake" error
[23:10:30] <madhuvishy>	 nuria: ohh
[23:10:30] <nuria>	 madhuvishy: a control exception for tests, right?
[23:10:32] <nuria>	  let me see
[23:11:58] <nuria>	 madhuvishy: ya, that is ehat it is, take a look at :  def test_validate_cohorts_when_invalid_project_causes_exception(
[23:12:07] <nuria>	 madhuvishy: and how mocks work on python
[23:12:40] <madhuvishy>	 nuria: aah
[23:15:59] <madhuvishy>	 nuria: did not know that
[23:16:05] <madhuvishy>	 it's raised as a warning
[23:16:19] <nuria>	 madhuvishy: ya, which is fine
[23:16:25] <madhuvishy>	 nuria: alright
[23:16:48] <nuria>	 madhuvishy: test is testing that that exception is caught and things are proceeding
[23:16:56] <madhuvishy>	 right
[23:17:13] <nuria>	 madhuvishy: so it doesn't turn out into an uncaught exception that terminates your program
[23:17:16] <madhuvishy>	 makes sense. i thought may be mocks but din't think it would call it blah_testing
[23:17:37] <nuria>	 madhuvishy: i probably did that I am the WORST with naming
[23:19:09] * YuviPanda pitches https://wikitech.wikimedia.org/wiki/We_suck_at_naming_and_you_probably_do,_too
[23:28:23] <nuria>	 YuviPanda: jaja
[23:29:38] <nuria>	 madhuvishy, YuviPanda ; i used to work with this guy that was quoting always the famous: " the two hardest things in cs are 1) cache eviction and 2) naming"
[23:30:18] <nuria>	 http://martinfowler.com/bliki/TwoHardThings.html
[23:30:47] <madhuvishy>	 nuria: so my theory is that https://phabricator.wikimedia.org/T78339 is not always true. Validate again works fine when you upload cohorts which are validated as user names
[23:31:09] <madhuvishy>	 it fails when you uploaded cohorts of user ids
[23:31:23] <nuria>	 madhuvishy: that sounds VERY possible
[23:31:24] <madhuvishy>	 it works the first time
[23:31:28] <madhuvishy>	 but after
[23:31:45] <madhuvishy>	 somehow it forgets to validated them as user ids may be
[23:31:49] <madhuvishy>	 or sth
[23:31:58] <madhuvishy>	 but that's where it's going wrong
[23:32:25] <madhuvishy>	 there are no tests for "cohorts with user ids" at the moment too