[05:12:05] <grrrit-wm>	 (CR) Gergő Tisza: [C: -1] "This will include users who are inactive for years, so the ratio is not very meaningful. Also I feel uncomfortable about using tsv files a" [analytics/multimedia] - https://gerrit.wikimedia.org/r/145595 (owner: MarkTraceur)
[05:16:41] <grrrit-wm>	 (CR) Gergő Tisza: [C: 2] "Nice!" [analytics/multimedia] - https://gerrit.wikimedia.org/r/145608 (owner: MarkTraceur)
[05:16:46] <grrrit-wm>	 (Merged) jenkins-bot: Use preference change events instead of our own [analytics/multimedia] - https://gerrit.wikimedia.org/r/145608 (owner: MarkTraceur)
[08:37:02] <grrrit-wm>	 (CR) Gergő Tisza: Add graphs for opt-outs (1 comment) [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/145599 (owner: MarkTraceur)
[13:19:25] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] Add basic deployment script [analytics/refinery] - https://gerrit.wikimedia.org/r/144677 (https://bugzilla.wikimedia.org/67129) (owner: QChris)
[13:20:13] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] Rename scripts in bin to start in 'refinery-' [analytics/refinery] - https://gerrit.wikimedia.org/r/145978 (owner: QChris)
[13:34:34] <ottomata>	 nuria_:  i thought talk was wed!~?
[13:35:02] <ottomata>	 GAHHHHH
[13:35:04] <ottomata>	 TUESDAY
[13:35:05] <ottomata>	 GAAHHH
[13:35:07] <ottomata>	 haha
[13:35:09] <ottomata>	 oh well
[13:35:11] <ottomata>	 can make it happen
[13:35:16] <ottomata>	 i should not have said I could do it this week
[13:35:17] <ottomata>	 gah
[13:35:53] <ottomata>	 can I put a snarky slide in that says "the cluster is not yet back online because I had to prepare this slide" :p
[13:35:54] <ottomata>	 ?
[13:35:54] <ottomata>	 haha
[13:38:47] <ottomata>	 milimetric: you there?  want a pre standup kafka consume instructional?
[13:50:32] <milimetric>	 hi ottomata
[13:50:59] <milimetric>	 yea, sure
[13:51:47] <ottomata>	 k, in the trap
[13:51:51] <ottomata>	 or hm, naw lets do batcave
[13:52:40] <ottomata>	 HMMM
[13:53:05] <ottomata>	 actually this might be more difficult than I thought, have to install kafka client, or kafkacat
[14:29:13] <milimetric>	 Ironholds / kevinator / DarTar: it turns out consuming stuff out of Kafka is a harder than we thought and if it's not necessary, we'd like to skip it
[14:29:17] <milimetric>	 so, would sampled logs do?
[14:29:47] <milimetric>	 is it just the ease of using Hive that was attractive?  If so, I can load the sampled logs for that period into mysql or something
[14:35:39] <nuria_>	 hello aharoni
[15:22:36] <grrrit-wm>	 (CR) MarkTraceur: "We should chat about what medium you'd prefer - I'm guessing this is why you wanted another database table, but that sounds nasty and unne" [analytics/multimedia] - https://gerrit.wikimedia.org/r/145595 (owner: MarkTraceur)
[15:23:20] <marktraceur>	 Hey y'all, I'm gonna bug you about a screm of screms thing for a sec
[15:23:25] <marktraceur>	 https://wikimedia.mingle.thoughtworks.com/projects/scrum_of_scrums/cards/126
[15:24:00] <marktraceur>	 tgr wants write access for a new table, basically, to have historic numbers for opt-outs to media viewer
[15:24:41] <marktraceur>	 Pretty much just counting the number of people with the preferences set every day, based on a filter (so I think EL is out of the picture), and saving it to a table. Then we can dump a subset of the table each day for our dataset.
[15:25:36] <marktraceur>	 Preferably you could give us read/write access to that table with a group username and password
[15:25:48] <marktraceur>	 (so it doesn't have to be one of us running the queries)
[16:10:58] <YuviPanda>	 marktraceur: can't you already get that data out of the PrefUpdate schema or something?
[16:12:14] <marktraceur>	 YuviPanda: Not with the matching for edit counts, activity, etc....I think
[16:12:19] <YuviPanda>	 ah, right
[17:17:07] <grrrit-wm>	 (CR) Gergő Tisza: "The table exists already, it's staging.mediaviewer_optout. As it turns out the research user has the right to create tables in the 'stagin" [analytics/multimedia] - https://gerrit.wikimedia.org/r/145595 (owner: MarkTraceur)
[17:26:15] <marktraceur>	 wtf http://multimedia-metrics.wmflabs.org/dashboards/mmv
[17:26:23] <marktraceur>	 502 bad gateway
[17:26:44] <marktraceur>	 Now totally down
[17:26:58] <marktraceur>	 NVM, labs thing
[17:28:32] <milimetric>	 wtf retracted?
[17:28:33] <milimetric>	 :)
[17:28:43] <marktraceur>	 Yeah for now
[17:28:44] <marktraceur>	 Or at least, redirected
[17:29:03] <marktraceur>	 Yay, 'tis back.
[17:30:01] <milimetric>	 also - marktraceur, we answered the table question last week on the lists and in IRC with gergo
[17:30:07] <milimetric>	 let us know if you still have trouble with it though
[17:30:13] <marktraceur>	 Oh, sure
[17:30:13] <milimetric>	 looks like Gergo commented
[17:30:19] <marktraceur>	 It...yeah, he did
[17:30:27] <marktraceur>	 I'm waiting to see if he wants it or if I should take it
[18:00:12] <grrrit-wm>	 (PS1) Ottomata: Add camus wrapper script and camus.webrequest.properties [analytics/refinery] - https://gerrit.wikimedia.org/r/146143
[18:23:23] <kevinator>	 hey milimetric ! ut?
[18:36:57] <milimetric>	 hi kevinator, yea
[18:37:10] <milimetric>	 troubled by python, but here
[18:39:27] <Ironholds>	 have you tried 'from carefree import all'?
[18:39:30] <Ironholds>	 that normally works for me.
[18:39:48] <Ironholds>	 warning: it's a python library so it's got no documentation, it's inconsistently debianised, and it may randomly break other code.
[18:41:17] <terrrydactyl>	 lol
[18:41:26] <terrrydactyl>	 i need a library like that
[18:41:49] <terrrydactyl>	 also def get_your_shit_together would be helpful
[18:41:57] <kevinator>	 milimetric: I wanted to clarify something we were talking about earlier today at the scrum…  I think I was looking at scrumbugz while you were looking at the spreadsheet
[18:42:23] <milimetric>	 ok
[18:42:51] <kevinator>	 I want to pull https://bugzilla.wikimedia.org/show_bug.cgi?id=66843 into this sprint (with 0 points) so we can track it… and I know it will probably go into the next sprint.
[18:43:34] <milimetric>	 ok kevinator, so that's something that Teresa will be working on mostly
[18:43:45] <kevinator>	 yup, it’s all hers
[18:43:47] <milimetric>	 therefore it doesn't need to be in scrumbugz or the spreadsheet
[18:44:05] <Ironholds>	 terrrydactyl, really you want a generalised get_it_together method you can apply to objects in the your_shit class.
[18:44:29] <Ironholds>	 because then we can also apply it to, like, Sandra Bullock and Hugh Grant in any film ever about 20 minutes in and not have to sit through the remaining hour and a half.
[18:44:40] <Ironholds>	 your_relationship.get_it_together(). Okay, we can all go home now! Saved!
[18:45:11] <terrrydactyl>	 hahaha
[18:45:15] <grrrit-wm>	 (PS1) Gergő Tisza: Fix action logging SQL query [analytics/multimedia] - https://gerrit.wikimedia.org/r/146158
[18:48:34] <kevinator>	 milimetric: I get that it doesn’t need to be there, and I want it there for convenience so I only have to look at scrumbugs when it’s time to summarize what was completed for a showcase.
[18:50:03] <milimetric>	 kevinator: but scrumbugz is a place where we prioritize and organize what our immediate team is working on.  Having community work there kind of confuses the purpose
[18:52:43] <milimetric>	 we can do simple queries in bugzilla to see everything that has happened in the analytics universe
[18:53:28] <milimetric>	 kevinator: I respect your request though, and understand what you mean - maybe you can write a message to -internal and ask what everyone else thinks?
[18:53:45] <milimetric>	 i won't stand in the way if everyone else agrees
[18:55:25] <kevinator>	 I guess so.  I was hoping to make obsolete Toby’s Vi macro to formatting bugzilla into a slide
[18:56:01] <kevinator>	 I don’t want to make a big fuss over it
[18:56:38] <kevinator>	 anyway, I recall I’m also supposed to send out an email with our original commitments made on Thursday.
[18:56:52] <kevinator>	 I’ll send that out to the public list shortly
[18:57:40] <grrrit-wm>	 (CR) MarkTraceur: [C: 2] Fix action logging SQL query [analytics/multimedia] - https://gerrit.wikimedia.org/r/146158 (owner: Gergő Tisza)
[18:57:45] <grrrit-wm>	 (Merged) jenkins-bot: Fix action logging SQL query [analytics/multimedia] - https://gerrit.wikimedia.org/r/146158 (owner: Gergő Tisza)
[19:02:01] <milimetric>	 kevinator: well, the sprint demo is about what our team accomplishes
[19:02:14] <milimetric>	 we can mention the community but I don't think we should prominently take credit for what they get done
[19:20:29] <milimetric>	 oh Ironholds, you around?
[19:20:49] <milimetric>	 I had pinged you guys about the kafka data earlier:
[19:21:06] <milimetric>	 and I see kevin wrote an email
[19:21:12] <milimetric>	 so - if you want to talk it over, let's
[19:50:00] <Ironholds>	 milimetric, I am now!
[19:50:18] <milimetric>	 k
[19:50:24] <milimetric>	 offer stands
[19:56:44] <milimetric>	 doh, Ironholds you replied to the email
[19:56:49] <milimetric>	 !! sorry
[19:57:02] <Ironholds>	 np!
[19:57:06] <milimetric>	 ok, so the sampled logs don't get hit by the packetloss after all, if I understand correctly
[19:57:40] <milimetric>	 only webstatscollector does
[20:09:02] <Ironholds>	 milimetric, oh, cool!
[20:09:06] <Ironholds>	 then yeah, this'll be a fun writeup :D
[20:10:29] <milimetric>	 Ironholds: I wish ottomata or qchris were around to give you their guarantee that no data was lost there
[20:10:45] <milimetric>	 I feel a little uneasy understanding all the puppet code that would let me do the same
[20:11:08] <milimetric>	 but to the best of my knowledge, yeah, you're ok to assume no loss there
[20:11:15] <Ironholds>	 cool :). We can do some fun stuff with that.
[20:11:25] <Ironholds>	 I will in fact extract the data now because I need it for some work leila and I are doing
[20:13:45] <ottomata>	 hiii?
[20:15:06] <ottomata>	 milimetric: , what's up?
[20:15:18] <milimetric>	 oh, glad you're here
[20:15:22] <milimetric>	 so - the sampled logs
[20:15:37] <milimetric>	 they don't have the data loss that would have happened due to the UDP packet loss right?
[20:15:49] <milimetric>	 on oxygen and an1003?
[20:17:34] <ottomata>	 nope, cause they come from erbium
[20:17:55] <ottomata>	 the sampled logs do not have nginx (ssl) requests in them though, but htey should have the proxied requests there just fine (meaning the IPs are wrong on ssl reqs)
[20:17:58] <milimetric>	 Ironholds: there you go ^ from the masters of the domain
[20:18:27] <milimetric>	 thanks ottomata!
[20:24:12] <Ironholds>	 milimetric, danke :)
[20:24:22] <Ironholds>	 and danke ottomata!
[20:25:05] <ottomata>	 yup!
[22:04:33] <grrrit-wm>	 (PS1) Milimetric: Avoid pickle max recursion while serializing chain [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/146297 (https://bugzilla.wikimedia.org/67823)
[22:15:31] <milimetric>	 nite!