[00:44:11] <Ironholds>	 ottomata, filled up stat2 again (very briefly). My apologies.
[00:44:16] <Ironholds>	 Not /entirely/ sure how but I'll work it out ;p
[03:02:23] <grrrit-wm>	 (PS1) Nuria: [WIP] Adding removing options from browsing component [analytics/dashiki] - https://gerrit.wikimedia.org/r/157777
[07:53:48] <wikibugs>	 Analytics / Tech community metrics: "Volume of open changesets" graph should show reviews pending every month - https://bugzilla.wikimedia.org/70278 (Quim Gil) NEW p:Unprio s:major a:None I just realized that "Volume of open changesets" at http://korma.wmflabs.org/browser/gerrit_review_queu...
[07:54:47] <wikibugs>	 Analytics / Tech community metrics: "Volume of open changesets" graph should show reviews pending every month - https://bugzilla.wikimedia.org/70278 (Quim Gil) p:Unprio>High
[13:17:49] <wikibugs>	 Analytics / General/Unknown: Webstatscollector's pagecounts file shaky between 2014-08-24 14:00 and 2014-08-27 21:00 - https://bugzilla.wikimedia.org/70118#c4 (christian) NEW>RESO/FIX (In reply to christian from comment #0) > Closer investigation of the files between 20140824 14:00 and > 2014-08-...
[14:24:27] <grrrit-wm>	 (PS2) Milimetric: [WIP] Adding removing options from browsing component [analytics/dashiki] - https://gerrit.wikimedia.org/r/157777 (owner: Nuria)
[14:25:05] <milimetric>	 nuria: https://gerrit.wikimedia.org/r/#/c/157777/ is a little more stylled
[14:32:17] <jgage>	 hi. hangout link please? it's not in the calendar event and nobody's in the one from the standup.
[14:32:29] <qchris_meeting>	 jgage: http://goo.gl/1pm5JI
[14:32:32] <jgage>	 thanks
[14:39:48] <grrrit-wm>	 (PS3) Milimetric: [WIP] Adding removing options from browsing component [analytics/dashiki] - https://gerrit.wikimedia.org/r/157777 (owner: Nuria)
[14:53:58] <tgr>	 is there a way to tell when a PrefUpdate event actually means a change in preference?
[14:54:09] <tgr>	 as in, the new and the old value is different
[14:54:59] <tgr>	 at least for the prefs in the beta features tab, the table seems to be full of disabled -> disabled "change" events
[15:47:29] <grrrit-wm>	 (PS1) BearND: Add app edit reports back to mobile report card [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/157822
[15:55:24] <bearND>	 yuvipanda: would you CR ^?
[15:58:15] <qchris>	 ottomata: Are you in deployment mood?
[15:58:21] <ottomata>	 yes siiiir i am
[15:58:28] <qchris>	 Wohooo :-D
[15:58:40] <ottomata>	 oh, and this is really just a filter restart, right?
[15:58:50] <qchris>	 Yes.
[15:58:54] <ottomata>	 i mean, we should upgrade collector too, but we don't need to restart it
[15:59:26] <qchris>	 Did anything change on the collector at all since the last deployment? (Let me check...)
[16:00:28] <qchris>	 I could not find changes to the collector process.
[16:01:02] <qchris>	 But I guess it's easier to package if one rebuilds both.
[16:01:35] <ottomata>	 _joe_, you don't want to ask for other opinions? :)
[16:01:38] <ottomata>	 oops
[16:01:40] <ottomata>	 wrong chat
[16:02:15] <ottomata>	 yes, i have to rebuild both, i've already updated apt with the new version
[16:02:17] <ottomata>	 0.3-1
[16:02:24] <ottomata>	 will upgrade filter now...
[16:04:11] <milimetric>	 kevinator: https://metrics-staging.wmflabs.org/static/public/datafiles/
[16:04:25] <kevinator>	 :-)
[16:04:28] <kevinator>	 beat you to it
[16:04:30] <kevinator>	 https://docs.google.com/a/wikimedia.org/presentation/d/1r1wlp7yfdT0i-FRMcYNd2uN8ndbvu4Z_a3scpvKi3PU/edit#slide=id.g3902d8a7e_02
[16:04:32] <milimetric>	 :)
[16:04:33] <kevinator>	 It’s already there
[16:04:45] <milimetric>	 good stuff, well it's running with qchris's replag stuff
[16:04:47] <milimetric>	 and working well
[16:05:52] <ottomata>	 ha, qchris, filter version 0.1 is currently running on oxygen, version 0.2 collector on gaodlinium
[16:06:45] <qchris>	 0.1? Really?
[16:07:08] <qchris>	 Hopefully the version number is just an artifact.
[16:07:40] <grrrit-wm>	 (PS1) Ottomata: Bumping version number with recent source change [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/157825
[16:08:37] <ottomata>	 hmm, yeah, ew
[16:08:38] <ottomata>	 hm
[16:08:49] <grrrit-wm>	 (CR) QChris: [C: 2 V: 2] Bumping version number with recent source change [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/157825 (owner: Ottomata)
[16:09:02] <ottomata>	 qchris, can you tell ifyour  Special:CentralAutoLogin change you made back in december is running in filter?
[16:09:03] <ottomata>	 based on output?
[16:09:14] <ottomata>	 i guess if dumps doesn't have any of those pages counted...
[16:09:24] <qchris>	 Yes. Wanted to test that :-) But then I wanted to CR your change.
[16:09:37] <ottomata>	 this does not look good!
[16:09:39] <ottomata>	 -rwxr-xr-x 1 root root 10392 Apr  3  2013 /usr/local/bin/filter
[16:09:56] <qchris>	 It might be thath you deployed by hand before.
[16:10:10] <qchris>	 Version numbers changed severeal times back then
[16:10:13] <ottomata>	 well, that's the timestamp on oxygen where filter is running
[16:10:29] <ottomata>	 your change was dec 2013,
[16:11:09] <ottomata>	 qchris, just checked, that change is not live :/
[16:11:25] <ottomata>	 i see Special:CentralAutoLogin counts in recent page dumps
[16:12:00] <qchris>	 Mhmm.
[16:12:08] <ottomata>	 ja and, Apr 3 2013 was the previous source change before that dec 2013 change
[16:13:16] <ottomata>	 well poopers, sorry about that, looks like whenever we deployed that change I deployed it on gadolinium only....qchris, maybe filter was running on a different host at that time?
[16:13:24] <ottomata>	 and we upgraded there, but then moved filter to oxygen and never upgraded it?
[16:14:00] <qchris>	 We moved it somewhere around spring, because ssl was not fed into it.
[16:16:34] <ottomata>	 ha, it was on erbium then
[16:16:35] <ottomata>	 hm ok
[16:16:42] <ottomata>	 well, i guess...are you ok for that change to come back to life?
[16:16:57] <qchris>	 Yes :-D Please.
[16:17:25] <qchris>	 That affects data from 2014-07-08 onwards.
[16:17:44] <qchris>	 Well .... 2014-07-07
[16:18:03] <grrrit-wm>	 (PS2) Milimetric: Add dashiki config generation [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/157761
[16:19:59] <ottomata>	 hokay!, upgraded. udp2log restarted on oxygen
[16:20:02] <qchris>	 Thanks!
[16:20:10] <ottomata>	 can't wait to compare that data in a couple of hours!
[16:20:46] <qchris>	 Yup. Me too.
[16:21:41] <qchris>	 Hopefully the drop is in line with what I computed the last few days.
[16:24:20] <wikibugs>	 Analytics / General/Unknown: Webstatscollector counting CentralAutoLogin requests again since 2014-07-07 - https://bugzilla.wikimedia.org/70295 (christian) NEW p:Unprio s:normal a:None When migrating webstatscollector's filter process from erbium to oxygen on 2014-07-07 for bug 67456, it se...
[16:25:34] <wikibugs>	 Analytics / General/Unknown: Webstatscollector counting CentralAutoLogin requests again since 2014-07-07 - https://bugzilla.wikimedia.org/70295#c1 (christian) A current binary was deployed just now. Thanks ottomata!  CentralAutoLogin requests should vanish soon again.
[16:25:48] <wikibugs>	 Analytics / Wikistats: Page Views counts too high for wikistats since July 2013 - https://bugzilla.wikimedia.org/57980 (christian)
[17:12:30] <nuria__>	 ottomata: do you want to take a look at a scala code test we received? I think it has some lacks but just get  a second opinion
[17:30:15] <grrrit-wm>	 (CR) Yuvipanda: [C: 2] Add app edit reports back to mobile report card [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/157822 (owner: BearND)
[17:30:20] <grrrit-wm>	 (Merged) jenkins-bot: Add app edit reports back to mobile report card [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/157822 (owner: BearND)
[17:43:29] <ottomata>	 nuria__: sorry, just saw your message
[17:43:53] <ottomata>	 i can look at it, but I am hardly a scala expert
[17:43:53] <ottomata>	 nuria, i would like to look at it, but i'm not sure how much help I can be reviewing it
[17:45:23] <kevinator>	 Ottomata, qchris: you guys could play rock paper scisors to pick who will talk about webstats collector at the showcase
[17:46:03] <ottomata>	 we could play Muk jji ppa
[17:46:53] <ottomata>	 qchris: are we just talking about the ulsfo change?
[17:47:39] <nuria__>	 ottomata: please take a look: http://pastebin.com/qDFA2Dvx
[17:47:53] <qchris>	 I planned to show the switch to tmpfs
[17:47:57] <qchris>	 (only in numbers)
[17:48:16] <nuria__>	 ottomata: Overall there is no doc, no tests and overusage of synchronized i think, but let me know what you think.
[17:48:18] <qchris>	 And briefly exhibit the service.
[17:48:29] <ottomata>	 oh ok
[17:48:35] <nuria__>	 going to 1st day of school, will be back in 1hr 30 min
[17:48:38] <ottomata>	 qchris: that is better
[17:49:03] <qchris>	 ottomata: ok.
[17:49:17] <Ironholds>	 hey, ottomata!
[17:49:17] <halfak>	 Hey ottomata, Ironholds and I are giving issues with bast1001/stat2/stat3.
[17:49:17] <ottomata>	 yoyo
[17:49:18] <halfak>	 Who is the right person to bother?
[17:49:23] <Ironholds>	 really slow handshakes and then periodic drops in responsiveness.
[17:49:39] <ottomata>	 hm
[17:49:39] <halfak>	 ^ oh yes.  those kind of problems.  Networky ones.
[17:49:39] <qchris>	 halfak, Ironholds: same here.
[17:49:53] <ottomata>	 like, in your ssh connections?
[17:50:11] <halfak>	 My ssh connections get dropped once per 5 minutes.
[17:50:20] <Ironholds>	 yup
[17:50:29] <halfak>	 But ping is OK (usually 17ms, now 45ms)
[17:50:31] <Ironholds>	 I've been alternating between dropping f-bombs and blessing screen for the last hour
[17:51:19] <ottomata>	 halfak, Ironholds, are you in different locations?
[17:51:22] <ottomata>	 oh, qchris gets it too
[17:51:26] <halfak>	 Yup.
[17:51:27] <Ironholds>	 yup
[17:51:41] <halfak>	 MN, MA & Austria
[17:51:44] <Ironholds>	 so we have Boston, Minneapolis, and [LOCATION HIDDEN CONTACT YOUR LOCAL CIA FIELD OFFICE]
[17:51:52] <halfak>	 Woops
[17:52:10] <Ironholds>	 halfak, it's okay, you'll get a free trip to a caribbean island on your own plane!
[17:52:23] <Ironholds>	 anyway, yeah. No geographic link I guess
[17:52:38] * halfak is taken away by a man that has been hiding in his closet for just such an occasion. 
[17:52:46] <Ironholds>	 baha.
[17:53:03] <ottomata>	 qchris, you will talk about the tmpfs change then?
[17:53:58] <ottomata>	 Ironholds: halfak: trying to replicate...
[17:55:11] <qchris>	 ottomata: Sure.
[17:55:26] <qchris>	 If you prefer to present, I'll gladly let you do it :-)
[17:56:39] <ottomata>	 naw you got it :)
[17:56:47] <qchris>	 :-(
[17:57:28] <Ironholds>	 haha
[17:57:41] <Ironholds>	 into the valley of death, rode the...dude and his beard. Not as poetic.
[18:12:28] <wikibugs>	 Analytics / Tech community metrics: "Volume of open changesets" graph should show reviews pending every month - https://bugzilla.wikimedia.org/70278#c1 (Alvaro) NEW>ASSI a:Alvaro Volume of open changesets is showing the amount of changesets that were "Pending" and "Waiting for review" in a spe...
[18:14:24] * yuvipanda waves at milimetric and nuria__ from across hangouts
[18:14:42] <milimetric>	 ha yuvipanda
[18:19:19] <ottomata>	 nice job qchris_meeting!
[18:21:25] <qchris_meeting>	 Hahaha :-) Beat me with sarcasm :-P
[18:21:32] <qchris_meeting>	 Next time you get to present again.
[18:22:37] <ottomata>	 no that was great!
[18:22:46] <ottomata>	 you had diagrams and everythign!
[18:22:58] <milimetric>	 qchris_meeting: totally awesome
[18:28:01] <halfak>	 so, ottomata about that issue with connections to bast1001, who should I bother?
[18:29:37] <ottomata>	 hm, i have not yet been kicked
[18:29:54] <qchris_meeting>	 It also affected downloading from dumps.wikimedia.org, or using ganglia.wikimedia.org
[18:30:03] * halfak just got disconnected again
[18:30:11] <qchris_meeting>	 It's probably more than bast1001.
[18:30:32] <ottomata>	 Sep  2 18:30:14 bast1001 sshd[19463]: Received disconnect from 75.72.203.28: 11: disconnected by user
[18:30:32] <ottomata>	 that's you halfak
[18:30:32] <ottomata>	 hm
[18:30:51] <ottomata>	 qchris_meeting: there's a problem with ganglia?
[18:31:02] <qchris_meeting>	 ottomata: Nono.
[18:31:34] <qchris_meeting>	 ottomata: The slowness from my ssh connections carried over to other non-ssh properties of wmf.
[18:31:54] <halfak>	 ottomata, I have my client set to disconnect if it waits for 5 seconds or more.
[18:32:30] <halfak>	 Oh wait... maybe I don't
[18:35:34] <ottomata>	 this is weird guys, it works for me™
[18:35:49] <ottomata>	 i'm going through bast1001 now, and connections are fine and i'm not getting kicked
[18:37:07] <qchris>	 For me, disconnects happen after some time of inactivity (ssh's keep-alive option did not help for me). If I leave "top" or something the like running, I do not see disconnects.
[18:37:43] <qchris>	 But since a few days, the network connections to wmf are choking a bit.
[18:37:59] <ottomata>	 and qchris, you are saying it is not just ssh  bast1001 connections
[18:38:01] <qchris>	 ssh connections are blocking for 10 seconds, then working fine again.
[18:38:28] <qchris>	 downloads from dumps.wikimedia.org sleep for a few seconds.
[18:38:42] <qchris>	 Then recover. Then sleep again. And so on.
[18:38:44] <qchris>	 ottomata: yes.
[18:38:44] <ottomata>	 hm
[18:40:14] <qchris>	 I monitored ifconfig output on bastions I use, and machines I connect to. But no bursts are visible (to me) from wmf side.
[18:40:14] <qchris>	 So it's not like someone is copying huge files and starving others' connections.
[18:40:20] <qchris>	 s/copying/scp-ing/
[18:40:55] <qchris>	 And since it also affects non-ssh connections, it seems to be a more general issue.
[18:41:04] <qchris>	 Is there any test I can do from my side?
[18:42:44] <qchris>	 When connections to wmf sites are choking, connections to other (non-wmf) sites work normally, and are not choking.
[18:43:23] <qchris>	 I could not find any regularity/pattern up to now.
[18:45:58] <ottomata>	 qchris: halfak, i'm not sure....
[18:46:59] <ottomata>	 bd808 says he is having similar problems
[18:47:42] <qchris>	 I guess I'll just keep on taking notes when it gets better/worse.
[18:47:49] <qchris>	 Maybe at some point it'll align.
[18:47:58] <ottomata>	 ok
[18:48:08] <ottomata>	 halfak: i'm really not sure, you could submit an RT ticket
[18:50:17] <halfak>	 OK.  Will do.  Thanks for looking ottomata
[18:51:42] <wikibugs>	 Analytics / Visualization: Story: AnalyticsEng has website for EEVS - https://bugzilla.wikimedia.org/68351 (Dan Andreescu) NEW>RESO/FIX
[18:54:48] <qchris>	 Thanks ottomata!
[19:04:13] <wikibugs>	 Analytics / General/Unknown: Webstatscollector counting CentralAutoLogin requests again since 2014-07-07 - https://bugzilla.wikimedia.org/70295#c2 (christian) NEW>RESO/FIX The deployment worked, and Special:CentralAutoLogin/ requests are gone again.  The first affected file is   https://dumps.wik...
[19:11:12] <ottomata>	 :)
[19:19:01] <ottomata>	 Ironholds: just a prototype!
[19:19:02] <ottomata>	 https://gist.github.com/ottomata/d29dba9d04ec786b2d41
[19:20:00] <Ironholds>	 oh, that's cool as all hell :D
[19:20:34] <ottomata>	 (did I do the query right?  I am no sql expert)
[19:21:07] <Ironholds>	 cannot see any obvious problem although I've never seen people put conditionals in ON statements before :D
[19:21:15] <Ironholds>	 but I've only been coding for <2 years, so.
[19:27:47] <marktraceur>	 DarTar, tnegrin - could one of you ask lzia to join us on the eye argh seas?
[19:28:05] <tnegrin>	 she is on vacation
[19:28:05] <marktraceur>	 ...IRC. It sounds like IRC.
[19:28:07] <marktraceur>	 Ahhh.
[19:28:15] <marktraceur>	 Her cloak request is all bogged down in silliness.
[19:28:21] <marktraceur>	 I'll email her instead
[19:28:30] <tnegrin>	 thanks
[19:34:42] <wikibugs>	 Analytics / Tech community metrics: "Volume of open changesets" graph should show reviews pending every month - https://bugzilla.wikimedia.org/70278#c2 (Quim Gil) (In reply to Alvaro from comment #1) >  > This metric is implemented aggregating the number of pending reviews that > remains pending at the...
[20:01:14] <wikibugs>	 Analytics / Tech community metrics: Wrong data at "Update time for pending reviews waiting for reviewer in days" - https://bugzilla.wikimedia.org/68436#c11 (Alvaro) Great, we will remove it tomorrow and it will disappear from the dashboard.  Thanks!
[20:03:13] <wikibugs>	 Analytics / Tech community metrics: "Volume of open changesets" graph should show reviews pending every month - https://bugzilla.wikimedia.org/70278#c3 (Alvaro) Ok Quim, you are right.   Right now the metric is showing the backlog of the current open issues.  Let us think more about the metric you are...
[20:16:28] <wikibugs>	 Analytics / General/Unknown: Webstatscollector counting CentralAutoLogin requests again since 2014-07-07 - https://bugzilla.wikimedia.org/70295#c4 (christian) PATC>RESO/FIX Setting status to fixed again, as the issue at hand has been fixed already. The change from comment #3 just helps to prevent...
[20:33:12] <grrrit-wm>	 (PS4) Milimetric: [WIP] Adding removing options from browsing component [analytics/dashiki] - https://gerrit.wikimedia.org/r/157777 (owner: Nuria)
[20:33:27] <wikibugs>	 Analytics / Tech community metrics: "Volume of open changesets" graph should show reviews pending every month - https://bugzilla.wikimedia.org/70278#c4 (Alvaro) (In reply to Alvaro from comment #3) > Ok Quim, you are right.   Yes, you are looking for a related but different metric.  Taking a look to th...
[20:52:10] <ottomata>	 Ironholds: i'm having fun with this sqoop stuff, the cool thing is you can do it too!
[20:52:15] <ottomata>	 you need no special privs
[20:52:22] <ottomata>	 you've got db access and you have hadoop access
[20:52:27] <ottomata>	 so you can run sqoop to do whatever you want
[20:52:40] <ottomata>	 we should probalby set up some automatic import of some things
[20:52:57] <ottomata>	 but in lieu of that,  you can import anything you want into a hive table
[20:54:16] <ottomata>	 steps are basically:
[20:54:16] <ottomata>	 - create a hive table (in your database)
[20:54:16] <ottomata>	 - write mysql query that selects data from your table
[20:54:16] <ottomata>	 - run sqoop command with that query (and some extra options)
[20:54:28] <ottomata>	 that's pretty much it, sqoop will launch a mapreduce job and select all of your data into a hive table
[20:54:35] <ottomata>	 then you can  join whatever you want with webrequest data
[20:55:12] <wikibugs>	 Analytics / Tech community metrics: "Volume of open changesets" graph should show reviews pending every month - https://bugzilla.wikimedia.org/70278#c5 (Quim Gil) I don't understand the confusion, and I don't want to make you work before assuring that we are oin the same page. "Number of pending submis...
[20:55:42] <Ironholds>	 ottomata, cool!
[20:55:50] <Ironholds>	 I'm currently arguing with sql, but maybe later :D
[20:57:18] <ottomata>	 ja no probs, i think you will like it
[21:29:30] <grrrit-wm>	 (CR) Nuria: "Looks good, small comment. I also think we need to create the static list of pretty names for projects and make that part of this changese" (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/157761 (owner: Milimetric)
[22:15:51] <grrrit-wm>	 (PS1) Milimetric: Fix Visualization of multiple lines [analytics/dashiki] - https://gerrit.wikimedia.org/r/158004
[23:04:44] <grrrit-wm>	 (PS5) Nuria: [WIP] Adding removing options from browsing component [analytics/dashiki] - https://gerrit.wikimedia.org/r/157777
[23:08:29] <grrrit-wm>	 (CR) Nuria: "I have fixed issues with build and css and icons. I think the convention of 'anything under css directory gets bundled up' is preferable" [analytics/dashiki] - https://gerrit.wikimedia.org/r/157777 (owner: Nuria)