[00:51:05] <awight|mtg>	 nuria: have u seen this bug? https://bugzilla.wikimedia.org/show_bug.cgi?id=38604
[03:31:28] <wikibugs>	 Analytics / General/Unknown: Requests for 'undefined' page increasing on wikipedias - https://bugzilla.wikimedia.org/66352#c16 (nuria) Looked at centranotice campaigns today to find whether there was a campaign scheduled on May and September where we can see the event going on. We found no common campa...
[04:05:13] <wikibugs>	 Analytics / General/Unknown: Requests for 'undefined' page increasing on wikipedias - https://bugzilla.wikimedia.org/66352#c17 (nuria) Looking at data for october in detail, there seems to be a bunch of repeats on referrers.
[04:17:14] <wikibugs>	 Analytics / General/Unknown: Requests for 'undefined' page increasing on wikipedias - https://bugzilla.wikimedia.org/66352#c18 (nuria) Referers for "wiki/undefined" pageviews for 09/10   Odd, they should be more widespread.       18 http://es.wikipedia.org/wiki/Historia_del_Sahara_Occidental      21 ht...
[04:17:28] <wikibugs>	 Analytics / General/Unknown: Requests for 'undefined' page increasing on wikipedias - https://bugzilla.wikimedia.org/66352#c19 (nuria) IPs are not widespread either.
[04:32:28] <wikibugs>	 Analytics / General/Unknown: Requests for 'undefined' page increasing on wikipedias - https://bugzilla.wikimedia.org/66352#c20 (nuria) Now, check this bug from two years ago: https://bugzilla.wikimedia.org/show_bug.cgi?id=38604
[09:26:53] <Nemo_bis>	 hashar: what can be done to help https://bugzilla.wikimedia.org/show_bug.cgi?id=65478 progress?
[09:27:16] <Nemo_bis>	 I'd also like something about captcha failures but I'm not sure that's anywhere in graphite
[09:37:44] <hashar>	 Nemo_bis: that bug follow an incident report iirc
[09:37:58] <hashar>	 Nemo_bis: we have different rate limiter which are logged using wfProfileIn() to graphite
[09:38:11] <hashar>	 so we have some new metrics that we could graph on gdash.wikimedia.org
[09:40:44] <Nemo_bis>	 hashar: yes, but for people without access to graphite it's hard to guess the format
[09:40:58] <Nemo_bis>	 so I have no idea how to help, but I'd like to help
[09:45:50] <hashar>	 Nemo_bis: I have updated the bug report with some informations
[09:45:58] <hashar>	 MediaWiki.User.pingLimiter-*.count  would do
[09:49:46] <Nemo_bis>	 nice
[09:49:48] <Nemo_bis>	 thanks
[11:42:46] <wikibugs>	 Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages every now and then - https://bugzilla.wikimedia.org/69667 (christian)
[11:42:47] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-08T23:xx:xx not marked successful - https://bugzilla.wikimedia.org/71876 (christian) NEW p:Unprio s:normal a:None For the hour 2014-10-08T23:xx:xx, bits, text, and upload [1] were not marked successful.  What happened?    [1] _____...
[11:43:15] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-08T23:xx:xx not marked successful - https://bugzilla.wikimedia.org/71876#c1 (christian) NEW>RESO/WON For bits and upload we saw both duplicates and missing. For mobile we only saw duplicates. The affected period was 23:02:00 -- 23:11:00...
[13:33:01] <qchris>	 !log Marked bits, text, and upload webrequest partitions for 2014-10-08T23:xx:xx ok {{bug|71876}}
[13:41:04] <hashar>	 message not logged master.
[13:41:23] * hashar lamely attempts to confuse coworkers
[14:00:16] <wikibugs>	 Analytics / General/Unknown: Udp2log missing parts of ulsfo traffic between 2014-10-08T22:00:00 and 2014-10-08T24:00:00 - https://bugzilla.wikimedia.org/71879 (christian) NEW p:Unprio s:normal a:None Created attachment 16732   --> https://bugzilla.wikimedia.org/attachment.cgi?id=16732&actio...
[14:00:59] <wikibugs>	 Analytics / General/Unknown: Udp2log missing parts of ulsfo traffic between 2014-10-08T22:00:00 and 2014-10-08T24:00:00 - https://bugzilla.wikimedia.org/71879#c1 (christian) Created attachment 16733   --> https://bugzilla.wikimedia.org/attachment.cgi?id=16733&action=edit Number of lines from ulsfo per...
[14:01:14] <wikibugs>	 Analytics / General/Unknown: Udp2log missing parts of ulsfo traffic between 2014-10-08T22:00:00 and 2014-10-08T24:00:00 - https://bugzilla.wikimedia.org/71879 (christian)
[14:01:15] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-08T23:xx:xx not marked successful - https://bugzilla.wikimedia.org/71876 (christian)
[14:01:30] <wikibugs>	 Analytics / General/Unknown: Udp2log missing parts of ulsfo traffic between 2014-10-08T22:00:00 and 2014-10-08T24:00:00 - https://bugzilla.wikimedia.org/71879 (christian) NEW>RESO/WON
[14:31:22] <grrrit-wm>	 (PS1) QChris: Stop counting 301, 302, 303 HTTP status codes [analytics/refinery] - https://gerrit.wikimedia.org/r/165725 (https://bugzilla.wikimedia.org/71790)
[14:32:06] <tnegrin>	 hey qchris
[14:32:13] <qchris>	 Hi tnegrin
[14:32:33] <tnegrin>	 would you have a few minutes to talk about Erik Z’s email from yesterday?
[14:32:45] <qchris>	 1 sec.
[14:32:48] <tnegrin>	 np
[14:33:12] <qchris>	 Booting the google machine and jumping into the hangout then...
[14:34:06] <tnegrin>	 we can do it on IRC if you want
[14:34:50] <qchris>	 Either way is fine by me.
[14:34:53] <qchris>	 I am in the hangout already
[14:35:53] <tnegrin>	 ok — I’ll go to the hangout
[14:47:14] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-08T23:xx:xx not marked successful - https://bugzilla.wikimedia.org/71876 (christian)
[14:47:14] <wikibugs>	 Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages every now and then - https://bugzilla.wikimedia.org/69667 (christian)
[14:47:29] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions that were not marked successful - https://bugzilla.wikimedia.org/70085 (christian)
[14:50:59] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions that were not marked successful - https://bugzilla.wikimedia.org/70085 (christian)
[14:51:00] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-08T1[89]:xx:xx not marked successful - https://bugzilla.wikimedia.org/71881 (christian) NEW p:Unprio s:normal a:None The two bits partitions [1] on 2014-10-08T1[89]:xx:xx, were not marked successful.  What happened?   [1] _________...
[14:55:59] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-08T1[89]:xx:xx not marked successful - https://bugzilla.wikimedia.org/71881#c1 (christian) Since checking missings across hour boundaries does not make the missing go away [1], it does not seem to be the race condition described in bug 69615....
[15:02:59] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions that were not marked successful - https://bugzilla.wikimedia.org/70085 (christian)
[15:03:00] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-07T1[789]:xx:xx not marked successful - https://bugzilla.wikimedia.org/71882 (christian) NEW p:Unprio s:normal a:None The three bits partitions [1] on 2014-10-07T1[789]:xx:xx, were not marked successful.  What happened?   [1] _____...
[15:10:44] <wikibugs>	 Analytics / Refinery: Duplicates/missing logs from esams bits for 2014-09-28T{18,19,20}:xx:xx - https://bugzilla.wikimedia.org/71435 (christian)
[15:10:44] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-07T1[789]:xx:xx not marked successful - https://bugzilla.wikimedia.org/71882#c1 (christian) Since checking missings across hour boundaries does not make the missing go away [1], it does not seem to be the neighbored race condition described in...
[15:11:14] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions for 2014-10-08T1[89]:xx:xx not marked successful - https://bugzilla.wikimedia.org/71881 (christian)
[15:11:14] <wikibugs>	 Analytics / Refinery: Duplicates/missing logs from esams bits for 2014-09-28T{18,19,20}:xx:xx - https://bugzilla.wikimedia.org/71435 (christian)
[15:41:27] <grrrit-wm>	 (PS1) QChris: [webstatscollector] Add condition to not count redirects [analytics/metrics] - https://gerrit.wikimedia.org/r/165748 (https://bugzilla.wikimedia.org/71790)
[16:09:47] <grrrit-wm>	 (CR) Ottomata: [C: 2] Stop counting 301, 302, 303 HTTP status codes [analytics/refinery] - https://gerrit.wikimedia.org/r/165725 (https://bugzilla.wikimedia.org/71790) (owner: QChris)
[16:12:34] <grrrit-wm>	 (CR) Nemo bis: "Unrelated question (this graph is useful!)." (1 comment) [analytics/metrics] - https://gerrit.wikimedia.org/r/99078 (owner: QChris)
[16:15:06] <ottomata>	 nuria_: dunno what you are backfilling, could it be done in hadoop if we sqoop stuff?
[16:30:01] <Ironholds>	 ottomata, ping
[16:30:18] <Ironholds>	 in talks with MaxMind; could you grab me the actual reporting name (171,177,whatever) for NetSpeedCell.dat?
[16:41:03] <nuria_>	 ottomata: maybe but the basic problem is repeated computations not scale
[16:41:35] <nuria_>	 ottomata: thus an "intermediate" table solves teh issue. this is editor data (small) not readers data(large)
[16:41:44] <nuria_>	 *the
[17:04:43] <wikibugs>	 Analytics / General/Unknown: Requests for 'undefined' page increasing on wikipedias - https://bugzilla.wikimedia.org/66352#c21 (nuria) Regarding UA (as Christian noted earlier) windows is the winner:  101 Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Sa...
[17:41:33] <milimetric>	 halfak: https://github.com/wikimedia/analytics-refinery/blob/master/hive/webrequest/select_missing_sequence_runs.hql#L47
[18:09:26] <grrrit-wm>	 (PS1) Mforns: Remove invalid user count in cohort detail [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/165766
[18:18:30] <grrrit-wm>	 (PS2) Nuria: Remove invalid user count in cohort detail [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/165766 (owner: Mforns)
[18:23:42] <grrrit-wm>	 (PS3) Mforns: Remove invalid user count in cohort detail [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/165766
[18:25:39] <grrrit-wm>	 (CR) Nuria: [C: 2] Remove invalid user count in cohort detail [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/165766 (owner: Mforns)
[18:47:45] <wikibugs>	 Analytics / Wikimetrics: Cohort validation: text is confusing "0 invalid" - https://bugzilla.wikimedia.org/71842 (Marcel Ruiz Forns) ASSI>PATC
[18:48:59] <wikibugs>	 Analytics / Wikimetrics: Cohort validation: text is confusing "0 invalid" - https://bugzilla.wikimedia.org/71842#c1 (Marcel Ruiz Forns) https://gerrit.wikimedia.org/r/#/c/165766/
[18:50:43] <wikibugs>	 Analytics / Wikimetrics: add endpoint to show invalid users as html page - https://bugzilla.wikimedia.org/60628#c2 (nuria) Invalid user names are also displaying in unicode chars rather than plain text.
[20:31:15] <wikibugs>	 Analytics / Wikimetrics: add endpoint to show invalid users as html page - https://bugzilla.wikimedia.org/60628 (Marcel Ruiz Forns) NEW>ASSI a:Marcel Ruiz Forns
[21:16:05] <grrrit-wm>	 (PS2) QChris: Enable optimizations at level 2 [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165630
[21:16:07] <grrrit-wm>	 (PS2) QChris: Stop counting 301, 302, 303 HTTP status codes [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165631 (https://bugzilla.wikimedia.org/71790)
[21:16:09] <grrrit-wm>	 (PS5) QChris: Release fix that stops counting [uU]ndefined and redirects [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165351 (https://bugzilla.wikimedia.org/66352)
[21:16:11] <grrrit-wm>	 (PS1) QChris: Allow array to hold full logline's keytext [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165887
[21:16:13] <grrrit-wm>	 (PS1) QChris: Make collector object file depend on include file [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165888
[21:16:15] <grrrit-wm>	 (PS1) QChris: Add tests for collector [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165889
[21:27:35] <qchris>	 ottomata: Do you feel like discussing the webstatscollector patches and maybe deploying a new version?
[21:31:59] <ottomata>	 qchris: , soon, how much longer will you be up and working?
[21:32:15] <qchris>	 If you want to .... $X hours.
[21:33:05] <qchris>	 2 hours?
[21:34:14] <ottomata>	 i am opinionating (rare, I know) in #wikimedia-office with new VP technology office hours :p
[21:34:20] <ottomata>	 and I want to write something up on wikihadoop for aaron
[21:34:33] <ottomata>	 qchris: what in the patches do we need to discuss?
[21:34:38] <ottomata>	 oh, there is that one...
[21:34:43] <ottomata>	 oh the fink one
[21:34:47] <ottomata>	 we can discuss that now
[21:34:56] <qchris>	 No worries, let's have the chat in -office.
[21:35:08] <qchris>	 And afterwards we can maybe sync up?
[21:35:30] <ottomata>	 naw, i can do both, just not build and deploy while chatting
[21:35:56] <qchris>	 Ok.
[21:36:26] <qchris>	 I don't need the paths there
[21:36:44] <qchris>	 But since they were in the Makefile, I thought I could keep them around
[21:37:05] <ottomata>	 naw, toss 'em
[21:37:11] <ottomata>	 we aren't going to build webstatscollector for mac anyway :p
[21:37:20] <ottomata>	 aside from that, I think I +2ed all the patches
[21:37:38] <qchris>	 Ok. I'll rip the paths out then.
[21:37:52] <qchris>	 There are a few missing.
[21:38:29] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165626/
[21:38:33] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165627/
[21:38:38] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165629/
[21:38:42] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165630/
[21:38:48] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165631/
[21:38:52] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165887/
[21:38:57] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165887/
[21:39:00] <grrrit-wm>	 (CR) Ottomata: [C: 2] Adapt type of list of group ids to what the setgroups function expects [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165626 (owner: QChris)
[21:39:12] <grrrit-wm>	 (CR) Ottomata: [C: 2] Specify return value for main function [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165627 (owner: QChris)
[21:39:27] <grrrit-wm>	 (CR) Ottomata: [C: 2] Enable optimizations at level 2 [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165630 (owner: QChris)
[21:39:31] <qchris>	 But I guess 1-2 of them are from just now.
[21:39:57] <grrrit-wm>	 (CR) Ottomata: [C: 2] Stop reading past the end of the array of whitelisted mediawiki wikis [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165629 (owner: QChris)
[21:40:30] <ottomata>	 qchris: why
[21:40:34] <ottomata>	 1499?
[21:40:43] <ottomata>	 in https://gerrit.wikimedia.org/r/#/c/165887/
[21:40:44] <ottomata>	 ?
[21:40:48] <qchris>	 Because that's the value used it snprintf
[21:40:51] <ottomata>	 oh ok
[21:40:52] <ottomata>	 cool
[21:41:08] <ottomata>	 cool see it.
[21:41:18] <grrrit-wm>	 (CR) Ottomata: [C: 2] Allow array to hold full logline's keytext [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165887 (owner: QChris)
[21:41:19] <qchris>	 line 158
[21:43:13] <grrrit-wm>	 (CR) Ottomata: [C: 2] Make collector object file depend on include file [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165888 (owner: QChris)
[21:43:48] <grrrit-wm>	 (CR) Ottomata: [C: 2] Stop counting 301, 302, 303 HTTP status codes [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165631 (https://bugzilla.wikimedia.org/71790) (owner: QChris)
[21:43:57] <grrrit-wm>	 (PS2) QChris: Allow array to hold full logline's keytext [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165887
[21:43:59] <grrrit-wm>	 (PS2) QChris: Remove unused variables [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165624
[21:44:01] <grrrit-wm>	 (PS2) QChris: Provide definition of setgroups function [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165625
[21:44:03] <grrrit-wm>	 (PS2) QChris: Adapt type of list of group ids to what the setgroups function expects [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165626
[21:44:05] <grrrit-wm>	 (PS2) QChris: Specify return value for main function [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165627
[21:44:07] <grrrit-wm>	 (PS2) QChris: Turn finalizer of projects list into proper list [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165628
[21:44:09] <grrrit-wm>	 (PS2) QChris: Stop reading past the end of the array of whitelisted mediawiki wikis [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165629
[21:44:11] <grrrit-wm>	 (PS3) QChris: Enable optimizations at level 2 [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165630
[21:44:13] <grrrit-wm>	 (PS3) QChris: Stop counting 301, 302, 303 HTTP status codes [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165631 (https://bugzilla.wikimedia.org/71790)
[21:44:15] <grrrit-wm>	 (PS6) QChris: Release fix that stops counting [uU]ndefined and redirects [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165351 (https://bugzilla.wikimedia.org/66352)
[21:44:17] <grrrit-wm>	 (PS2) QChris: Make collector object file depend on include file [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165888
[21:44:19] <grrrit-wm>	 (PS2) QChris: Add tests for collector [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165889
[21:44:21] <grrrit-wm>	 (PS2) QChris: Drop MacOS paths [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165619
[21:44:23] <grrrit-wm>	 (PS2) QChris: Remove unneeded commented out linking of libdb [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165620
[21:44:25] <grrrit-wm>	 (PS2) QChris: Drop unneeded dependencies from rules [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165621
[21:44:27] <grrrit-wm>	 (PS2) QChris: Use GNU Make's implicit rules to build object and executable files [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165622
[21:44:29] <grrrit-wm>	 (PS2) QChris: Provide definitions to system level functions [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165623
[21:45:26] <grrrit-wm>	 (CR) QChris: "Community did not veto" [analytics/metrics] - https://gerrit.wikimedia.org/r/165395 (https://bugzilla.wikimedia.org/66352) (owner: QChris)
[21:45:47] <grrrit-wm>	 (CR) QChris: "Community did not veto" [analytics/refinery] - https://gerrit.wikimedia.org/r/165378 (https://bugzilla.wikimedia.org/66352) (owner: QChris)
[21:46:13] <grrrit-wm>	 (CR) QChris: "Community did not veto" [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165351 (https://bugzilla.wikimedia.org/66352) (owner: QChris)
[21:52:35] <ottomata>	 oh qchris
[21:52:36] <ottomata>	 what's this about?
[21:52:38] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/165889/2/collector.c
[21:52:45] <ottomata>	 produceDump?
[21:52:59] <qchris>	 Yup.
[21:53:19] <qchris>	 And I was too scared to throw in a second strcmp in the main iteration.
[21:53:35] <qchris>	 So I hid it in the if ... =='-' check.
[21:53:59] <qchris>	 But we don't need that change right now.
[21:54:08] <qchris>	 We can do without tests for the collector.
[21:58:59] <ottomata>	 is that new?
[21:59:28] <ottomata>	 qchris: ^?
[21:59:42] <qchris>	 the -produceDump? That is new.
[21:59:53] <qchris>	 I need it to force a dump, so the tests have something to check.
[21:59:55] <ottomata>	 for testing?
[21:59:57] <ottomata>	 ahhh
[21:59:58] <ottomata>	 ok cool
[22:00:06] <qchris>	 Otherwise, we'd have to wait for the full hour.
[22:00:22] <qchris>	 ... 3 times ;-)
[22:00:24] <grrrit-wm>	 (CR) Ottomata: [C: 2] Add tests for collector [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165889 (owner: QChris)
[22:00:30] <ottomata>	 aye
[22:00:32] <ottomata>	 cool
[22:00:37] <ottomata>	 ok, did I miss any?
[22:00:44] <ottomata>	 i haven't merged, because you -2ed a dep
[22:00:48] <ottomata>	 oh, did you rebase that?
[22:00:55] <qchris>	 Yup.
[22:01:04] <qchris>	 But gerrit is smart enough to keep CR+2
[22:01:11] <grrrit-wm>	 (CR) Ottomata: [C: 2] Drop MacOS paths [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165619 (owner: QChris)
[22:01:19] <grrrit-wm>	 (Merged) jenkins-bot: Remove unneeded commented out linking of libdb [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165620 (owner: QChris)
[22:01:21] <grrrit-wm>	 (Merged) jenkins-bot: Drop unneeded dependencies from rules [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165621 (owner: QChris)
[22:01:23] <grrrit-wm>	 (Merged) jenkins-bot: Use GNU Make's implicit rules to build object and executable files [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165622 (owner: QChris)
[22:01:25] <grrrit-wm>	 (Merged) jenkins-bot: Provide definitions to system level functions [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165623 (owner: QChris)
[22:01:27] <grrrit-wm>	 (Merged) jenkins-bot: Remove unused variables [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165624 (owner: QChris)
[22:01:29] <qchris>	 There goes jenkins :-)
[22:01:29] <grrrit-wm>	 (Merged) jenkins-bot: Provide definition of setgroups function [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165625 (owner: QChris)
[22:01:31] <grrrit-wm>	 (Merged) jenkins-bot: Adapt type of list of group ids to what the setgroups function expects [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165626 (owner: QChris)
[22:01:34] <grrrit-wm>	 (Merged) jenkins-bot: Specify return value for main function [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165627 (owner: QChris)
[22:01:35] <grrrit-wm>	 (Merged) jenkins-bot: Turn finalizer of projects list into proper list [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165628 (owner: QChris)
[22:01:37] <grrrit-wm>	 (Merged) jenkins-bot: Stop reading past the end of the array of whitelisted mediawiki wikis [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165629 (owner: QChris)
[22:01:40] <grrrit-wm>	 (Merged) jenkins-bot: Allow array to hold full logline's keytext [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165887 (owner: QChris)
[22:01:40] <ottomata>	 whoa
[22:01:40] <ottomata>	 ha
[22:01:41] <grrrit-wm>	 (Merged) jenkins-bot: Make collector object file depend on include file [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165888 (owner: QChris)
[22:01:43] <ottomata>	 i merged the firs one
[22:01:43] <ottomata>	 haha
[22:01:43] <grrrit-wm>	 (Merged) jenkins-bot: Enable optimizations at level 2 [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165630 (owner: QChris)
[22:01:44] <ottomata>	 cool!
[22:01:45] <grrrit-wm>	 (Merged) jenkins-bot: Stop counting 301, 302, 303 HTTP status codes [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165631 (https://bugzilla.wikimedia.org/71790) (owner: QChris)
[22:02:12] <qchris>	 https://gerrit.wikimedia.org/r/#/c/165351/
[22:02:14] <grrrit-wm>	 (CR) Ottomata: [C: 2] Release fix that stops counting [uU]ndefined and redirects [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165351 (https://bugzilla.wikimedia.org/66352) (owner: QChris)
[22:02:18] <qchris>	 Ah. Ok.
[22:02:27] <grrrit-wm>	 (Merged) jenkins-bot: Add tests for collector [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165889 (owner: QChris)
[22:02:40] <ottomata>	 ok!
[22:02:45] <qchris>	 Yay!
[22:02:47] <ottomata>	 qchris, can you build deb, or do I need to?
[22:03:07] <qchris>	 Never done thaht for wmf.
[22:03:25] <qchris>	 Which machine do you typically build on?
[22:04:14] <ottomata>	 i use a vm
[22:04:15] <ottomata>	 i can do it.
[22:04:23] <qchris>	 That would be great.
[22:04:26] <qchris>	 Thanks.
[22:13:16] <ottomata>	 hm, how do I run tests?
[22:13:47] <qchris>	 make check
[22:14:06] <ottomata>	 ok yeah, getting som efailed tests
[22:14:14] <qchris>	 The collector ones?
[22:14:40] <ottomata>	 ja
[22:15:00] <ottomata>	 FAIL! (Pagecounts files do not match)
[22:15:00] <ottomata>	 Test #64: collector-tests/0002-Simple project ... --- dumps/pagecounts-*	1970-01-01 00:00:00.000000000 +0000
[22:15:14] <qchris>	 Currently, they use a timeout to "guess" when the collector is done with the export.
[22:15:19] <qchris>	 That might be too small
[22:15:41] <ottomata>	 yeah it look slike it is just showing the expected ones
[22:15:52] <ottomata>	 +_ lines
[22:16:42] <ottomata>	 make hte sleep longer and try?
[22:17:00] <grrrit-wm>	 (PS1) QChris: Use more conservative timeouts for tests [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165906
[22:17:10] <qchris>	 That change ^ should do the trick
[22:17:13] <ottomata>	 k
[22:17:58] <ottomata>	 hmm, still happening
[22:18:01] <ottomata>	 but more slowly :p
[22:18:06] <ottomata>	   # of failed tests : 3 (first failed test is #63: 'collector-tests/0001-Simple line')
[22:18:15] <ottomata>	 Test #63: collector-tests/0001-Simple line ... --- dumps/pagecounts-*	1970-01-01 00:00:00.000000000 +0000
[22:18:15] <ottomata>	 +++ /home/vagrant/webstatscollector/tests/collector-tests/0001-Simple line/expected.pagecounts	2014-10-09 22:05:51.137888001 +0000
[22:18:15] <ottomata>	 @@ -0,0 +1 @@
[22:18:15] <ottomata>	 +en Foo 1 100
[22:18:15] <ottomata>	 FAIL! (Pagecounts files do not match)
[22:18:19] <qchris>	 So ... all tests failed?
[22:18:21] <Ironholds>	 ottomata, lemme know when you have a free 30s? (literally should take 30s ;p)
[22:18:21] <ottomata>	 no
[22:18:22] <ottomata>	 just 3
[22:18:27] <ottomata>	 Ironholds: tell me!
[22:18:32] <qchris>	 Yup. 3 is all collector tests.
[22:18:36] <ottomata>	 yes
[22:18:38] <qchris>	 Mhmm.
[22:18:46] <Ironholds>	 ottomata, NetSpeedCell.dat: what product ID is that?171,177?
[22:18:47] <ottomata>	 and they all only have + lines in the diff
[22:18:52] <ottomata>	 oh
[22:18:55] <ottomata>	 yeah you asked that somewhere...
[22:19:28] <qchris>	 Tests pass for my machine and jenkins :-/
[22:19:33] <ottomata>	 hm ok
[22:19:45] <qchris>	 Do you use bash?
[22:20:02] <qchris>	 Or is that symlinked to a different shell for you?
[22:20:15] <ottomata>	 yes bash
[22:20:42] <ottomata>	 Ironholds: 171 is without Celluarl
[22:20:45] <ottomata>	 177 is with Cellular
[22:21:04] <ottomata>	 so Cell shoudl be with Cellular, 177.
[22:21:10] <ottomata>	 GeoIPNetSpeedCell.dat
[22:21:13] <ottomata>	 i assume!
[22:22:12] <qchris>	 Can you try to start the collector, then
[22:22:46] <qchris>	 echo some lines to udp 127.0.0.1 port 3815
[22:23:08] <qchris>	 echo a "-produceDump" line to that port, and see what the collector says?
[22:24:21] <Ironholds>	 ottomata, ta
[22:24:24] <qchris>	 Meh ... maybe the collector is even more broken ...?
[22:24:38] <qchris>	 Could you try Making the -O2 into a -O0 in the main Makefile?
[22:24:46] <qchris>	 and then try "make check" again?
[22:25:31] <ottomata>	 qchris: first, doing what you say makes collector just output pagecount output
[22:25:40] <ottomata>	 actually, any netcat to that port does that.
[22:25:42] <ottomata>	 hmmm
[22:26:08] <ottomata>	 qchris: -O0 has the same test failures
[22:26:23] <qchris>	 Glad that -O0 does not help.
[22:28:04] <qchris>	 Meh ... the tests pass on labs too.
[22:28:17] <qchris>	 Any idea how I colud reproduce?
[22:28:41] <ottomata>	 hm
[22:29:32] <qchris>	 So thinking about it again ... every netcat to the port makes collector output something?
[22:30:28] <qchris>	 That sounds weird.
[22:30:58] <qchris>	 collector should only output something when it's told to dump or when it's $X:00:00.
[22:31:14] <qchris>	 Stdout should by empty while it's collecting.
[22:31:28] <ottomata>	 hmm, i wonder if this is aVM thing
[22:31:36] <ottomata>	 ls /dev/udp
[22:31:36] <ottomata>	 ls: cannot access /dev/udp: No such file or directory
[22:31:38] <ottomata>	 qchris^
[22:32:04] <qchris>	  /dev/udp is bash magic
[22:32:12] <qchris>	 Bash intercepts it
[22:32:28] <qchris>	 It is not exported through udev, devfs, ...
[22:33:05] <qchris>	 Can you try
[22:33:06] <qchris>	 echo "-produceDump" >/dev/udp/127.0.0.1/3815
[22:33:34] <qchris>	 If that does not show an error, bash should correctly intercept it
[22:33:50] <ottomata>	 no error
[22:33:54] <ottomata>	 tha'ts bash magic!?
[22:33:56] <ottomata>	 weiiird!
[22:34:16] <qchris>	 Ja. It's a nice way to pipe to the network :-D
[22:34:19] <ottomata>	 oh
[22:34:24] <ottomata>	 as non root it does error
[22:34:32] <ottomata>	 ah
[22:34:33] <ottomata>	 nope
[22:34:33] <ottomata>	 sorry
[22:34:34] <ottomata>	 typo.
[22:34:35] <ottomata>	 :)
[22:34:42] <qchris>	 :-D
[22:35:01] <qchris>	 Could you try removing the "&>/dev/null" in
[22:35:12] <qchris>	 test/tests.sh:150
[22:35:25] <qchris>	 Then the tests should spit out some more output
[22:35:54] <qchris>	 Like ".temp.db: unable to flush page: 1" twice per test (for me)
[22:36:22] <ottomata>	 nuthin.
[22:36:23] <qchris>	 (We saw that output also on analytics1003, so it's nothing new)
[22:36:32] <qchris>	 Mhmmm...
[22:36:48] <ottomata>	 hm, qchris, its ok, probably my build env is crazy messed up
[22:36:51] <ottomata>	 i can build on labs
[22:36:53] <qchris>	 Do you hove something else listening on udp port 3815?
[22:36:53] <ottomata>	 what instance you using?
[22:36:59] <qchris>	 qchris-worker2
[22:37:08] <ottomata>	 OH
[22:37:10] <ottomata>	 i think I might!
[22:37:12] <ottomata>	 have a 3815er
[22:37:37] <ottomata>	 test pass now!
[22:37:38] <ottomata>	 duuhHH
[22:37:38] <qchris>	 Code on qchris-worker2 is at /home/qchris/webstatscollector
[22:37:45] <qchris>	 Yay!
[22:37:51] <ottomata>	 ja service webstats-collector was runnig
[22:37:51] <ottomata>	 !
[22:37:58] <ottomata>	 probably from a preivous deb check
[22:38:06] <qchris>	 Sadly the collector port cannot be configured  :-(
[22:39:21] <qchris>	 I guess I should make the tests check whether or not the port is unused, so they can give a better error message
[22:43:41] <ottomata>	 hm, qchris, it does seem that your timeout increase did help thouhg
[22:43:44] <ottomata>	 tests pass with that
[22:43:45] <ottomata>	 not on master head
[22:43:47] <ottomata>	 so I will merge that
[22:44:00] <grrrit-wm>	 (CR) Ottomata: [C: 2] Use more conservative timeouts for tests [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165906 (owner: QChris)
[22:46:40] <ottomata>	 cool ok, built ok
[22:46:49] <ottomata>	 qchris: should we deploy this today...or tomorrow? :)
[22:46:54] <qchris>	 Awesome!
[22:47:05] <qchris>	 Tomorrow is friday.
[22:47:14] <ottomata>	 yes, and today is close to end of day thursday...:)
[22:47:20] <qchris>	 Hahahaha.
[22:47:23] <qchris>	 Ok. Monday?
[22:47:24] <ottomata>	 should we run it on analytics1003 for a while?
[22:47:33] <ottomata>	 monday is holiday, and I am taking tuesday and friday off next week :/
[22:47:52] <qchris>	 Meh. Let's not deploy then. Deployment gods are againts it.
[22:48:19] <qchris>	 Honestly, I have no preference.
[22:48:29] <qchris>	 Whenevery you think is a good time, we deploy.
[22:48:59] <ottomata>	 HAHAHHAhahah
[22:49:03] <ottomata>	 TYPICAL christian
[22:49:04] <ottomata>	 love it
[22:49:04] <ottomata>	 ok!
[22:49:38] <ottomata>	 time for a snack and wikihadoop
[22:49:56] <qchris>	 Any plan when we deploy (I should send an update to the lists)
[22:57:39] <ottomata>	 wednesday?
[22:57:50] <ottomata>	 unless, you want to get another opsen to help ya?
[22:57:53] <ottomata>	 i can put it in apt
[23:00:45] <ottomata>	 ah we changed to ensure=> latest though :)
[23:14:15] <qchris>	 Wednesday is fine.
[23:35:51] <grrrit-wm>	 (PS1) QChris: Use a proper template for tests' temprorary directory [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165931
[23:35:59] <grrrit-wm>	 (PS1) QChris: Cleanup trailing whitespace [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165932
[23:36:08] <grrrit-wm>	 (PS1) QChris: Abort tests if UDP port 3815 is not available for collector [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165933
[23:36:18] <grrrit-wm>	 (PS1) QChris: Capture and check collector's stdout and stderr during tests [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165934
[23:47:00] <ottomata>	 ah, qchris, one of the reasons we might want push permissions on analytics repos
[23:47:02] <ottomata>	 the original push!
[23:47:09] <ottomata>	 i want to fork wikihadoop into gerrit now
[23:47:11] <ottomata>	 but i need to push
[23:47:16] <ottomata>	 all the original stuff
[23:47:31] <qchris>	 Yup. I set that up for the repos we do that.
[23:47:40] <qchris>	 wikihadoop ... that is new :-)
[23:47:49] <qchris>	 Let me set it up there ...
[23:48:11] <ottomata>	 oh i'm looking at it now
[23:48:26] <ottomata>	 do I need to change a permission temporaily?  or do I need to push differently
[23:48:31] <ottomata>	 i'm getting an author mismatch error
[23:48:32] <ottomata>	 right now
[23:48:45] <ottomata>	 committer email address whym@whym.org
[23:48:45] <ottomata>	 does not match your user account.
[23:48:45] <qchris>	 How often do you want to push?
[23:48:48] <ottomata>	 jsut initial
[23:49:06] <ottomata>	 OR, if upstream makes changes
[23:49:07] <ottomata>	 i'd push those
[23:49:10] <ottomata>	 but I don't expect that to happen
[23:49:17] <qchris>	 Ok. Then temporary would not help much.
[23:49:22] <qchris>	 Ah.
[23:49:26] <qchris>	 Ok. Then temporary.
[23:49:30] <ottomata>	 that does happen with camus though :)
[23:50:10] <grrrit-wm>	 (CR) Ottomata: [C: 2] Use a proper template for tests' temprorary directory [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165931 (owner: QChris)
[23:50:17] <grrrit-wm>	 (CR) Ottomata: [V: 2] Use a proper template for tests' temprorary directory [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165931 (owner: QChris)
[23:50:22] <qchris>	 Try again to push.
[23:50:26] <qchris>	 Does it work now?
[23:50:35] <ottomata>	 nope, same error
[23:50:50] <ottomata>	 git push gerrit master
[23:50:52] <ottomata>	 is all i'm doing
[23:51:26] <grrrit-wm>	 (PS2) Ottomata: Cleanup trailing whitespace [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165932 (owner: QChris)
[23:51:32] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] Cleanup trailing whitespace [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165932 (owner: QChris)
[23:51:48] <grrrit-wm>	 (PS2) Ottomata: Abort tests if UDP port 3815 is not available for collector [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165933 (owner: QChris)
[23:51:53] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] Abort tests if UDP port 3815 is not available for collector [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165933 (owner: QChris)
[23:52:16] <grrrit-wm>	 (PS2) Ottomata: Capture and check collector's stdout and stderr during tests [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165934 (owner: QChris)
[23:52:22] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] Capture and check collector's stdout and stderr during tests [analytics/webstatscollector] - https://gerrit.wikimedia.org/r/165934 (owner: QChris)
[23:52:35] <ottomata>	 qchris: push does not work, still same author error
[23:52:54] <qchris>	 What is the repo you want to import?
[23:53:23] <ottomata>	 https://github.com/whym/wikihadoop
[23:53:33] <qchris>	 Ok. I'll try to clone and push
[23:53:35] <ottomata>	 k
[23:54:41] <ottomata>	 k
[23:56:16] <qchris>	 ottomata: Done.
[23:56:36] <ottomata>	 danke
[23:58:28] <qchris>	 Btw ... to get someone into the wmf ldap group ... that needs an RT ticket. Right?
[23:58:39] <qchris>	 ottomata: ^