[01:09:42] (PS5) Yuvipanda: Store results in SQLite databases [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154333 [01:29:36] (PS6) Yuvipanda: Store results in SQLite databases [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154333 [02:34:36] (PS7) Yuvipanda: Store results in SQLite databases [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154333 [02:42:53] (PS8) Yuvipanda: Store results in SQLite databases [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154333 [11:10:27] Analytics / EventLogging: EventLogging qunit test failure in ?debug=1 mode - https://bugzilla.wikimedia.org/66100#c3 (Andre Klapper) PATC>RESO/FIX All patches mentioned in this report were merged - assuming this bug is FIXED. If that is not the case: Please reopen and elaborate what is left to do... [11:14:27] Analytics / Wikimetrics: Recurrent reports disappear after 30 days - https://bugzilla.wikimedia.org/67030#c3 (Andre Klapper) PATC>RESO/FIX Patch mentioned in this report was merged - assuming this bug is FIXED. If that is not the case: Please reopen and elaborate what is left to do here to get th... [11:42:44] nuria ... you there under a different nick? [12:01:15] Analytics / General/Unknown: Logging filling up disk on oxygen - https://bugzilla.wikimedia.org/69661#c2 (christian) Created attachment 16218 --> https://bugzilla.wikimedia.org/attachment.cgi?id=16218&action=edit Maximum-disk-space-used-oxygen-2014-08-07 [12:04:13] Analytics / General/Unknown: Logging filling up disk on oxygen - https://bugzilla.wikimedia.org/69661#c3 (christian) Since noon 2014-08-16, the disk space on root of oxygen started to fill up (see attachment Maximum-disk-space-used-oxygen-2014-08-07): qchris@oxygen:~$ df -h Filesystem Si... [12:48:46] (CR) Yuvipanda: [C: 2] Store results in SQLite databases [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154333 (owner: Yuvipanda) [12:48:52] (Merged) jenkins-bot: Store results in SQLite databases [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154333 (owner: Yuvipanda) [13:11:45] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663 (christian) NEW p:Unprio s:normal a:None We had an packetloss alert on oxygen: [23:03:09] PROBLEM - Packetloss_Average on oxygen is CRITICAL: packet_loss_average CRI... [13:12:28] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663 (christian) [13:13:14] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663 (christian) [13:13:14] Analytics / General/Unknown: Logging filling up disk on oxygen - https://bugzilla.wikimedia.org/69661 (christian) [13:19:43] Analytics / General/Unknown: Logging filling up disk on oxygen - https://bugzilla.wikimedia.org/69661#c5 (christian) PATC>RESO/FIX ottomata also cleaned up the udp2log log file on oxygen, and oxygen is all green again in icinga. Thanks ottomata! [15:09:32] (PS1) Yuvipanda: Fix typo, calling state as status [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154466 [15:09:55] (CR) Yuvipanda: [C: 2] Fix typo, calling state as status [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154466 (owner: Yuvipanda) [15:10:00] (Merged) jenkins-bot: Fix typo, calling state as status [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154466 (owner: Yuvipanda) [15:31:17] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-13 ~10:00 - https://bugzilla.wikimedia.org/69665 (christian) NEW p:Unprio s:normal a:None Created attachment 16219 --> https://bugzilla.wikimedia.org/attachment.cgi?id=16219&action=edit Produce-Re... [15:31:43] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-13 ~10:00 - https://bugzilla.wikimedia.org/69665#c1 (christian) Created attachment 16220 --> https://bugzilla.wikimedia.org/attachment.cgi?id=16220&action=edit BrokerTopicMetrics-MessagesInPerSec [15:32:58] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-13 ~10:00 - https://bugzilla.wikimedia.org/69665#c2 (christian) NEW>RESO/FIX This was fixed by ottomata on 2014-08-13 ~14:00 [15:37:45] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-16 ~07:30 - https://bugzilla.wikimedia.org/69666 (christian) NEW p:Unprio s:normal a:None Created attachment 16221 --> https://bugzilla.wikimedia.org/attachment.cgi?id=16221&action=edit Produce-Re... [15:38:58] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-16 ~07:30 - https://bugzilla.wikimedia.org/69666#c1 (christian) Created attachment 16222 --> https://bugzilla.wikimedia.org/attachment.cgi?id=16222&action=edit BrokerTopicMetrics-MessagesInPerSec [15:41:23] (PS1) Yuvipanda: Use a cron based query killer instead of celery based one [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154470 [15:41:28] (CR) jenkins-bot: [V: -1] Use a cron based query killer instead of celery based one [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154470 (owner: Yuvipanda) [15:43:02] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-06 ~1:44 - https://bugzilla.wikimedia.org/69244 (christian) [15:43:02] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages every now and then - https://bugzilla.wikimedia.org/69667 (christian) NEW p:Unprio s:normal a:None Every few days, analytics1021 gets kicked out of its partition leader role. The last few occurrences are: 2014-0... [15:43:02] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-13 ~10:00 - https://bugzilla.wikimedia.org/69665 (christian) [15:43:03] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages since 2014-08-16 ~07:30 - https://bugzilla.wikimedia.org/69666 (christian) [15:43:11] (PS2) Yuvipanda: Use a cron based query killer instead of celery based one [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154470 [15:43:21] (CR) Yuvipanda: [C: 2] Use a cron based query killer instead of celery based one [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154470 (owner: Yuvipanda) [15:43:27] (Merged) jenkins-bot: Use a cron based query killer instead of celery based one [analytics/quarry/web] - https://gerrit.wikimedia.org/r/154470 (owner: Yuvipanda) [15:46:28] Analytics / General/Unknown: Kafka broker analytics1021 not receiving messages every now and then - https://bugzilla.wikimedia.org/69667#c1 (Toby Negrin) Thanks for bugging this Christian. IMHO, this has been a fairly regular problem with this host and attempts to reimage/fix have not been successful.... [17:40:46] (PS1) AndyRussG: Move get_cohort_by_name() to CohortService [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154479 [17:46:20] (CR) AndyRussG: "This change is the first of two that will replace Change-Id I398992f6503cc47552e3ea8b45dedac0de23f971, as per Milimetric's comment on PS6 " [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154479 (owner: AndyRussG) [19:02:08] (PS1) AndyRussG: Add validators module and use it in cohort_upload [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154482 [19:05:33] (CR) AndyRussG: "This change is the second of two that replace Change-Id I398992f6503cc47552e3ea8b45dedac0de23f971, as per Milimetric's comment on PS6 of t" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154482 (owner: AndyRussG) [19:11:29] (CR) AndyRussG: "Thanks, all set! :) I divided this into two changes:" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/138151 (owner: AndyRussG) [19:13:01] (Abandoned) AndyRussG: Improve server-side cohort upload form validation [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/138151 (owner: AndyRussG)