[09:57:33] <joal|night>	 nick joal
[14:00:24] <joal>	 ottomata: Godd morning !
[14:00:39] <joal>	 s/godd/good/
[14:00:43] <ottomata>	 mooorning!
[14:01:03] <joal>	 Have some time for me to discuss budgeting before this afternoon meeting ?
[14:05:01] <ottomata>	 oh meeting about bugdgeting it today??
[14:05:05] <ottomata>	 oh, hm
[14:05:07] <ottomata>	 sure
[14:05:18] <ottomata>	 gimme some minutes i'm trying to make my labs hadoop cluster HA
[14:05:22] <joal>	 Let me know when you want
[14:09:08] <halfak>	 o/ Hey folks
[14:09:42] <halfak>	 It looks like there's a gap in EL data from yesterday between 1800 and 2300 UTC.  Is that a loss or is it being filled?
[14:10:21] <joal>	 halfak: No idea, milimetric and mforns have been working on EL yesterday night, so it's difficult to answer
[14:10:30] <joal>	 halfak: Better to wait for them
[14:10:39] <halfak>	 kk
[14:10:42] <halfak>	 Thanks joal
[14:10:46] <joal>	 np :)
[14:10:56] <halfak>	 also good afternoon :)
[14:11:25] <joal>	 Thank you halfak, good morning to you :)
[14:12:24] <halfak>	 The EL gods are conspiring to make my work difficult these days.
[14:12:42] <halfak>	 Every time I try to test VE's event logging, there's a lag spike and/or data loss.
[14:31:14] <joal>	 halfak: Murphy's law ... I deal with it with altiscale on my side :)
[14:31:39] <halfak>	 How's that going BTW?
[14:31:42] <halfak>	 joal, ^
[14:31:51] <joal>	 halfak: not so good
[14:31:55] <halfak>	 Boo.
[14:31:59] <ottomata>	 joal, ok can talk now
[14:32:19] <joal>	 Can't find a way to troubleshoot proxy, and answers they give on the ticketing side are not very relevant :(
[14:32:26] <joal>	 halfak: -^
[14:32:36] <joal>	 We'll talk at the next meeting
[14:32:40] <joal>	 halfak: --^
[14:32:44] <joal>	 ottomata: batcave ?
[14:32:45] <halfak>	 joal, my experience as well.  There's a weekly meeting with them where we can address this.
[14:33:06] <joal>	 halfak: Yup, I have noted that
[14:33:18] <joal>	 I'll try to be present when it can be helpful to me
[14:33:24] <halfak>	 :)  We should talk about how late in the day it is for you.
[14:33:28] <halfak>	 I'd be down for rescheduling.
[14:33:54] <joal>	 Don't worry, once in a while it's not that bad :)
[14:33:58] <joal>	 halfak: --^
[15:21:50] <grrrit-wm>	 (PS3) Mforns: Fixing encoding on json responses at the encoder level [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/203505 (https://phabricator.wikimedia.org/T93023) (owner: Nuria)
[16:51:26] <joal>	 ottomata: Is that true ? You have already patched for CDH 5.4 ?
[16:57:28] <ottomata>	 ?
[16:57:35] <ottomata>	 patched?
[16:57:49] <ottomata>	 joal, i have installed cdh4 in labs and in vagrant so far.
[16:57:54] <ottomata>	 need to plan downtime to do migration for production
[16:58:04] <ottomata>	 also,  I am very hungry, bbiab!
[16:58:11] <joal>	 gerribot was saying merged :)
[16:58:23] <joal>	 So I said patched
[16:58:32] <ottomata>	 i updated apt.w to mirror 5.4.0 packages, tis all
[16:58:33] <joal>	 But I understand it still nedds to be deployed :)
[16:58:43] <ottomata>	 looks like no puppet changes needed for this, so that is good!
[16:58:49] <joal>	 cool
[16:59:35] <milimetric>	 mforns: so about backfilling data
[16:59:39] <milimetric>	 I don't think we can backfill, right?
[16:59:41] <mforns>	 milimetric, hi
[16:59:57] <milimetric>	 also, since my internet crapped out today.  My status update:
[16:59:59] <mforns>	 milimetric, not for yesterday night's problem, right?
[17:00:05] <milimetric>	 * worked on testing for dashiki for a bit
[17:02:15] <milimetric>	 * writing the incident report for EL outage yesterday (got interrupted a million times :))
[17:02:25] <mforns>	 hehehe
[17:02:44] <milimetric>	 seriously, I have no idea how people get work done in the office
[17:02:49] <milimetric>	 I think they should all just go remote
[17:02:59] <mforns>	 :]
[17:03:03] <milimetric>	 ok, back to work
[17:03:19] <milimetric>	 so yeah, mforns, for yesterday, we can't backfill because I stopped the processor that one time
[17:03:24] <milimetric>	 when I thought it was the processor's fault
[17:03:31] <mforns>	 milimetric, yes...
[17:03:39] <milimetric>	 ok, that's what I'll write in the report.
[17:03:59] <mforns>	 milimetric, the processor crashed also during some time, it was not your fault
[17:04:26] <mforns>	 milimetric, the processor also gets the schema from the mediawiki api
[17:04:45] <milimetric>	 and if we really really wanted the data, couldn't we find it in the webrequest log?
[17:05:17] <mforns>	 milimetric, and the raw logs?
[17:06:51] <wikibugs>	 Analytics-Kanban, Analytics-Visualization, Patch-For-Review: limn-mobile-data requirements.txt file is missing python-dateutil - https://phabricator.wikimedia.org/T75432#1246063 (Milimetric) Open>Resolved a:Milimetric
[17:07:01] <mforns>	 milimetric, oh, you mean we stopped the forwarder? oh, that's true, at some point I stopped the whole EL pipeline
[17:08:15] <wikibugs>	 Analytics-Kanban, Analytics-Visualization, Patch-For-Review: limn-mobile-data requirements.txt file needs distribute version - https://phabricator.wikimedia.org/T75431#1246081 (Milimetric) Open>Resolved a:Milimetric
[17:09:29] <mforns>	 milimetric, I'm not sure if the data is in the webrequest logs, but it makes sense
[17:12:58] <milimetric>	 yeah, i'll just say we can't easily backfill right now
[17:14:52] <mforns>	 milimetric, ok
[18:33:47] <joal>	 off now !
[18:33:52] <joal>	 See you guys tomorrow :)
[19:09:29] <grrrit-wm>	 (PS1) Ottomata: Fix kafkatee.postinst to work in Jessie - use service command [analytics/kafkatee] (debian) - https://gerrit.wikimedia.org/r/207530
[19:10:42] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] Fix kafkatee.postinst to work in Jessie - use service command [analytics/kafkatee] (debian) - https://gerrit.wikimedia.org/r/207530 (owner: Ottomata)
[19:17:03] <grrrit-wm>	 (PS1) Ottomata: Properly restarting rsyslog using service [analytics/kafkatee] - https://gerrit.wikimedia.org/r/207531
[19:17:39] <grrrit-wm>	 (Abandoned) Ottomata: Properly restarting rsyslog using service [analytics/kafkatee] - https://gerrit.wikimedia.org/r/207531 (owner: Ottomata)
[19:18:15] <grrrit-wm>	 (PS1) Ottomata: Properly restarting rsyslog using service [analytics/kafkatee] (debian) - https://gerrit.wikimedia.org/r/207532
[19:18:27] <grrrit-wm>	 (CR) Ottomata: [C: 2 V: 2] Properly restarting rsyslog using service [analytics/kafkatee] (debian) - https://gerrit.wikimedia.org/r/207532 (owner: Ottomata)
[19:32:58] <grrrit-wm>	 (PS1) Mooeypoo: Hide partial week data from charts [analytics/limn-flow-data] - https://gerrit.wikimedia.org/r/207534 (https://phabricator.wikimedia.org/T95249)
[21:41:42] <wikibugs>	 Analytics-Dashiki: Dashiki misses data for “Daily Pageviews (webstatscollector)” metric between 2015-04-09 and 2015-04-12 - https://phabricator.wikimedia.org/T97588#1247226 (QChris) NEW
[21:43:30] <wikibugs>	 Analytics-Dashiki: Dashiki misses data for “Daily Pageviews (webstatscollector)” metric between 2015-04-09 and 2015-04-12 - https://phabricator.wikimedia.org/T97588#1247233 (QChris) a:QChris
[21:47:26] <grrrit-wm>	 (PS1) QChris: Backfill missing data from 2015-04-09 until 2015-04-12 [analytics/aggregator/data] - https://gerrit.wikimedia.org/r/207645 (https://phabricator.wikimedia.org/T97588)
[21:48:21] <grrrit-wm>	 (CR) QChris: [C: 2 V: 2] "Self-merging per team decision on such pure data commits :-/" [analytics/aggregator/data] - https://gerrit.wikimedia.org/r/207645 (https://phabricator.wikimedia.org/T97588) (owner: QChris)
[22:10:51] <wikibugs>	 Analytics-Dashiki, Patch-For-Review: Dashiki misses data for “Daily Pageviews (webstatscollector)” metric between 2015-04-09 and 2015-04-12 - https://phabricator.wikimedia.org/T97588#1247317 (QChris) Open>Resolved
[23:31:49] <wikibugs>	 Analytics-Cluster: Admin can perform "Database Auditing" on the cluster to be aware of actions of cluster users - https://phabricator.wikimedia.org/T85288#1247735 (kevinator)
[23:59:05] <grrrit-wm>	 (CR) Madhuvishy: [C: 1] "Tested and working on wikimetrics on top of the patch - looks good to me." [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/206343 (owner: Milimetric)