[04:29:41] 10Analytics, 10DBA, 10Data-Services, 10Research, 10cloud-services-team (Kanban): Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3757559 (10MZMcBride) >>! In T173511#3755509, @jcrespo wrote: > As a pro- I have run many times into p... [08:37:03] 10Analytics, 10Operations, 10ops-eqiad, 10User-Elukey: Possibly faulty BBU on analytics1029 - https://phabricator.wikimedia.org/T178742#3757724 (10elukey) 05Open>03Resolved Everything seems good, removed downtime for the host. Thanks Chris! [08:54:13] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure. - https://phabricator.wikimedia.org/T175212#3757749 (10mobrovac) [09:33:02] Hi elukey - Would be around by any chance? [09:34:08] yes :) [09:34:30] elukey: I'd a need a minute of superpowaa [09:35:12] elukey: on stat1005, Erik Zachte stores data that could be useful for me in quality checks (he sent me the link in an email) - But it's non accessible :( [09:37:03] what data? [09:37:29] elukey: /home/ezachte/wikistats_data/dumps/csv/csv_wp [09:40:08] checking [09:41:07] drwxrwxr-x 15 ezachte wikidev 253952 Nov 14 06:23 /home/ezachte/wikistats_data/dumps/csv/csv_wp should be good no? [09:41:38] I get a permission denied - I think it;s comming from one of the top folders [09:42:19] yep probably [09:43:11] ahh there you go drwx------ 10 ezachte wikidev 4096 Aug 23 16:47 /home/ezachte/wikistats_data/dumps [09:45:15] joal: you should be good now [09:45:24] Yay :) [09:45:29] Many thanks elukey :) [09:45:52] !log executed chmod g+rx /home/ezachte/wikistats_data/dumps to unblock Joseph (should be safe) [09:45:53] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:51:37] 10Analytics, 10DBA, 10Data-Services, 10Research, 10cloud-services-team (Kanban): Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3757896 (10jcrespo) > Technical contributors could certainly propose patches for this system, but an o... [10:39:38] 10Analytics, 10ChangeProp, 10EventBus, 10Services (watching): Export burrow metrics to prometheus - https://phabricator.wikimedia.org/T180442#3758088 (10Pchelolo) [10:49:16] 10Analytics, 10User-Elukey: Send burrow lag statistics to prometheus - https://phabricator.wikimedia.org/T120852#3758120 (10elukey) [10:49:19] 10Analytics, 10ChangeProp, 10EventBus, 10Services (watching): Export burrow metrics to prometheus - https://phabricator.wikimedia.org/T180442#3758123 (10elukey) [10:49:32] 10Analytics, 10ChangeProp, 10EventBus, 10Services (watching), 10User-Elukey: Export burrow metrics to prometheus - https://phabricator.wikimedia.org/T180442#3758088 (10elukey) [11:10:47] (03PS1) 10Joal: Correct mediawiki_archive table creation script [analytics/refinery] - 10https://gerrit.wikimedia.org/r/391185 [11:11:15] (03CR) 10Joal: [V: 032 C: 032] "Self merging bug correction." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/391185 (owner: 10Joal) [11:11:50] (03PS4) 10Joal: Add clickstream oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/390226 (https://phabricator.wikimedia.org/T175844) [11:42:55] joal: (still a draft) https://grafana.wikimedia.org/dashboard/db/prometheus-druid [11:43:42] Woooohooo elukey :) [11:44:21] \o/ [11:51:19] * elukey lunch! [11:51:34] later elukey :) [12:06:00] 10Analytics: Add documentation for .m suffix code to pagecounts-ez doc page - https://phabricator.wikimedia.org/T180452#3758404 (10mforns) [12:13:19] (03PS1) 10Joal: Correct clickstream job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/391193 [12:15:44] (03PS5) 10Joal: Add clickstream oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/390226 (https://phabricator.wikimedia.org/T175844) [12:16:19] (03CR) 10Joal: "Better now :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/390226 (https://phabricator.wikimedia.org/T175844) (owner: 10Joal) [12:16:54] (03Abandoned) 10Addshore: WDCM libs [analytics/wmde/WDCM-packages] - 10https://gerrit.wikimedia.org/r/391069 (owner: 10Addshore) [12:40:28] heloooo [12:48:42] Heya mforns [12:48:51] hola joal :] [12:59:32] taking a break a-team [12:59:39] k! [13:04:41] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Add a prometheus metric exporter to all the Druid daemons - https://phabricator.wikimedia.org/T177459#3758668 (10elukey) https://grafana.wikimedia.org/dashboard/db/prometheus-druid [13:06:29] 10Analytics: Add documentation for .m suffix code to pagecounts-ez doc page - https://phabricator.wikimedia.org/T180452#3758672 (10mforns) Here's the wiki pages where the dump's docs have been historically: https://en.wikipedia.org/wiki/User:Killiondude/stats https://archive.org/details/wikipedia_visitor_stats&t... [13:14:11] (03PS13) 10Mforns: [WIP] Add scala-spark core class and job to import data sets to Druid [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/386882 (https://phabricator.wikimedia.org/T166414) [13:30:02] very interesting - https://grafana.wikimedia.org/dashboard/db/prometheus-druid [13:30:10] Heap used vs Max [13:34:07] on the brokers it seems that we are way over provisioned [13:37:55] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add scala-spark core class and job to import data sets to Druid [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/386882 (https://phabricator.wikimedia.org/T166414) (owner: 10Mforns) [14:00:44] 10Analytics, 10ChangeProp, 10EventBus, 10Services (watching), 10User-Elukey: Export burrow metrics to prometheus - https://phabricator.wikimedia.org/T180442#3758829 (10Ottomata) NICE! We've been putting off doing this for graphite for some time, so great that there is already a burrow exporter! There i... [14:42:07] 10Analytics-Kanban: Update html language for per-domain uniques - https://phabricator.wikimedia.org/T168477#3759102 (10Milimetric) [15:11:13] (03CR) 10Fdans: [V: 032 C: 032] "LGTM! Just one minor comment but it doesn't prevent merging." (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/390551 (https://phabricator.wikimedia.org/T178317) (owner: 10Nuria) [16:13:04] (03CR) 10Milimetric: [C: 032] Fixes to table UI [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/389460 (https://phabricator.wikimedia.org/T179441) (owner: 10Fdans) [16:15:32] (03Merged) 10jenkins-bot: Fixes to table UI [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/389460 (https://phabricator.wikimedia.org/T179441) (owner: 10Fdans) [16:36:40] 10Analytics, 10Proton, 10Readers-Web-Backlog, 10MW-1.31-release-notes (WMF-deploy-2017-10-31 (1.31.0-wmf.6)), and 2 others: Implement Schema:Print purging strategy - https://phabricator.wikimedia.org/T175395#3759645 (10Tbayer) @mforns It's not necessary for analysis purposes, but can't hurt much either. B... [17:09:03] (03PS6) 10Joal: Add clickstream oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/390226 (https://phabricator.wikimedia.org/T175844) [17:31:36] Gone for diner - back after [17:34:33] tested all possible metrics for the druid exporter - https://gerrit.wikimedia.org/r/#/c/389475/15/test/test_collector.py [17:34:36] looks good :D [17:38:00] nice [18:24:02] (03PS1) 10Lucas Werkmeister (WMDE): Use HTTPS for links [analytics/wmde/WDCM-ShinyServerFrontPage] - 10https://gerrit.wikimedia.org/r/391270 [18:25:02] * elukey off! [18:33:47] elukey: ran away! [18:33:48] :P [18:34:08] ottomata: I have https://gerrit.wikimedia.org/r/#/c/391269/ for you please :) [18:34:46] (03CR) 10Addshore: [C: 032] Use HTTPS for links [analytics/wmde/WDCM-ShinyServerFrontPage] - 10https://gerrit.wikimedia.org/r/391270 (owner: 10Lucas Werkmeister (WMDE)) [18:34:54] (03Merged) 10jenkins-bot: Use HTTPS for links [analytics/wmde/WDCM-ShinyServerFrontPage] - 10https://gerrit.wikimedia.org/r/391270 (owner: 10Lucas Werkmeister (WMDE)) [18:35:06] merged addshore :) [18:35:22] thanks ottomata! [18:58:39] milimetric: Job finished ;) [19:10:21] hi all! [19:10:36] anyone feel like jumping on a quick call with fr-tech to talk about replacements for kafkatee? [19:10:51] ottomata: joal: ottomata ^ [19:13:40] AndyRussG: I think ottomata is in meeting now, and it makes no sense without him [19:13:46] AndyRussG: Can you send an invite? [19:15:43] joal: yea thanks! [19:16:34] joal: just sent! :) [19:17:19] some other time is also ok.... [19:17:31] AndyRussG: I was thinking of a calendar invite, planning for a meeting when ottomata is available :) [19:17:43] joal: ah OK will do :) [19:18:37] joal: we got ottomata :D [19:18:46] hi ya i'm in hangout [19:18:58] Joining 1 [19:49:40] we'll need superset ottomata I guess [19:57:47] yeaaaa [19:58:05] ottomata: just sayin' ;) [19:58:32] ottomata: If we could manage to have them use druid, that'd be really great [23:45:26] neilpquinn: omg so sorry I forgot to ping you [23:45:32] neilpquinn: your snapshot was done hours ago [23:45:53] but my backup heater blew a fuse so I had to run get one from the store or the baby would freeze, and then I forgot [23:45:53] milimetric: oh, awesome, thank you! [23:45:58] I'll go update our metrics :D [23:46:03] sorry if this messes up your other job [23:46:37] milimetric: nope, it's all good. as far as priorities go I think "baby > metrics" is the correct order :) [23:46:55] :) thanks for understanding