[03:52:18] Analytics-Cluster, Labs, wikitech.wikimedia.org: Include role::analytics::hadoop roles in default list of labs puppet groups - https://phabricator.wikimedia.org/T70391#1478063 (yuvipanda) Open>declined I have cleaned the default roles to a minimum, and these should just be project specific roles... [07:12:38] Analytics-Kanban: grant TBayer access to Hue - https://phabricator.wikimedia.org/T106787#1478246 (kevinator) NEW a:Ottomata [07:14:48] Analytics-Kanban: grant TBayer access to Hue - https://phabricator.wikimedia.org/T106787#1478255 (kevinator) @Tbayer needs to do some reporting on Pageviews and using Hue + Hive on the table wmf.projectcounts will give him numbers. [07:19:21] Analytics-Kanban: Restart Pentaho - https://phabricator.wikimedia.org/T105107#1478283 (kevinator) {T106787} has just been logged. [09:13:14] Analytics-Kanban: grant TBayer access to Hue - https://phabricator.wikimedia.org/T106787#1478445 (Tbayer) See also {T105748} [09:16:35] Analytics-Backlog: Provide the Wikimedia DE folks with Hive access/training - https://phabricator.wikimedia.org/T106042#1478460 (fgiunchedi) [09:16:37] Analytics-Backlog, Ops-Access-Requests, operations, Patch-For-Review: Provide daniel (Daniel Kinzler) with Hive access - https://phabricator.wikimedia.org/T106047#1478457 (fgiunchedi) Open>Resolved a:fgiunchedi merged, access should be available shortly [13:40:15] * mforns tests [13:47:31] mforns: What do you test? [13:47:45] xD my IRC client [13:48:05] huhu :) [13:48:10] seems to work ! [13:48:14] yes [13:48:15] :] [13:48:29] by the way: Hi ! [13:48:35] hehe, hi! [14:07:14] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [14:09:15] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [14:19:34] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [14:21:44] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [14:30:09] Analytics-EventLogging: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1479141 (Ottomata) AH Cool! That is what I was looking for! Awesome. > ["topics"]=> > array(1) { > ["test1"]=> I hope you saw more than just “test1” in prod :) Yes, then I think that will do it. Ea... [15:33:11] \but comments are still welcome Marcel ! [15:33:14] oop [15:33:26] O.o [15:33:44] communication channel misfit [15:33:48] mforns: --^ [16:05:37] Analytics-Kanban: POC RestBase with cassandra in labs on test data - https://phabricator.wikimedia.org/T106821#1479353 (JAllemandou) NEW a:JAllemandou [16:06:04] Analytics-Backlog, Analytics-EventLogging: Make EventLogging code mark new tables for purging as default {tick} - https://phabricator.wikimedia.org/T106558#1479364 (kevinator) [16:07:00] Analytics-Kanban: POC RestBase with cassandra in labs on test data {slug} [? pts] - https://phabricator.wikimedia.org/T106821#1479373 (JAllemandou) [16:07:34] Analytics-Kanban: POC RestBase with cassandra in labs on test data [? pts] {slug} - https://phabricator.wikimedia.org/T106821#1479353 (JAllemandou) [16:08:17] Analytics-Backlog, Analytics-EventLogging: Make EventLogging code mark new tables for purging as default {tick} - https://phabricator.wikimedia.org/T106558#1479385 (ggellerman) p:Triage>Normal [16:12:42] Analytics-Backlog, Analytics-EventLogging: Make EventLogging alerts based on Kafka metrics {stag} - https://phabricator.wikimedia.org/T106254#1479399 (ggellerman) p:Normal>High [16:18:39] Analytics-Backlog: Change Icinga graphite alert for EventLogging delay - https://phabricator.wikimedia.org/T106495#1479431 (ggellerman) [16:23:26] Analytics-Backlog: Provide the Wikimedia DE folks with Hive access/training - https://phabricator.wikimedia.org/T106042#1479435 (ggellerman) p:Triage>Normal [16:31:17] Analytics-Backlog: Event Logging sends mysql consumer stats to statsd - https://phabricator.wikimedia.org/T105935#1479514 (ggellerman) p:Triage>High [16:34:05] Analytics, Analytics-Kanban, Research-and-Data: Too few page views for June/July 2015 - https://phabricator.wikimedia.org/T106034#1479572 (DarTar) p:Triage>Normal [16:34:25] Analytics-Kanban, Research-and-Data, Research management: Pipeline from Research to productization - https://phabricator.wikimedia.org/T105815#1479575 (DarTar) p:Triage>High [16:36:20] Analytics-Backlog, Research-and-Data: Workshop to teach analysts, etc about Quarry, Hive, Wikimetrics and EL - https://phabricator.wikimedia.org/T105544#1446198 (DarTar) [16:45:52] kevinator: yt? are you still in grooming? [16:48:52] i'm in 1:1 for next hour [16:49:51] k, just sent email to analytics internal with link to draft of event system email [16:49:52] would like feedback [16:49:54] before I send [16:59:20] kevinator, joal, I gues you'll like to extend your 1x1 a little bit, let me know if you want me to be a little late [16:59:36] thx mforns :) [16:59:39] mforns: give me 5 minutes [16:59:42] ok [18:17:16] ottomata: this would be a usecase for the scalable event system may be? https://phabricator.wikimedia.org/T100082 [18:18:32] oo, madhuvishyyes [18:18:44] i think rcstream should be moved to this [18:18:57] i think that whtaever work they do now is fine though [18:19:08] we can change how they produce the events later if we need to [18:19:28] ottomata: yeah, but i think they need it to be public [18:19:44] may be rest proxy? [18:20:20] maybe, or maybe rcstream service can still do socket.io but pull from kafka [18:20:24] instead of receiving directly from mw [18:20:24] yeah [18:20:44] RCStream folks - whoever they are - could be stakeholders? [18:27:36] thanks madhuvishy yeah, indeed. i had forgotten about https://phabricator.wikimedia.org/T84923, now referencing in email [18:29:31] oh cool [18:29:36] i've seen this before [18:30:20] ottomata: just seeing this [18:30:21] https://meta.wikimedia.org/wiki/Research:MediaWiki_events:_a_generalized_public_event_datasource [18:30:40] o/ [18:30:40] I think Aaron will be interested in this :) [18:30:46] ha ha [18:30:50] just on cue [18:31:00] nice! the more references the better! [18:31:03] * halfak has 'research' as a ping word [18:31:05] mwhahaha [18:31:13] halfak: aaah no wonder [18:31:36] halfak: do you have any references to eventlogging viz editor analysis work you did? wiki doc or phab ticket? [18:32:15] ottomata, this? https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors/May_2015_study [18:33:38] that'll do thank you! [18:34:04] ottomata, there's this too https://phabricator.wikimedia.org/T99181 [18:34:35] k danke [18:35:40] kevinator: lemme know when you have 5 minutes [18:36:00] I do... I was reading over your message to send [18:36:24] ottomata: batcave? IRC? [18:36:47] sho [18:38:30] kevinator: am there [19:18:10] ottomata: BTW how long should it take someone to complete the DevOps task? [19:18:26] hm, an hour or two? [19:18:33] say 2 [19:18:47] ok, I'll add that to the doc. [19:20:32] k [20:44:43] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [20:46:43] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [20:51:47] Analytics-Cluster, operations: Build new latest stable (0.8.2.1?) Kafka package and upgrade Kafka brokers - https://phabricator.wikimedia.org/T106581#1480936 (Ottomata) OO! I was able to build this! Amazing! Alex's Makefiles are super simple! I had some merge conflicts and auto-merges I wasn't able to... [23:06:45] Analytics, Analytics-Kanban, Research-and-Data, Research consulting: Too few page views for June/July 2015 - https://phabricator.wikimedia.org/T106034#1481367 (DarTar) [23:07:16] Analytics-Backlog, Research-and-Data, Research consulting: Workshop to teach analysts, etc about Quarry, Hive, Wikimetrics and EL - https://phabricator.wikimedia.org/T105544#1481370 (DarTar) [23:08:47] Analytics, Engineering-Community, MediaWiki-API, Research-and-Data, and 3 others: Metrics about the use of the Wikimedia web APIs - https://phabricator.wikimedia.org/T102079#1481377 (DarTar) [23:09:41] Analytics: find out what browsers Wikimedia projects editors use - https://phabricator.wikimedia.org/T78539#1481379 (DarTar) [23:13:44] Analytics: Referrer data for en:Glitter for shareafact test - https://phabricator.wikimedia.org/T93270#1481402 (DarTar) [23:22:28] Analytics-Kanban, Research-and-Data, Research consulting: Validate Uniques using Last Access cookie {bear} - https://phabricator.wikimedia.org/T101465#1481482 (DarTar) [23:23:55] Analytics-Kanban, Research-and-Data, Research consulting: Analysis on traffic through the HTTPS transition - https://phabricator.wikimedia.org/T102431#1481491 (DarTar)