[00:58:43] Analytics, Analytics-EventLogging, VisualEditor, VisualEditor pre-2015 work: http://edit-reportcard.wmflabs.org/ doesn't seem to be getting updates (cron job not running yet?) - https://phabricator.wikimedia.org/T76921#1509102 (Jdforrester-WMF) [01:24:56] Analytics-EventLogging, Beta-Cluster, VisualEditor: Beta cluster is sending VisualEditor events to production bits.wikimedia.org/statsv - https://phabricator.wikimedia.org/T98196#1509356 (Jdforrester-WMF) [11:19:47] (PS1) Joal: Correct Pageview host regexps and tests [analytics/refinery/source] - https://gerrit.wikimedia.org/r/229369 (https://phabricator.wikimedia.org/T107764) [11:27:23] Analytics-Kanban, Reading-Admin, Research-and-Data, Research consulting: request for data: sites traffic by topics/ subject areas and geographies - https://phabricator.wikimedia.org/T107613#1510383 (ezachte) @SVentura, awesome initiative, will contact you offline for call today As for World Bank da... [12:12:37] (PS1) Joal: Prevent failure in case of wrong line format [analytics/aggregator] - https://gerrit.wikimedia.org/r/229372 (https://phabricator.wikimedia.org/T107764) [12:46:15] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [12:48:34] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [13:49:30] whoaaa: https://gerrit.wikimedia.org/r/#/c/229172/ [13:49:32] !!! :D :D :D [13:52:31] joal: hiyaaaa [13:53:06] want to walk through a kafka upgrade in labs with me today? as a second pair of eyes? [13:53:16] oh you might have to run soon, yes? [13:53:21] I have time ! [13:53:38] Will go around 8pm my time (in 4 hours) [13:54:11] ottomata: I need some more explanation on the monolog things, just to be sure I understand what it does :) [13:54:22] Let's upgrade :) [13:54:47] k, um, gimme 5 mins, gonna run upstairs and get settled [13:55:31] please take your time :) [13:58:24] k joal, in batcave [13:58:26] https://etherpad.wikimedia.org/p/kafka_0.8.2.1_migration_labs [14:01:21] actually, brb [14:03:33] I'm when you want [14:51:59] Analytics-Wikistats: Adding Odia (Oriya) Wikisource to Stats Wiki - https://phabricator.wikimedia.org/T108012#1510774 (Aklapper) p:Normal>Triage a:Aklapper>None [14:53:08] Analytics-Wikistats: Adding Odia (Oriya) Wikisource to Stats Wiki - https://phabricator.wikimedia.org/T108012#1510311 (Aklapper) @psubhashish1: Please do not assign task to random people and [[ https://www.mediawiki.org/wiki/Phabricator/Project_management#Setting_task_priorities | please do not set the Prior... [15:00:40] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [15:01:45] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [15:23:16] Analytics-Kanban, Patch-For-Review: Add User Agent refined data to intermediate pageview aggregation. [5 pts] {hawk} - https://phabricator.wikimedia.org/T107436#1510882 (kevinator) Open>Resolved [15:23:30] Analytics-Kanban: POC RestBase with cassandra in labs on test data [8 pts] {slug} - https://phabricator.wikimedia.org/T106821#1510884 (kevinator) Open>Resolved [15:25:15] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [15:27:34] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [15:59:19] joal: making soem lunch, am hungry, lets continue shortly... [16:05:55] ottomata: back, no problemob [16:06:08] milimetric: you around for a quick chat ? [16:06:18] joal: of course [16:06:24] cave or text? [16:06:27] cave ? [16:06:29] omw [16:06:33] thx [16:29:50] ottomata: ready ! [16:35:30] Analytics-Tech-community-metrics, ECT-August-2015: Tech community KPIs for the WMF metrics meeting - https://phabricator.wikimedia.org/T107562#1511175 (Qgil) @aklapper, thank you, this is great. Terry wants me to present these metrics at the WMF Engineering management meeting on a monthly basis. This is a... [17:40:42] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [17:42:44] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [18:02:47] Gone for today ! [18:02:52] Have a good one lads [18:02:57] nick joal|night [18:05:42] laters! [18:07:11] bah! [18:07:11] https://github.com/linkedin/camus [18:07:17] Camus is being phased out and replaced by https://github.com/linkedin/gobblin. For those using or interested in Camus, we suggest taking a look at Gobblin. [18:08:22] ottomata: question - can i use puppet and put in an entry in /etc/hosts for statsd. So I can just call the hostname statsd in my code - and it will point to the server hostname in prod [18:09:57] Analytics-Tech-community-metrics, ECT-August-2015: Tech community KPIs for the WMF metrics meeting - https://phabricator.wikimedia.org/T107562#1511700 (Aklapper) a:Aklapper>Qgil I interpret the last comment as a "I'll change the assignee from Andre to Quim". :) [18:11:21] Analytics-Backlog, Release-Engineering, operations, Varnish: Verify traffic to static resources from past branches does indeed drain - https://phabricator.wikimedia.org/T102991#1511716 (Krinkle) [18:12:51] madhuvishy: not in etc hosts, but you shoudn't hardcoe the name anyway [18:12:56] it should be a parameter somehow, right? [18:13:02] like, in the consumer url even [18:13:11] if it isn't provided then no stats will be sent [18:13:20] ottomata: in the consumer url hmmm [18:13:26] that should be fine [18:13:28] ...&statsd=statsd.eqiad.wmnet [18:13:32] yeah [18:13:35] that'd work [18:13:38] thanks [18:13:43] yup! [18:13:49] making coffee, brb [18:33:50] Analytics-EventLogging, Patch-For-Review: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1511824 (Ottomata) In https://gerrit.wikimedia.org/r/229172, Eric wrote: >We might also want to think about Avro schema storage. For this first use case I'm ok with embedding the schema in the... [18:35:31] (CR) Ottomata: [C: 2 V: 2] Correct Pageview host regexps and tests [analytics/refinery/source] - https://gerrit.wikimedia.org/r/229369 (https://phabricator.wikimedia.org/T107764) (owner: Joal) [18:35:57] (CR) Ottomata: [C: 2 V: 2] Prevent failure in case of wrong line format [analytics/aggregator] - https://gerrit.wikimedia.org/r/229372 (https://phabricator.wikimedia.org/T107764) (owner: Joal) [18:40:01] Analytics-EventLogging, Patch-For-Review: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1511879 (Ironholds) Thankfully I don't :). If I want to consume it I plan to use Hive queries. I am going to start writing an avro client in R, in my spare time, though. [18:41:27] Analytics-EventLogging, Patch-For-Review: Kafka Client for MediaWiki - https://phabricator.wikimedia.org/T106256#1511880 (Legoktm) [18:51:03] Quarry: Large queries results not displaying (Aug 2015) - https://phabricator.wikimedia.org/T108084#1511927 (Dispenser) NEW [18:54:10] (Abandoned) Ottomata: Modify access rules [analytics/camus] (refs/meta/config) - https://gerrit.wikimedia.org/r/202728 (owner: Alexey) [19:02:14] madhuvishy: ping when you want to have lunch. no rush on my end, and feel free to not to if you have a meeting or something. ;-) [19:02:23] leila: now? :) [19:02:28] sure. :D [19:02:43] i was just thinking of going. 2 minutes - need to use restroom [19:22:22] Analytics, Labs, Labs-Infrastructure, Labs-Sprint-108, Patch-For-Review: Set up cron job on labstore to rsync data from stat* boxes into labs. - https://phabricator.wikimedia.org/T107576#1512028 (Ottomata) @mark or @akosiaris or @faidon, we'll need a hole punched in the Analytics VLAN ACL for th... [19:23:21] Anyone know mforns on-wiki username? [19:23:59] Nevermind. FOund it! User:Mforns_(WMF) [19:28:56] Analytics-Backlog, Release-Engineering, operations, Varnish: Verify traffic to static resources from past branches does indeed drain - https://phabricator.wikimedia.org/T102991#1512047 (Krinkle) Raising priority and adding to #Performance-Team workboard. The fact that our front ends seem to be servin... [19:41:20] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 26.67% of data above the critical threshold [30.0] [19:42:10] Analytics, MediaWiki-Authentication-and-authorization, Reading-Infrastructure-Team, MW-1.26-release, and 2 others: Create dashboard to track key authentication metrics before, during and after AuthManager rollout - https://phabricator.wikimedia.org/T91701#1512096 (bd808) [19:45:40] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [19:47:50] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [19:52:20] Analytics-EventLogging, Patch-For-Review: Kafka Client for MediaWiki - https://phabricator.wikimedia.org/T106256#1512152 (Ironholds) Thankfully I don't :). If I want to consume it I plan to use Hive queries. I am going to start writing an avro client in R, but that's a spare-time thing. Will this do anyth... [20:43:19] Analytics-Cluster, Analytics-Kanban, operations, Patch-For-Review: Build new latest stable (0.8.2.1?) Kafka package and upgrade Kafka brokers - https://phabricator.wikimedia.org/T106581#1512300 (Ottomata) Phew, ok, Joseph and I tested this migration again in labs: https://etherpad.wikimedia.org/p/ka... [21:55:10] Analytics-Kanban, Analytics-Wikistats: {lama} Wikistats 2.0 - https://phabricator.wikimedia.org/T107175#1512659 (Tbayer)