[08:45:10] Analytics-Backlog: Clean up mobile-reportcard dashboards {frog} [13 pts] - https://phabricator.wikimedia.org/T104379#1484151 (Aklapper) @Milimetric: This task has been "Unbreak Now!" priority for four weeks. Please set an assignee and/or correct the priority now. [15:31:06] hmn [15:31:12] what are the costs of more partitions in a Hadoop table? [15:35:48] (CR) Madhuvishy: [C: 2 V: 2] "LGTM" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/217857 (https://phabricator.wikimedia.org/T74117) (owner: Milimetric) [16:05:40] Analytics-Cluster, Analytics-Kanban: Test Cassandra as a storage strategy {slug} [5 pts] - https://phabricator.wikimedia.org/T101786#1485174 (kevinator) [16:06:46] Analytics-Backlog, Analytics-Cluster: Test PostgreSQL as a storage strategy {slug} [5 pts] - https://phabricator.wikimedia.org/T101787#1485175 (kevinator) Open>Resolved a:kevinator canceling this task. Cassandra already is the better choice. [16:06:56] Analytics-Backlog: Clean up mobile-reportcard dashboards {frog} [13 pts] - https://phabricator.wikimedia.org/T104379#1485178 (Milimetric) The task this is blocked on is now resolved (we'll update shortly). We'll get to this soon, but I'll change it from Unbreak Now to High. [16:07:01] Analytics-Backlog: Clean up mobile-reportcard dashboards {frog} [13 pts] - https://phabricator.wikimedia.org/T104379#1485179 (Milimetric) p:Unbreak!>High [16:08:44] Analytics-Kanban: Clean up mobile-reportcard dashboards {frog} [13 pts] - https://phabricator.wikimedia.org/T104379#1485180 (kevinator) [16:13:33] Analytics-Kanban: grant TBayer access to Hue - https://phabricator.wikimedia.org/T106787#1485190 (kevinator) [16:13:48] Analytics-Kanban: enforce policy for each Schema [8 pts] {tick} - https://phabricator.wikimedia.org/T102518#1485193 (madhuvishy) [16:16:44] Analytics-Kanban: Python Aggregator: Solve inconsistencies in data ranges when using --all-projects flag {musk} - https://phabricator.wikimedia.org/T106554#1485199 (kevinator) [16:19:37] Analytics-Backlog, Analytics-Cluster: Python Aggregator: Solve inconsistencies in data ranges when using --all-projects flag {musk} - https://phabricator.wikimedia.org/T106554#1485206 (kevinator) [16:20:04] Analytics-Backlog, Analytics-Cluster: Python Aggregator: Solve inconsistencies in data ranges when using --all-projects flag {musk} - https://phabricator.wikimedia.org/T106554#1485211 (kevinator) p:Triage>Normal [16:21:45] Analytics-Backlog: Restart Pentaho - https://phabricator.wikimedia.org/T105107#1485231 (kevinator) [16:24:01] Analytics-EventLogging, Analytics-Kanban: Build .deb package for pykafka {stag} - https://phabricator.wikimedia.org/T106252#1485234 (kevinator) a:Ottomata [16:24:02] Analytics-Kanban: Clean up mobile-reportcard dashboards {frog} [13 pts] - https://phabricator.wikimedia.org/T104379#1485235 (mforns) a:mforns [16:29:30] Analytics-Kanban, Wikimania-Hackathon-2015: Dockerize Hadoop Cluster, Druid, and Samza + Load Test - https://phabricator.wikimedia.org/T102980#1485247 (Milimetric) [16:30:48] Analytics-Kanban, Wikimania-Hackathon-2015: Dockerize Hadoop Cluster, Druid, and Samza + Load Test - https://phabricator.wikimedia.org/T102980#1379336 (Milimetric) We did not dockerize everything we wanted to because it turned out Docker was not as great of an ecosystem as we guessed going into the hackat... [16:32:59] Analytics-Backlog, Analytics-EventLogging: Puppetize parallel eventlogging-processor {stag} - https://phabricator.wikimedia.org/T104228#1485263 (kevinator) [16:33:01] Analytics-Cluster, operations: Build new latest stable (0.8.2.1?) Kafka package and upgrade Kafka brokers - https://phabricator.wikimedia.org/T106581#1485264 (kevinator) [16:48:11] http://hatjitsu.wmflabs.org/35791 [17:36:34] Analytics-Backlog: Event Logging sends mysql consumer stats to statsd [5 pts] {oryx} - https://phabricator.wikimedia.org/T105935#1485437 (Milimetric) [17:42:42] Analytics-Backlog: create first RESTBase endpoint [8 pts] {slug} - https://phabricator.wikimedia.org/T107053#1485466 (Milimetric) NEW [17:43:39] Analytics-Backlog: create second RESTBase endpoint [8 pts] {slug} - https://phabricator.wikimedia.org/T107054#1485473 (Milimetric) NEW [17:44:28] Analytics-Backlog: create third RESTBase endpoint [8 pts] {slug} - https://phabricator.wikimedia.org/T107055#1485480 (Milimetric) NEW [17:45:16] Analytics-Backlog: Puppetize a server with a role that sets up Cassandra on Analytics machines [13 pts] {slug} - https://phabricator.wikimedia.org/T107056#1485486 (Milimetric) NEW [17:45:54] Analytics-Backlog: Puppetize a server with a role that sets up Cassandra on Analytics machines [13 pts] {slug} - https://phabricator.wikimedia.org/T107056#1485486 (Milimetric) p:Triage>Normal [17:45:59] Analytics-Backlog: create third RESTBase endpoint [8 pts] {slug} - https://phabricator.wikimedia.org/T107055#1485480 (Milimetric) p:Triage>Normal [17:46:05] Analytics-Backlog: create second RESTBase endpoint [8 pts] {slug} - https://phabricator.wikimedia.org/T107054#1485499 (Milimetric) p:Triage>Normal [17:46:08] Analytics-Backlog: create first RESTBase endpoint [8 pts] {slug} - https://phabricator.wikimedia.org/T107053#1485502 (Milimetric) p:Triage>Normal [17:49:09] * milimetric is going to lunch [18:09:33] Analytics-EventLogging: EL unable to decode mobile events due to appinstallid - https://phabricator.wikimedia.org/T96940#1485613 (Deskana) a:Deskana>None [18:18:48] kevinator: Can you approve this when you get a chance? https://phabricator.wikimedia.org/T104020 [18:25:59] Analytics-Cluster, Ops-Access-Requests, operations: Sudo permissions for hdfs user madhuvishy on analytics-hadoop - https://phabricator.wikimedia.org/T104020#1485664 (kevinator) Approved! [18:26:08] madhuvishy: I just did [18:26:16] kevinator: thanks :) [18:32:34] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [18:34:43] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [18:49:11] Analytics-EventLogging, Beta-Cluster, VisualEditor: Beta cluster is sending VisualEditor events to production bits.wikimedia.org/statsv - https://phabricator.wikimedia.org/T98196#1485738 (Jdforrester-WMF) [20:02:22] madhuvishy: hiya! is kevinator around? tell him I'm still on for our 1:1 if he is [20:02:32] ah he just got it [20:02:32] nm [20:08:41] ottomata: cool. Sorry was at lunch [20:32:00] kevinator: I'm in the meeting now and I think it'd still be useful from a requirements / walk-through Hue point of view [20:32:24] did you but if you'd rather postpone, I'm ok [20:59:42] Analytics-Tech-community-metrics, ECT-August-2015: Jenkins-mwext-sync appears in "Who contributes code" - https://phabricator.wikimedia.org/T105983#1486220 (Qgil) [21:01:24] Analytics, Analytics-Backlog, Performance-Team, Patch-For-Review: Collect HTTP statistics about load.php requests - https://phabricator.wikimedia.org/T104277#1486251 (Krinkle) Open>Resolved [21:16:06] Analytics-Cluster, operations, ops-eqiad, Patch-For-Review: rack new hadoop worker nodes - https://phabricator.wikimedia.org/T104463#1486306 (Ottomata) 1042-1045 are installed and part of the Hadoop cluster. [21:37:20] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [21:37:57] Analytics-Cluster, Ops-Access-Requests, operations: Sudo permissions for hdfs user madhuvishy on analytics-hadoop - https://phabricator.wikimedia.org/T104020#1486379 (RobH) a:RobH [21:39:21] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [21:47:31] Analytics-Cluster, Ops-Access-Requests, operations: Sudo permissions for hdfs user madhuvishy on analytics-hadoop - https://phabricator.wikimedia.org/T104020#1486406 (RobH) Analytics-admins grants the user rights to sudo as hdfs, as well as oozie and hive. I think this is indeed the correct group to... [21:48:31] Analytics-Cluster, Ops-Access-Requests, operations: Sudo permissions for hdfs user madhuvishy on analytics-hadoop - https://phabricator.wikimedia.org/T104020#1486407 (RobH) a:RobH>Ottomata [22:45:01] halfak: is your sensitive data discussion with legal still ongoing? [23:22:42] Analytics-Wikimetrics, Community-Wikimetrics: Description of metrics includes link to on-wiki metrics documentation - https://phabricator.wikimedia.org/T93659#1486812 (Fhocutt) a:Fhocutt>None [23:23:11] Analytics-Engineering, Analytics-Wikimetrics, Community-Wikimetrics: User reads result of validation after creating a cohort - https://phabricator.wikimedia.org/T76914#1486814 (Fhocutt) a:Fhocutt>None