[00:38:40] Quarry: Quarry does not respect ORDER BY sort order in result set - https://phabricator.wikimedia.org/T87829#1462986 (Huji) Fantastic! [05:37:04] Analytics-EventLogging: Make eventlogging-reporter work with generic processor uris, not just 0mq sockets. - https://phabricator.wikimedia.org/T93415#1463091 (Ottomata) Open>declined Doesn't make sense. Metrics will come from Kafka and reporter will be deprecated. [05:48:42] Analytics-EventLogging, Analytics-Kanban, Patch-For-Review, WMF-deploy-2015-07-07_(1.26wmf13): Prep work for Eventlogging on Kafka {stag} - https://phabricator.wikimedia.org/T102831#1463105 (Ottomata) So, there are two high level {stag} Kafka on EventLogging tickets. This one, and T102225. I thin... [05:49:45] Analytics-EventLogging, Analytics-Kanban: Build .deb package for pykafka - https://phabricator.wikimedia.org/T106252#1463109 (Ottomata) NEW [05:50:55] Analytics-EventLogging, Analytics-Kanban: Build .deb package for pykafka - https://phabricator.wikimedia.org/T106252#1463109 (Ottomata) Probably done! https://github.com/wikimedia/operations-debs-python-pykafka/tree/debian/debian http://apt.wikimedia.org/wikimedia/pool/main/p/python-pykafka/ Will need... [05:52:01] Analytics-Backlog, Analytics-EventLogging: Puppetize parallel eventlogging-processor {stag} - https://phabricator.wikimedia.org/T104228#1463119 (Ottomata) [05:52:03] Analytics-EventLogging, Analytics-Kanban: {stag} EventLogging on Kafka - https://phabricator.wikimedia.org/T102225#1463118 (Ottomata) [05:53:40] Analytics-EventLogging, Analytics-Kanban: Regularly purge EventLogging data in Hadoop - https://phabricator.wikimedia.org/T106253#1463120 (Ottomata) NEW [05:54:52] Analytics-EventLogging, Analytics-Kanban: Make EventLogging alerts based on Kafka metrics - https://phabricator.wikimedia.org/T106254#1463126 (Ottomata) NEW a:Ottomata [05:57:15] Analytics-EventLogging, Analytics-Kanban: Send raw client side events to Kafka using varnishkafka instead of varnishncsa - https://phabricator.wikimedia.org/T106255#1463133 (Ottomata) NEW [06:04:31] Analytics-EventLogging: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1463144 (Ottomata) NEW a:Ottomata [06:05:01] Analytics-EventLogging: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1463144 (Ottomata) [06:05:50] Analytics-EventLogging, Analytics-Kanban: {stag} EventLogging on Kafka - https://phabricator.wikimedia.org/T102225#1463157 (Ottomata) [06:05:51] Analytics-EventLogging: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1463144 (Ottomata) [06:07:25] Analytics-EventLogging, Analytics-Kanban: Send raw server side events to Kafka using a PHP Kafka Client - https://phabricator.wikimedia.org/T106257#1463159 (Ottomata) NEW [06:07:39] Analytics-EventLogging: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1463144 (Ottomata) [06:07:52] Analytics-EventLogging: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1463144 (Ottomata) [06:07:54] Analytics-EventLogging, Analytics-Kanban: {stag} EventLogging on Kafka - https://phabricator.wikimedia.org/T102225#1463170 (Ottomata) [06:08:26] Analytics-EventLogging, Analytics-Kanban: Send raw server side events to Kafka using a PHP Kafka Client {stag} - https://phabricator.wikimedia.org/T106257#1463159 (Ottomata) [06:08:30] Analytics-EventLogging, Analytics-Kanban: Build .deb package for pykafka {stag} - https://phabricator.wikimedia.org/T106252#1463174 (Ottomata) [06:08:36] Analytics-EventLogging, Analytics-Kanban: Regularly purge EventLogging data in Hadoop {stag} - https://phabricator.wikimedia.org/T106253#1463176 (Ottomata) [06:08:41] Analytics-EventLogging, Analytics-Kanban: Send raw client side events to Kafka using varnishkafka instead of varnishncsa {stag} - https://phabricator.wikimedia.org/T106255#1463178 (Ottomata) [06:08:55] Analytics-EventLogging: {stag} Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1463180 (Ottomata) [06:09:02] Analytics-EventLogging: Kafka Client for Mediawiki - https://phabricator.wikimedia.org/T106256#1463144 (Ottomata) [06:12:48] Analytics-EventLogging, Analytics-Kanban: Make EventLogging alerts based on Kafka metrics {stag} - https://phabricator.wikimedia.org/T106254#1463196 (Ottomata) [07:07:51] Analytics-EventLogging, Analytics-Kanban: Deploy EventLogging on Kafka to eventlog1001 (aka production!) {stag} - https://phabricator.wikimedia.org/T106260#1463223 (Ottomata) NEW [07:12:11] Analytics-EventLogging, Analytics-Kanban: {stag} EventLogging on Kafka - https://phabricator.wikimedia.org/T102225#1463230 (Ottomata) {F198341 size=full} [10:50:10] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [10:53:59] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [12:08:08] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [12:11:50] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [13:41:08] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [13:42:59] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [15:26:59] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [15:28:59] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [16:46:17] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [16:51:57] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [16:58:06] Quarry: Add database selector - https://phabricator.wikimedia.org/T76466#1463750 (yuvipanda) duplicate>Open [16:58:18] Quarry: Add database selector - https://phabricator.wikimedia.org/T76466#801318 (yuvipanda) I think this is different than running it on multiple wikis, and probably easier to tackle... [20:08:14] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [20:11:55] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [20:27:48] Quarry: Allow comments on queries - https://phabricator.wikimedia.org/T71543#1464058 (yuvipanda) So we can actually apparently create unattached (to a page) flow boards that can be used!