[01:12:00] joal ahh sorry! lets talk thurs [06:18:53] morningggg [06:18:58] I am reimaging aqs1008 [06:49:44] aand done [06:49:51] aqs1009 left [07:57:48] 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Review analytics-in4 rules on cr1/cr2 eqiad - https://phabricator.wikimedia.org/T198623 (10Gehel) For WDQS, we should keep access to at least 2 nodes in both eqiad and codfw. I propose: wdqs1003: 10.64.0.14 wdqs1004: 10.64.0.17 wdqs2001: 10.192.32.1... [08:10:03] 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Review analytics-in4 rules on cr1/cr2 eqiad - https://phabricator.wikimedia.org/T198623 (10elukey) First batch of changes: ``` delete firewall family inet filter analytics-in4 term logstash delete firewall family inet filter analytics-in4 term event... [08:46:43] dcausse: o/ - I just updated the analytics firewall rules to https://phabricator.wikimedia.org/T198623#4396939, let me know if you see any trouble later on [08:46:59] elukey: thanks! [09:15:05] addshore: updated also the wdqs rules int --^, but I am afraid that those are not as you wished :( [09:17:07] :) [09:26:32] addshore: whenever you have time can you add a comment in your code where you have hardcoded wdqs1003? [09:27:16] I can do, linking to that ticket or? [09:34:26] yep exactly, so we'll remember why :) [09:35:35] 10Analytics, 10Patch-For-Review, 10User-Elukey: Upgrade AQS to Debian Stretch - https://phabricator.wikimedia.org/T196138 (10ops-monitoring-bot) Completed auto-reimage of hosts: ``` ['aqs1009.eqiad.wmnet'] ``` and were **ALL** successful. [09:38:55] (03PS1) 10Addshore: Add note about wdqs access through firewall [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/443803 (https://phabricator.wikimedia.org/T198623) [09:39:12] elukey: ^^ also linked the commit to the ticket so it works the other way too :) [09:40:07] (03CR) 10Addshore: [C: 032] Add note about wdqs access through firewall [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/443803 (https://phabricator.wikimedia.org/T198623) (owner: 10Addshore) [09:40:13] (03PS1) 10Addshore: Add note about wdqs access through firewall [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/443804 (https://phabricator.wikimedia.org/T198623) [09:40:21] (03Merged) 10jenkins-bot: Add note about wdqs access through firewall [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/443803 (https://phabricator.wikimedia.org/T198623) (owner: 10Addshore) [09:41:18] (03CR) 10Addshore: [C: 032] Add note about wdqs access through firewall [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/443804 (https://phabricator.wikimedia.org/T198623) (owner: 10Addshore) [09:41:32] (03Merged) 10jenkins-bot: Add note about wdqs access through firewall [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/443804 (https://phabricator.wikimedia.org/T198623) (owner: 10Addshore) [09:43:03] aand aqs fully reimaged to stretch! [09:48:29] \o/ [10:21:46] addshore: thanks! [10:33:25] * elukey lunch + errand! [10:42:21] 10Analytics: Order Data Lake Hardware - https://phabricator.wikimedia.org/T198424 (10JAllemandou) Thanks for the very accurate summary @elukey :) [11:17:25] https://twitter.com/JohnHackworth/status/1014466821439852545 [12:01:06] fdans: Let's DO IT ! [12:01:10] :D [12:16:05] joal: do you maybe have a couple minutes to talk about the task about correcting the data removal jobs in mw history? [13:09:45] 10Analytics, 10Analytics-Kanban: Reimage thorium to Debian Stretch - https://phabricator.wikimedia.org/T192641 (10elukey) a:05elukey>03None [14:07:17] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10MediaWiki-extensions-ORES, and 4 others: ORESFetchScoreJob fails quite a lot - https://phabricator.wikimedia.org/T196076 (10Ladsgroup) hmm, yeah. I think we should make another phabricator ticket because that's about the ores service and not the extension wa... [15:41:54] if anybody is available to talk about mediawiki private tables import I'd be super glad :) [15:47:18] Hey folks [15:47:22] Here I am [15:48:32] fdans, elukey: pre-standup talk, or post? [15:52:30] (03PS2) 10Sahil505: Make bar-chart and line-chart resilient to breakdowns with null values [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/443640 (https://phabricator.wikimedia.org/T198630) (owner: 10Mforns) [15:54:51] joal: I am free now if you have time! [15:54:58] To the cave ! [16:01:28] (03CR) 10Mforns: Make bar-chart and line-chart resilient to breakdowns with null values (032 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/443640 (https://phabricator.wikimedia.org/T198630) (owner: 10Mforns) [16:04:54] (03PS3) 10Sahil505: Make bar-chart and line-chart resilient to breakdowns with null values [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/443640 (https://phabricator.wikimedia.org/T198630) (owner: 10Mforns) [16:05:45] (03CR) 10Sahil505: [C: 031] Make bar-chart and line-chart resilient to breakdowns with null values (032 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/443640 (https://phabricator.wikimedia.org/T198630) (owner: 10Mforns) [16:08:26] (03CR) 10Mforns: [C: 031] "LGTM!" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/443640 (https://phabricator.wikimedia.org/T198630) (owner: 10Mforns) [16:54:31] * elukey off! [20:18:59] (03PS6) 10Joal: [WIP] Add validation step in mediawiki-history jobs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/440005 (https://phabricator.wikimedia.org/T192481) [20:40:55] (03CR) 10Mforns: "Code looks good, a couple comments on naming. Cheers" (032 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/443592 (https://phabricator.wikimedia.org/T198183) (owner: 10Sahil505) [21:15:43] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10MediaWiki-extensions-ORES, and 4 others: ORESFetchScoreJob fails quite a lot - https://phabricator.wikimedia.org/T196076 (10Ladsgroup) Made {T198819}