[00:47:40] 10netops, 10Operations: automatically sample from all FPCs on core routers - https://phabricator.wikimedia.org/T257392 (10CDanis) [00:47:56] 10netops, 10Operations: automatically sample from all FPCs on core routers - https://phabricator.wikimedia.org/T257392 (10CDanis) [00:48:21] 10netops, 10Operations: automatically sample from all FPCs on core routers - https://phabricator.wikimedia.org/T257392 (10CDanis) p:05Triage→03Low [00:48:28] 10netops, 10Operations: automatically sample from all FPCs on core routers - https://phabricator.wikimedia.org/T257392 (10CDanis) a:05CDanis→03None [00:51:43] 10netops, 10Operations, 10ops-codfw: (Need by: End of July-2020 ) codfw:rack/setup/new management switches - https://phabricator.wikimedia.org/T253154 (10Papaul) [01:50:30] 10Traffic, 10Varnish, 10Analytics-Radar, 10Operations, 10Privacy: Add request_id to webrequest logs as well as other event records ingested into Hadoop - https://phabricator.wikimedia.org/T113817 (10Ottomata) :) [03:53:50] 10Traffic, 10OpenRefine, 10Operations, 10serviceops, and 2 others: Clients failing API login due to dependence on "Set-Cookie" header name casing - https://phabricator.wikimedia.org/T249680 (10Antigng) Just to mention that apache httpd does a camel casing by default when proxying back from an http2-talking... [07:17:43] 10netops, 10Analytics-Clusters, 10Operations, 10Patch-For-Review: Move netflow data to Eventgate Analytics - https://phabricator.wikimedia.org/T248865 (10elukey) Some high level issues that came up while talking about netflow on eventgate: * `pmacct` (the daemon that collects netflow data from routers) se... [08:00:24] 10Traffic, 10Desktop Improvements, 10Operations, 10Performance-Team (Radar): Turn off CDN cache for up to one week on several wikis for desktop improvements deployment - https://phabricator.wikimedia.org/T256750 (10ema) As @Krinkle pointed out, instead of turning off caching altogether we can invalidate th... [08:00:51] 10Traffic, 10Desktop Improvements, 10Operations, 10Performance-Team (Radar): CDN cache revalidation on several wikis for desktop improvements deployment - https://phabricator.wikimedia.org/T256750 (10ema) [12:38:26] 10netops, 10Operations, 10Patch-For-Review: fastnetmon spamming /var/log on netflow hosts leading to disk saturation - https://phabricator.wikimedia.org/T240658 (10ayounsi) 05Stalled→03Resolved a:03ayounsi Confirmed that the logs are now quieter, from ~42/min to ~4/10min right now in esams. Which I th... [13:18:35] 10netops, 10Analytics-Clusters, 10Operations, 10Patch-For-Review: Move netflow data to Eventgate Analytics - https://phabricator.wikimedia.org/T248865 (10Ottomata) > doesn't support emitting HTTP POST to an endpoint (like eventgate). Well if it doesn't support HTTP POST then you won't be moving it to Event... [13:49:06] 10Acme-chief, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Change CI image for acme-chief from Stretch to Buster - https://phabricator.wikimedia.org/T257456 (10hashar) [13:54:52] 10Acme-chief, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Change CI image for acme-chief from Stretch to Buster - https://phabricator.wikimedia.org/T257456 (10hashar) Turns out we can still have CI runs u... [13:58:08] 10Acme-chief, 10Patch-For-Review, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Change CI image for acme-chief from Stretch to Buster - https://phabricator.wikimedia.org/T257456 (10hashar) a:03hashar [14:46:27] 10Acme-chief, 10Patch-For-Review, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Change CI image for acme-chief from Stretch to Buster - https://phabricator.wikimedia.org/T257456 (10hashar) a:05hashar→0... [16:16:09] 10netops, 10DC-Ops, 10Operations, 10ops-eqiad, 10cloud-services-team (Hardware): (Need By: 2020-06-12) rack/setup/install WMCS 10G switches - https://phabricator.wikimedia.org/T251632 (10wiki_willy) @Cmjohnson - I chatted with Arzhel a bit earlier today, and he's going to get these dedicated 10g switches... [16:42:17] 10netops, 10Operations, 10ops-eqiad: (Need by: 2019-09-30) upgrade msw1-eqiad from EX4200 to EX4300 - https://phabricator.wikimedia.org/T225121 (10Cmjohnson) [16:44:18] 10netops, 10Operations, 10ops-eqiad: (Need by: 2019-09-30) upgrade msw1-eqiad from EX4200 to EX4300 - https://phabricator.wikimedia.org/T225121 (10Cmjohnson) 05Open→03Resolved netbox updated, old switch removed. updated cable id's. Resolving [17:02:18] 10netops, 10Operations, 10ops-eqiad: (Need by: 2019-09-30) upgrade msw1-eqiad from EX4200 to EX4300 - https://phabricator.wikimedia.org/T225121 (10ayounsi) 05Resolved→03Open I think there is something wrong: https://netbox.wikimedia.org/dcim/devices/50/ is the old one but still have all the cables https... [18:28:04] 10Traffic, 10Operations, 10Core Platform Team Workboards (Clinic Duty Team), 10MW-1.35-notes (1.35.0-wmf.39; 2020-06-30), 10Patch-For-Review: Move all purge traffic to kafka - https://phabricator.wikimedia.org/T250781 (10Pchelolo) 05Open→03Resolved a:03Pchelolo After deploying the latest config cha... [18:28:13] 10Traffic, 10Operations, 10Core Platform Team Workboards (Clinic Duty Team): Move wikitech purges to kafka - https://phabricator.wikimedia.org/T254828 (10Pchelolo) 05Open→03Resolved a:03Pchelolo [18:28:16] 10Traffic, 10Operations, 10Core Platform Team Workboards (Clinic Duty Team), 10MW-1.35-notes (1.35.0-wmf.39; 2020-06-30), 10Patch-For-Review: Move all purge traffic to kafka - https://phabricator.wikimedia.org/T250781 (10Pchelolo) [19:56:42] 10Traffic, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10Operations, and 2 others: Implement redirect for hide banner cookie issue - https://phabricator.wikimedia.org/T251780 (10MBeat33) p:05Lowest→03High Post meeting followup, to request a Cookie-Setting-Thing (CST) URL that Donor Ser... [20:06:53] 10Traffic, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10Operations, and 2 others: Implement redirect for hide banner cookie issue - https://phabricator.wikimedia.org/T251780 (10TSkaff) @MBeat33 I just emailed Peter to ask what he thinks about the complexity of this; I could have just ping... [23:28:05] 10Traffic, 10Operations, 10observability: Collect client network errors, deprecation, intervention and crash reports - https://phabricator.wikimedia.org/T207860 (10CDanis)