[00:54:13] 10Traffic, 10Operations, 10ops-esams: cp3037 is currently unreachable - https://phabricator.wikimedia.org/T222041 (10ayounsi) >>! In T222041#5260488, @Papaul wrote: > cp3037 is dead, it needs new mainboard since it is out of warranty better decommission it. Based on that comment I changed the status of tha... [08:50:21] 10Traffic, 10Operations: ATS is currently adding its own server header - https://phabricator.wikimedia.org/T224119 (10ema) 05Open→03Resolved a:03ema ATS now sets `Server` only if missing in the origin server response. Also, Varnish now does not send `Via` any longer (the header wasn't used at all). [13:27:20] 10Traffic, 10Operations, 10Zero, 10Patch-For-Review: Zero VCL removal - https://phabricator.wikimedia.org/T213769 (10ema) 05Stalled→03Open [13:42:29] 10Traffic, 10Discovery, 10Operations, 10WMDE-Analytics-Engineering, and 3 others: Allow access to wdqs.svc.eqiad.wmnet on port 8888 - https://phabricator.wikimedia.org/T176875 (10Ottomata) @Addshore, just saw T218710 and clicked through to here. If you use https://wikitech.wikimedia.org/wiki/HTTP_proxy, y... [14:47:38] 10netops, 10Operations: RPKI Validation - https://phabricator.wikimedia.org/T220669 (10ayounsi) Confirmed with a given test peer that was sending us RPKI invalids and unknown. We now only receive the unknown. And for a given invalid prefix that we used to receive via peering is now going through transit. [15:31:09] 10Traffic, 10Discovery, 10Operations, 10WMDE-Analytics-Engineering, and 3 others: Allow access to wdqs.svc.eqiad.wmnet on port 8888 - https://phabricator.wikimedia.org/T176875 (10elukey) Not really, I wish myself from the past added more info. I asked to @ayounsi and he didn't come up with a reason not to,... [15:41:17] 10netops, 10Operations: RPKI Validation - https://phabricator.wikimedia.org/T220669 (10MelchiorAelmans) Good job team!! [15:49:16] 10netops, 10Operations: RPKI Validation - https://phabricator.wikimedia.org/T220669 (10ayounsi) Thanks! We now reject RPKI invalids on all our private/public peering sessions. Next step is to review/merge https://gerrit.wikimedia.org/r/c/520337/ so we have almost real-time visibility on which % of our traffi... [16:24:03] 10Traffic, 10Operations, 10Wikimedia-Logstash, 10Patch-For-Review, 10User-Addshore: Changing Kibana filters is ridiculously slow - https://phabricator.wikimedia.org/T189333 (10Addshore) [20:20:06] 10netops, 10Analytics, 10Operations, 10LDAP: LDAP ldap-ro.eqiad.wikimedia.org not reachable from Analytics VLAN - https://phabricator.wikimedia.org/T227611 (10Ottomata) [20:25:36] 10netops, 10Analytics, 10Operations, 10LDAP: LDAP ldap-ro.eqiad.wikimedia.org not reachable from Analytics VLAN - https://phabricator.wikimedia.org/T227611 (10Ottomata) p:05Triage→03High [21:31:02] 10Traffic, 10MediaWiki-extensions-CentralAuth, 10Operations, 10TimedMediaHandler, and 3 others: Consistent HTTP 503 Error on some urls for some logged-in users (CentralAuth Set-Cookie storm) - https://phabricator.wikimedia.org/T226840 (10Krinkle)