[09:34:16] 10Traffic, 10ORES, 10Operations, 10Scoring-platform-team: Pass on name of the node serving ORES requests as response header to the user - https://phabricator.wikimedia.org/T204600 (10ema) p:05Triage>03Normal [10:52:05] bblack: lvs101[12] have puppet disabled with the log "bblack-decom", puppet disabled makes them unreachable from the new Cumin server (the the IP is granted to ferm via puppet). it's probably unsafe to enable puppet? If so, I can add the new IP manually to the ferm conf so that the server is reachable via Cumin until eventually decomissioned [11:26:00] 10Traffic, 10ORES, 10Operations, 10Scoring-platform-team, 10Patch-For-Review: Pass on name of the node serving ORES requests as response header to the user - https://phabricator.wikimedia.org/T204600 (10ema) 05Open>03Resolved a:03ema Done: ``` $ curl -v https://ores.wikimedia.org/v3/scores/wikidat... [11:35:31] 10Traffic, 10ORES, 10Operations, 10Scoring-platform-team, 10Patch-For-Review: Pass on name of the node serving ORES requests as response header to the user - https://phabricator.wikimedia.org/T204600 (10Ladsgroup) Thank you! [11:56:31] 10Wikimedia-Apache-configuration, 10Wikidata, 10Wikimedia-Site-requests, 10wikidata-tech-focus, and 4 others: wikidata.org/entity/Q12345 should do content negotiation immediately, instead of redirecting to wikidata.org/wiki/Special:EntityData/Q36661 first - https://phabricator.wikimedia.org/T119536 (10Addsh... [11:57:01] 10Wikimedia-Apache-configuration, 10Wikidata, 10Wikimedia-Site-requests, 10wikidata-tech-focus, and 4 others: wikidata.org/entity/Q12345 should do content negotiation immediately, instead of redirecting to wikidata.org/wiki/Special:EntityData/Q36661 first - https://phabricator.wikimedia.org/T119536 (10Addsh... [12:36:18] moritzm: we can *probably* re-enable puppet on those, but should check in with XioNoX whether the IPs have been removed from pybal bgp config stuff on the routers first. [12:36:37] that's the only real risk, that for some reason they're still in router config and might steal traffic routing [12:36:41] (if pybal were to start) [12:52:36] 10Traffic, 10Varnish, 10Maps-Sprint, 10Operations, and 3 others: Tilerator should purge Varnish cache - https://phabricator.wikimedia.org/T109776 (10Mholloway) Confirmed that events are being sent by Tilerator upon tile generation, and received and produced to kafka by `eventlogging-service` on beta: ```... [13:12:47] bblack: ack, thanks [13:13:24] ^ XioNoX: when you can find the time, could you check the above, please? [13:40:41] that and in general, we should finish decomming them heh [13:40:58] lvs1007-12 were never in real production use and never will be, there's nothing fundamentally blocking their full decom [15:31:33] 10netops, 10Operations: Enable cumin2001 in router ACLs - https://phabricator.wikimedia.org/T204730 (10MoritzMuehlenhoff) [16:23:25] 10netops, 10Operations: Enable cumin2001 in router ACLs - https://phabricator.wikimedia.org/T204730 (10ayounsi) 05Open>03Resolved a:03ayounsi Done! [16:37:53] looking [16:38:52] bblack, moritzm, it prompts for a password when trying to ssh to lvs1011 [16:39:16] (same with 1012) [16:40:17] yeah I'm sure they're half-configured or whatever [16:40:37] but the ??? is whether their IPs are allowed by the routers (to speak BGP as a pybal) [16:41:10] I don't remember where we last left that. At one point the lvs1007-12 set was configured on the routers, I think. Probably shouldn't be anymore. [16:41:52] moritzm, bblack, all good [16:42:21] no sign of their IPs, and no down BGP sessions [16:59:56] ok, thanks! [17:57:40] 10netops, 10Operations, 10ops-eqiad: Ensure scs-c1-eqiad:eth1 is not connected - https://phabricator.wikimedia.org/T204743 (10ayounsi) p:05Triage>03Low [18:37:39] 10Traffic, 10Operations, 10Performance-Team, 10Wikimedia-General-or-Unknown, and 2 others: Search engines continue to link to JS-redirect destination after Wikipedia copyright protest - https://phabricator.wikimedia.org/T199252 (10Imarlier) @Nemo_bis Regarding your comment above, about Google not respectin... [18:41:32] 10Traffic, 10Operations, 10Performance-Team, 10Wikimedia-General-or-Unknown, and 2 others: Search engines continue to link to JS-redirect destination after Wikipedia copyright protest - https://phabricator.wikimedia.org/T199252 (10Imarlier) @kaldari The >500,000 number is based on Google Search Console, wh... [21:42:42] 10Traffic, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission radon - https://phabricator.wikimedia.org/T202040 (10RobH) a:03Cmjohnson Please note I did all the decom steps on the wrong task, putting them on radium task not radon. This is ready for disk wipe. >>! In T203861#4... [21:43:02] 10Traffic, 10Operations, 10decommission, 10ops-eqiad: Decommission radon - https://phabricator.wikimedia.org/T202040 (10RobH) [22:24:59] 10netops, 10Operations, 10ops-ulsfo: cr2-ulsfo crash - https://phabricator.wikimedia.org/T204782 (10ayounsi) p:05Triage>03High [23:33:08] 10netops, 10Operations, 10ops-ulsfo, 10Patch-For-Review: cr2-ulsfo crash - https://phabricator.wikimedia.org/T204782 (10ayounsi)