[07:57:49] 10netops, 06Infrastructure-Foundations, 10SRE-tools: Evaluate automatic MAC-based DHCP for production servers - https://phabricator.wikimedia.org/T396712 (10Volans) 03NEW p:05Triage→03Medium [08:02:13] 10netops, 06Infrastructure-Foundations, 10SRE-tools: Evaluate automatic MAC-based DHCP for production servers - https://phabricator.wikimedia.org/T396712#10907935 (10Volans) I've run some custom code with spicerack-shell and get the audit data for the whole fleet and comparing the MAC address retrieved from... [12:57:25] FIRING: SystemdUnitFailed: prometheus_liberica_cp_checks.service on lvs7002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [12:59:14] 06Traffic: Replace Digicert TLS certs with Google Trust Services ones - https://phabricator.wikimedia.org/T395131#10908991 (10Vgutierrez) [13:01:29] 06Traffic, 10Liberica: Test katran forwarding plane on lvs1013 - https://phabricator.wikimedia.org/T395228#10908999 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=5df8241b-6dee-4131-8bad-8d9a73984d08) set by vgutierrez@cumin1003 for 1 day, 0:00:00 on 1 host(s) and their services with r... [13:21:18] 06Traffic: Upgrade to ATS 9.2.10 - https://phabricator.wikimedia.org/T390912#10909080 (10ssingh) [13:23:06] 10netops, 06cloud-services-team, 06Infrastructure-Foundations, 10Toolforge: [infra] Reports of slow connectivity from APAC - https://phabricator.wikimedia.org/T395135#10909085 (10cmooney) The latency is also reduced when I check for it here (there are no manual overrides of the traffic path in place either... [13:26:18] 06Traffic, 10Liberica: liberica forwarding plane fails to start on systems with 48 CPUs using katran - https://phabricator.wikimedia.org/T396751 (10Vgutierrez) 03NEW [13:28:19] 10netops, 06Infrastructure-Foundations, 06SRE: Stage and configure new Juniper switches in codfw rows E/F - https://phabricator.wikimedia.org/T394021#10909127 (10cmooney) 05Open→03Resolved [13:29:21] 06Traffic: Upgrade to ATS 9.2.10 - https://phabricator.wikimedia.org/T390912#10909134 (10ssingh) a:03BCornwall [13:43:07] 10netops, 06Infrastructure-Foundations, 06SRE: Netbox: Use LAG interface MAC address field to store LACP system-id for MC-LAG - https://phabricator.wikimedia.org/T392056#10909207 (10cmooney) p:05Medium→03Low > We could (ab)use the FHRP group feature to group members of a MC-LAG and add common variables... [13:45:13] 10netops, 06Infrastructure-Foundations, 06SRE: Consolidate Automation Templates for DC Switches - https://phabricator.wikimedia.org/T312635#10909214 (10cmooney) 05Open→03Resolved [13:48:53] 10netops, 06Infrastructure-Foundations, 06SRE: Sub-optimal cloud routing for WMCS in eqiad when link fails - https://phabricator.wikimedia.org/T367203#10909233 (10cmooney) 05Open→03Resolved This problem is now resolved as we are using IBGP with next-hops announced as the loopbacks of each switch, and... [13:51:56] 10netops, 06Infrastructure-Foundations, 06SRE: Create cookbook to set up ganeti host network - https://phabricator.wikimedia.org/T378346#10909245 (10cmooney) 05Open→03Declined [14:07:53] 06Traffic, 10Liberica, 13Patch-For-Review: liberica forwarding plane fails to start on systems with 48 CPUs using katran - https://phabricator.wikimedia.org/T396751#10909289 (10Vgutierrez) 05Open→03Resolved [14:38:31] 06Traffic, 06Movement-Insights, 10Data-Engineering (Q4 2025 April 1st - June 30th): NEW BUG REPORT: Investigate rise in May 2025 Reader metrics - https://phabricator.wikimedia.org/T395934#10909406 (10OSefu-WMF) [15:13:26] 06Traffic, 10Experimentation Lab (Experiment Platform Sprint 7): EventGate: Investigate data loss during the SDS 2.4.11 Synthetic A/A Test experiment - https://phabricator.wikimedia.org/T396474#10909604 (10phuedx) [15:42:24] 10netops, 06Infrastructure-Foundations, 10SRE-tools: Evaluate automatic MAC-based DHCP for production servers - https://phabricator.wikimedia.org/T396712#10909876 (10ayounsi) [16:36:16] 06Traffic, 10Experimentation Lab (Experiment Platform Sprint 7): EventGate: Investigate data loss during the SDS 2.4.11 Synthetic A/A Test experiment - https://phabricator.wikimedia.org/T396474#10910213 (10Milimetric) [16:36:40] vgutierrez: dumb question, did we only ship the maxmind isps table to haproxy or did we also send city? [16:36:58] isp.mmdb one [16:37:14] any reason why we couldn't ship city or even just country as well? [16:37:33] nope [16:38:04] but if we bring geoip capabilities to haproxy I'd like to move the GeoIP cookie there as well [16:38:09] 06Traffic, 10Experimentation Lab (Experiment Platform Sprint 7): EventGate: Investigate data loss during the SDS 2.4.11 Synthetic A/A Test experiment - https://phabricator.wikimedia.org/T396474#10910225 (10Milimetric) 05Open→03Resolved a:03Milimetric [16:40:53] sure :D [16:56:57] 06Traffic, 10Liberica: Switch to katran as forwarding plane on non-core DCs - https://phabricator.wikimedia.org/T396561#10910378 (10Vgutierrez) [17:29:44] 06Traffic: varnish 7.1.1-2~bpo11+wmf1 crash - https://phabricator.wikimedia.org/T396581#10910496 (10BCornwall) [18:33:44] 06Traffic, 10DNS, 06serviceops, 06SRE: Create redirect from tj.*.org to tg.*.org - https://phabricator.wikimedia.org/T393803#10910716 (10Dzahn) works for me now. thanks @jasmine_ for deploying my patches!:) ` curl -vv https://tj.wikipedia.org ..

The document has moved