[00:52:33] FIRING: SystemdUnitFailed: update-ubuntu-mirror.service on mirror1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [02:49:38] RESOLVED: SystemdUnitFailed: update-ubuntu-mirror.service on mirror1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:25:59] I'm upgrading Routinator in eqiad now [09:26:25] thx! [09:36:05] 10netbox, 06Infrastructure-Foundations: Netbox: add IPAddress validator to ensure unique dns_name - https://phabricator.wikimedia.org/T428546#12003795 (10cmooney) 05Open→03Resolved a:03cmooney IPAddress validator updated to prevent this now. [09:58:19] 10netops, 06Infrastructure-Foundations, 13Patch-For-Review: Add (some) collection for Nokia SR-Linux components - https://phabricator.wikimedia.org/T428685#12003881 (10cmooney) >>! In T428685#12003236, @ayounsi wrote: >> I can't remember exactly the issue there, > It was causing the `sr_oc_mgmt_server` deamo... [10:17:51] 10netops, 06Infrastructure-Foundations, 06ServiceOps new: codfw: rack A5 maintenance - https://phabricator.wikimedia.org/T428020#12003962 (10jijiki) [10:59:19] 10netops, 06Infrastructure-Foundations, 06ServiceOps new: codfw: rack A5 maintenance - https://phabricator.wikimedia.org/T428020#12004114 (10jcrespo) Re: **backup2013**, it needs no special treatment other than downtime, it has no issue with a temporary network maintenance unless it gets extended for a few d... [11:39:39] dear foundations, I am trying to reimage a server and ipmi.py in spicerack blew up [11:39:49] would you like me to file a task, or paste the error here? [11:40:17] and I think it is simply that it couldnt connect [11:44:00] nevermind, it is a Error: Unable to establish IPMI v2 / RMCP+ session [11:59:52] 10Mail, 06Infrastructure-Foundations, 10Znuny: DKIM / DMARC for domains on VRTS - https://phabricator.wikimedia.org/T428540#12004410 (10Arnoldokoth) Looks similar to T369443 @Krd Take a look and maybe merge them? [13:13:04] effie: did you solve? [13:27:48] yeah see -sre, tx! [16:26:42] 10netops, 06Infrastructure-Foundations, 06SRE: Network QoS: use the 'CS1' DSCP code point for low-priority instead of AF41 - https://phabricator.wikimedia.org/T424640#12005652 (10cmooney) 05Open→03Resolved All work on this is now complete.