[05:29:55] <jinxer-wm>	 FIRING: [2x] SystemdUnitFailed: pt-heartbeat-wikimedia.service on pc1022:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[05:30:15] <marostegui>	 ^ expected, downtime expired and I am decommissioning this host
[05:34:55] <jinxer-wm>	 RESOLVED: [2x] SystemdUnitFailed: pt-heartbeat-wikimedia.service on pc1022:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[09:42:48] <jynus>	 I will silence notifications from this task after https://phabricator.wikimedia.org/T426633#11941406 but feel free to ping me here or any other way if I missed anything
[09:45:42] <jynus>	 I will soon today have worked 40 hours this week, so tomorrow I will just show up to say hello, see notifs and for a meeting
[09:48:14] <dhinus>	 doing some clouddb reboots for T426563
[11:46:02] <jynus>	 expect some lag on backup1-codfw, I am doing one of the last large maintenances for the migration
[11:46:12] <jynus>	 I will downtime lag until tomorrow
[12:55:18] <cezmunsta>	 Doing a switchover for x3 codfw
[12:56:20] <Amir1>	 🍀
[13:25:38] <federico3>	 the new version of rolling_restart has been running since the morning and looped across replicas in es6 and es7 in codfw and eqiad as expected
[14:45:42] <Amir1>	 marostegui: I think you're still doing parsercache refreshes, if you're done, can you let me know so I start the switch to nftables. Otherwise, I can pick a random replica in s5 or s6
[14:49:46] <marostegui>	 Amir1: you can do pc1 and pc2 if you want
[15:06:47] <Amir1>	 Thanks!
[15:44:40] <federico3>	 the es6 and es7 reboots are completed, I'll start s* with the new script on Monday unless we want to run one of the s* sections in the evening
[16:29:00] <Amir1>	 pc2022 is being rebooted
[16:29:06] <Amir1>	 with nftables
[16:32:41] <jynus>	 reimage --new-vlan --nftables --upgrade-os --new-dba --touch-grass
[17:16:33] <Amir1>	 ladsgroup@pc2022:~$ sudo iptables -L
[17:16:33] <Amir1>	 sudo: iptables: command not found
[17:16:40] <Amir1>	 \o/
[17:26:55] <jynus>	 hopefully "nft list ruleset" works, and you haven't just setup a host without a firewall ;-)
[19:05:49] <Amir1>	 jynus: just saw this. Yeah yeah. It is using nftables and works 💪