[00:45:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[04:45:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[08:45:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[09:41:49] <dhinus>	 federico3: checked nupwiki_p in clouddb* and it's looking good now
[09:42:12] <dhinus>	 I re-ran the cookbook and the views were created successfully T390714
[09:42:12] <stashbot>	 T390714: [wikireplicas] Create views for new wiki nupwiki - https://phabricator.wikimedia.org/T390714
[09:42:14] <federico3>	 dhinus: thanks! Can we close the related tasks?
[09:43:08] <dhinus>	 yep, just resolved that one, the other one I think was already resolved?
[09:43:19] <dhinus>	 T390710
[09:43:20] <stashbot>	 T390710: Prepare and check storage layer for nupwiki - https://phabricator.wikimedia.org/T390710
[09:43:22] <federico3>	 yup, thank you
[09:43:39] <federico3>	 I'll chase the related CR
[09:44:18] <dhinus>	 btw, could you review this somewhat related patch? https://gerrit.wikimedia.org/r/c/operations/puppet/+/1137019
[10:49:09] <federico3>	 marostegui, Amir1: can I move on with more host updates and DC master updates for https://phabricator.wikimedia.org/T391056 ?
[10:59:23] <hnowlan>	 o/ I'd like to migrate purge_parsercache_p1 to k8s today. It won't run until tomorrow morning so it'll have no impact until then. Any objections? https://gerrit.wikimedia.org/r/c/operations/puppet/+/1139422 
[12:00:12] <Amir1>	 federico3: for s1 and s8, you need master switchover, which I suggest you bundle with reboots
[12:06:05] <Amir1>	 hnowlan: no objections from my side. I'll monitor
[12:45:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[14:18:54] <hnowlan>	 Amir1: thank you! <3
[16:44:12] <urandom>	 This is a very satisfying graph:
[16:44:14] <urandom>	 https://usercontent.irccloud-cdn.com/file/iT1vMtnz/image.png
[16:44:42] <urandom>	 It starts out all 😱, and then ends up all 😴
[16:46:55] <jinxer-wm>	 FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[18:38:59] <sobanski>	 urandom: is this with the new drives?
[18:51:04] <urandom>	 sobanski: yes
[18:51:29] <urandom>	 I just added them in an ad hoc way, I still need to go back and reimage once I have a workable JBOD partman recipe
[18:52:24] <urandom>	 (at which point we'll get even more storage out of them)
[20:46:55] <jinxer-wm>	 FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed