[09:30:19] <elukey>	 Emperor: o/ I tried my best to put strain on the apus' radosgw but so far everything seems blazing fast (especially dropping and syncing)
[09:30:35] <elukey>	 so the new Reef version seems really good
[09:48:40] <Emperor>	 that's good :) Want to retry moving /restricted to it again at some point then?
[10:10:02] <elukey>	 yeah I already filed the patches :)
[10:11:36] <Emperor>	 🤞
[12:59:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: orchestrator.service on dborch1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[13:04:51] <federico3>	 yep, it's being set up
[13:04:58] <jynus>	 I think I am going to migrate backup to xfs, I am getting consistently better performance
[13:05:04] <jynus>	 *backups
[13:19:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: apache2.service on dborch1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[13:29:25] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: orchestrator.service on dborch1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[14:43:22] <jinxer-wm>	 FIRING: PuppetFailure: Puppet has failed on dborch1003:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure
[15:31:25] <jynus>	 It's so nice to see rsync report writing speeds of 693.40MB/s on HDs... of no real bytes
[15:32:03] <jynus>	 I guess it's a measure of how fast it can read metadata?
[16:14:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: wmf_auto_restart_apache2.service on dborch1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[16:19:30] <federico3>	 the setup of orchestrator behind the CDN might take longer so I'll disable the notifications in the meantime 
[16:24:51] <Emperor>	 thanks :)
[20:14:40] <jinxer-wm>	 FIRING: SystemdUnitFailed: wmf_auto_restart_apache2.service on dborch1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed