[09:39:25] FIRING: SystemdUnitFailed: pt-heartbeat-wikimedia.service on db1125:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [10:07:02] that's test-s4 [10:09:41] > Sep 15 09:28:27 db1125 mysqld[1634436]: 2024-09-15 9:28:27 7194 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type varbinary(255), found type long> [10:14:49] internet says mysql_upgrade should fix it but it didn't [10:19:00] I can do alter table and turn it into varbinary(255) [10:19:05] but rather not [12:09:25] FIRING: [3x] SystemdUnitFailed: pt-heartbeat-wikimedia.service on db1125:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [12:24:25] FIRING: [3x] SystemdUnitFailed: pt-heartbeat-wikimedia.service on db1125:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [16:24:25] FIRING: SystemdUnitFailed: pt-heartbeat-wikimedia.service on db1125:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [20:24:25] FIRING: SystemdUnitFailed: pt-heartbeat-wikimedia.service on db1125:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed