[00:50:50] FIRING: [3x] MysqlReplicationThreadCountTooLow: MySQL instance db1171:13318 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [00:55:50] FIRING: [5x] MysqlReplicationThreadCountTooLow: MySQL instance db1150:13314 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [02:00:50] FIRING: [5x] MysqlReplicationThreadCountTooLow: MySQL instance db1150:13314 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [03:15:50] FIRING: [3x] MysqlReplicationThreadCountTooLow: MySQL instance db1150:13314 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [03:25:50] FIRING: [3x] MysqlReplicationThreadCountTooLow: MySQL instance db1150:13314 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [03:35:50] RESOLVED: [2x] MysqlReplicationThreadCountTooLow: MySQL instance db1150:13314 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [05:15:50] FIRING: MysqlReplicationThreadCountTooLow: MySQL instance db1150:13313 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&refresh=1m&var-job=All&var-server=db1150&var-port=13313 - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [05:19:44] ^ looks like this wasn't fixed [05:19:50] arnaudb: ^ [05:35:50] FIRING: [2x] MysqlReplicationThreadCountTooLow: MySQL instance db1150:13313 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [06:45:50] RESOLVED: MysqlReplicationThreadCountTooLow: MySQL instance db1150:13313 has replication issues. - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&refresh=1m&var-job=All&var-server=db1150&var-port=13313 - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationThreadCountTooLow [07:13:43] yep I saw the email as well, they are way more readable than what IRC is able to display :D i've been able to spot that I should exclude those hosts from critical severity as well