[01:44:57] <wikibugs>	 10DBA: "chemical" major mime type was never added to production database - https://phabricator.wikimedia.org/T277354 (10Ladsgroup)
[07:53:21] <wikibugs>	 10DBA, 10Data-Persistence-Backup: recover-mariadb should use logging (logger) to indicate actions taken - https://phabricator.wikimedia.org/T277162 (10Rohitesh-Kumar-Jain) a:03Rohitesh-Kumar-Jain
[09:54:26] <wikibugs>	 10Data-Persistence-Backup, 10SRE-tools: Make recover-dump show the time taken - https://phabricator.wikimedia.org/T277160 (10h.krishna) a:03h.krishna
[12:33:37] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on pc2008 is CRITICAL: 2.8 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104
[12:38:15] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on pc2008 is OK: (C)2 ge (W)1 ge 0.2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104
[18:45:50] <Amir1>	 I'm going to run some schema changes on wikitech (labswiki)
[18:53:04] <Amir1>	 !log run schema changes for varbinary on wikitech (T269348)
[18:53:04] <stashbot>	 Amir1: Not expecting to hear !log here
[18:53:04] <stashbot>	 T269348: wikitech database has almost all of its varbinary fields wrong - https://phabricator.wikimedia.org/T269348
[19:04:25] <wikibugs>	 10DBA, 10wikitech.wikimedia.org, 10cloud-services-team (Kanban): wikitech database has almost all of its varbinary fields wrong - https://phabricator.wikimedia.org/T269348 (10Ladsgroup) All changed now to binary in labstestwiki, I let it wait for a couple of days to see if things break and if not, I'll chang...
[21:02:34] <wikibugs>	 10DBA, 10MediaWiki-General, 10Patch-For-Review, 10Schema-change: Convert primary key integers and references thereto from int to bigint (unsigned) - https://phabricator.wikimedia.org/T63111 (10Ladsgroup)
[21:15:23] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on pc2008 is CRITICAL: 2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104
[21:17:49] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on pc2008 is OK: (C)2 ge (W)1 ge 0.2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104
[22:50:57] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on pc2008 is CRITICAL: 2.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104
[22:58:05] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on pc2008 is OK: (C)2 ge (W)1 ge 0.8 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104
[23:31:35] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on pc2008 is CRITICAL: 2.8 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104
[23:33:57] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on pc2008 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2008&var-port=9104