[06:14:09] <wikibugs>	 10DBA, 10Operations, 10ops-codfw: db2140 crashed due to HW memory errors - https://phabricator.wikimedia.org/T271084 (10Marostegui) 05Open→03Resolved Data was checked and came back up clean. Closing this - thanks for getting on this so fast Papaul!
[06:27:23] <wikibugs>	 10DBA, 10SRE-tools, 10IPv6: Some Data Persistence DB clusters apparently do not support IPv6 - https://phabricator.wikimedia.org/T271140 (10Marostegui) The databases are mostly blocked on the grants audit and cleanup, which is not an easy task T270101
[06:37:21] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Increase size of content_models.model_id - https://phabricator.wikimedia.org/T270053 (10Marostegui)
[06:39:22] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Increase size of content_models.model_id - https://phabricator.wikimedia.org/T270053 (10Marostegui) s5 is done: ` # /home/marostegui/section s5 | while read host port; do echo "$host:$port"; mysql.py -h$host:$port dewiki -e "show create table content_models\G" | grep "model_i...
[06:39:36] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Increase size of content_models.model_id - https://phabricator.wikimedia.org/T270053 (10Marostegui)
[06:43:51] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Increase size of content_models.model_id - https://phabricator.wikimedia.org/T270053 (10Marostegui) wikitech done: ` root@db1128.eqiad.wmnet[labswiki]> ALTER TABLE /*_*/content_models     ->   MODIFY model_id INT AUTO_INCREMENT NOT NULL; Query OK, 9 rows affected (0.006 sec)...
[06:44:16] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Increase size of content_models.model_id - https://phabricator.wikimedia.org/T270053 (10Marostegui)
[06:45:23] <wikibugs>	 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) On-going transfer from db1074 to db1155
[06:47:12] <wikibugs>	 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) {P13648}
[08:38:26] <wikibugs>	 10DBA, 10Orchestrator, 10User-Kormat: Enable report_host on candidate masters - https://phabricator.wikimedia.org/T271106 (10Marostegui) ` # for i in es2020 es2024 db2127; do mysql.py -h$i -e "select @@report_host";done +--------------------+ | @@report_host      | +--------------------+ | es2020.codfw.wmnet...
[08:38:34] <wikibugs>	 10DBA, 10Orchestrator, 10User-Kormat: Enable report_host on candidate masters - https://phabricator.wikimedia.org/T271106 (10Marostegui)
[09:03:53] <wikibugs>	 10DBA, 10Wikimedia-Rdbms, 10Goal, 10MW-1.36-notes (1.36.0-wmf.20; 2020-12-01), and 5 others: FY18/19 TEC1.6 Q4: Improve or replace the usage of GTID_WAIT with pt-heartbeat in MW - https://phabricator.wikimedia.org/T221159 (10Kormat)
[10:29:27] <wikibugs>	 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) s2 has been brought up on db1155, so far no InnoDB errors - it was cloned from the sanitarium master, db1074. Sanitization done. Triggers and filters working fine InnoDB compre...
[10:29:34] <wikibugs>	 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui)
[12:24:17] <wikibugs>	 10DBA, 10Orchestrator, 10User-Kormat: Enable report_host on candidate masters - https://phabricator.wikimedia.org/T271106 (10Marostegui)
[13:36:23] <wikibugs>	 10DBA, 10SRE-tools, 10IPv6: Some Data Persistence DB clusters apparently do not support IPv6 - https://phabricator.wikimedia.org/T271140 (10fgiunchedi) >>! In T271140#6720449, @LSobanski wrote: > - ms-be and ms-fe are media storage and not managed by us (yet). @fgiunchedi, could you comment on these?  I'm fa...