[05:12:24] 10DBA: Compress new Wikibase tables - https://phabricator.wikimedia.org/T232446 (10Marostegui) [05:13:19] 10DBA: Compress new Wikibase tables - https://phabricator.wikimedia.org/T232446 (10Marostegui) 05Open→03Stalled Only the primary database masters are pending. To do those we'd either need a DC switchover or switch them individually. [07:34:49] 10DBA, 10cloud-services-team (Kanban): Reimage labsdb1011 to Buster and 10.4 - https://phabricator.wikimedia.org/T249188 (10Marostegui) labsdb1011 was reimaged yesterday with Buster and 10.4 again after the RAID got destroyed and recreated (T251481). As we cannot reclone it from labsdb1012 just yet (as it is r... [07:38:44] Finished dump at: 2020-04-30 21:18:41 [07:39:15] jynus: go back to your holidays [07:39:38] just telling you to start loading that in 18 threads so it finishes by tuesday :-D [07:39:48] where is that dump? [07:40:27] labsdb1009:/srv/backups/dumps/ongoing/dump.wikireplica.2020-04-30--06-46-23 [07:40:46] jynus: excellent! now go back to your holidays or I will ban you! [07:40:52] thank you :) [10:13:31] 10DBA, 10Dumps-Generation, 10MediaWiki-extensions-CodeReview, 10Security-Team: Publish SQL dumps of CodeReview tables - https://phabricator.wikimedia.org/T243055 (10ArielGlenn) I don't know the extension either. Hey @brion :-D In the meantime I guess these should get added to the https://dumps.wikimedia.o... [10:16:06] 10DBA, 10cloud-services-team (Kanban): Reimage labsdb1011 to Buster and 10.4 - https://phabricator.wikimedia.org/T249188 (10Marostegui) Jaime let me know that the mysqldump already finished, so I have stopped and disabled replication and I have started to import it on labsdb1011 with 20 threads. For what is wo... [11:13:27] 10DBA: Drop wb_terms in production from s4 (commonswiki, testcommonswiki), s3 (testwikidatawiki), s8 (wikidatawiki) - https://phabricator.wikimedia.org/T248086 (10Marostegui) [13:34:40] marostegui: Look at this: https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&refresh=5m&var-server=db1104&var-datasource=eqiad%20prometheus%2Fops&var-cluster=mysql&fullscreen&panelId=12&from=1587714109162&to=1588340016719 [13:34:53] with the compressions it went from 70% to 35% [13:34:59] yeah [13:35:11] Amir1: keep in mind that it is compression + defragmentation [13:35:24] Fragmentation will show up sooner or later again [13:36:33] Amir1: that also includes the drop of our wb_terms friend [13:36:43] yup [13:36:46] \o/ [13:36:50] which make it gain another 20% [13:36:51] :) [13:59:49] 10DBA, 10Operations: Upgrade and restart s3 and s7 primary DB master: Thu 7th May - https://phabricator.wikimedia.org/T251158 (10Marostegui) Added the slot on the deployment's page [13:59:53] 10DBA, 10Operations: Upgrade and restart s5 and s6 primary DB master: Tue 5th May - https://phabricator.wikimedia.org/T251154 (10Marostegui) Added the slot on the deployment's page [14:20:04] 10DBA, 10Operations: Upgrade and restart s3 and s7 primary DB master: Thu 7th May - https://phabricator.wikimedia.org/T251158 (10Marostegui) Day before: - Install the 10.1.43-2 package on both masters (db1123 and db1086) Maintenance day: - Silence all hosts in s3 and s7 - Set read only on s3 and s7: ` dbctl -... [14:22:01] cdanis: If you can review the dbctl commands, that'd be nice: https://phabricator.wikimedia.org/T251154#6096408 [14:23:07] marostegui: lgtm! [14:23:17] cdanis: thanks :)