[07:07:31] 10DBA: enwiki database on db1065 missing rev_timestamp index - https://phabricator.wikimedia.org/T158454#3037574 (10Marostegui) Hello, We are aware of those differences, see: T132416. Unfortunately, it is not an easy ALTER table, given the size of the DB it can take up to 4-5 days to get it altered. We have be... [08:37:32] 10DBA, 10fundraising-tech-ops: fundraising database tuning - https://phabricator.wikimedia.org/T158446#3039787 (10Marostegui) Hey @Jgreen Thanks for all the information. Some points - The hardware isn't too powerful (I guess that is the reason you are buying new one). By increasing the memory so much, from... [10:02:43] 10DBA, 13Patch-For-Review: Deploy gtid_domain_id flag in our mysql hosts - https://phabricator.wikimedia.org/T149418#3040047 (10Marostegui) The patch to enable gtid_domain_id for hosts in s6 has been deployed. As this only takes effect upon restart, I have manually changed it on all of them (reminder, this val... [13:36:37] 10DBA: Install and reimage dbstore1001 as jessie - https://phabricator.wikimedia.org/T153768#3040579 (10Marostegui) I have started the transfer from dbstore1001:/srv/backups (the last two generated backups (find -mtime -14) to dbstore2001:/srv/backup/dbstore1001. It is being encrypted too. [13:48:26] 10DBA, 13Patch-For-Review: run pt-table-checksum before decommissioning db1015, db1035,db1044,db1038 - https://phabricator.wikimedia.org/T154485#3040625 (10Marostegui) The idea would be to run the pt-table-checksum with the following tables on the same batch for nlwiki: ``` -rw-rw---- 1 mysql mysql 112K Feb 2... [14:02:19] 10DBA, 13Patch-For-Review: Rampant differences in indexes on enwiki.revision across the DB cluster - https://phabricator.wikimedia.org/T132416#3040725 (10Marostegui) I have started to ALTER db2048 enwiki.revision with the following command: `./osc_host.sh --host=db2048.codfw.wmnet --port=3306 --db=enwiki --tab... [14:21:57] 10DBA: Install and reimage dbstore1001 as jessie - https://phabricator.wikimedia.org/T153768#3040773 (10Marostegui) In order to backup `dbstore1001:/srv/sqldata` we'd need 4.3T available (probably less as it would be compressed) but still, quite a big amount of space. The only hosts with enough space within eqia... [14:24:05] 10DBA, 10Wiki-Loves-Monuments-Database, 13Patch-For-Review: mysqldump is timing out preventing all tables from being included in the dump - https://phabricator.wikimedia.org/T138517#3040791 (10jcrespo) Toolsdb were finally upgraded to MariaDB 10/jessie at T157358 last week , so the version bump is not longer... [14:30:08] the cross dc transfer between dbstore1001 and dbstore2001 has copied around 40G in one hour :) [14:30:53] it is true that it is compressing+encrypting [14:59:01] avoid compressing [15:20:32] 10DBA, 10Wiki-Loves-Monuments-Database, 13Patch-For-Review: mysqldump is timing out preventing all tables from being included in the dump - https://phabricator.wikimedia.org/T138517#3040980 (10Lokal_Profil) >>! In T138517#3040791, @jcrespo wrote: > Toolsdb were finally upgraded to MariaDB 10/jessie at T15735... [15:30:24] 10DBA, 06Operations, 10procurement: New DBs purchase: codfw and eqiad final figures - https://phabricator.wikimedia.org/T158580#3040989 (10Marostegui) [16:39:54] 10DBA, 06Operations: Puppetize grants for mysql backups on dbstore hosts - https://phabricator.wikimedia.org/T111929#3041133 (10jcrespo)