[05:13:36] 10Blocked-on-schema-change, 10DBA, 10Wikidata, 10Schema-change: Drop eu_touched in production - https://phabricator.wikimedia.org/T144010 (10Marostegui) [05:13:37] 10Blocked-on-schema-change, 10DBA, 10Schema-change: Truncate SHA-1 indexes - https://phabricator.wikimedia.org/T51190 (10Marostegui) [05:13:50] 10DBA, 10Schema-change: Convert UNIQUE INDEX to PK in Production - https://phabricator.wikimedia.org/T199368 (10Marostegui) [05:15:53] 10Blocked-on-schema-change, 10DBA, 10Schema-change: Truncate SHA-1 indexes - https://phabricator.wikimedia.org/T51190 (10Marostegui) As always with the first iteration of a schema change - I will do them host by host in codfw and once we are sure all is fine, the passive DC will be done with replication. So... [05:15:55] 10Blocked-on-schema-change, 10DBA, 10Wikidata, 10Schema-change: Drop eu_touched in production - https://phabricator.wikimedia.org/T144010 (10Marostegui) As always with the first iteration of a schema change - I will do them host by host in codfw and once we are sure all is fine, the passive DC will be done... [05:16:08] 10DBA, 10Schema-change: Convert UNIQUE INDEX to PK in Production - https://phabricator.wikimedia.org/T199368 (10Marostegui) As always with the first iteration of a schema change - I will do them host by host in codfw and once we are sure all is fine, the passive DC will be done with replication. So this is s5... [07:03:22] 10DBA, 10Schema-change: Convert UNIQUE INDEX to PK in Production - https://phabricator.wikimedia.org/T199368 (10Marostegui) These are how the tables look like after the alters: ``` *************************** 1. row *************************** Table: interwiki Create Table: CREATE TABLE `interwiki` (... [07:04:12] 10Blocked-on-schema-change, 10DBA, 10Schema-change: Truncate SHA-1 indexes - https://phabricator.wikimedia.org/T51190 (10Marostegui) This is how the tables look like after the alters (same as they were basically): ``` Table: oldimage Create Table: CREATE TABLE `oldimage` ( `oi_name` varbinary(255) N... [07:43:21] 10DBA, 10Operations, 10ops-eqiad: db1069 bad disk - https://phabricator.wikimedia.org/T199056 (10Marostegui) 05Resolved>03Open This has happened again, same disk, disk #0, can we get another one? Please ping me before replacing it so I can manually put it offline ``` Enclosure Device ID: 32 Slot Number:... [13:37:14] working with db2095? [13:37:20] Yeah, I am investigating [13:37:23] or should I take a look? [13:37:29] I will take care of it, no worries [13:37:33] I will leave you then [13:48:07] 10DBA, 10Patch-For-Review, 10Schema-change: Convert UNIQUE INDEX to PK in Production - https://phabricator.wikimedia.org/T199368 (10Marostegui) [13:48:36] 10Blocked-on-schema-change, 10DBA, 10Wikidata, 10Patch-For-Review, 10Schema-change: Drop eu_touched in production - https://phabricator.wikimedia.org/T144010 (10Marostegui) [14:22:39] <_joe_> jynus, marostegui I am going on vacation tomorrow, so deployment will have to wait anyways, but I have a beta for dbctl; here is how things will work more or less https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/conftool/+/7935a7432537b2018ac1ebf48311bf01dbb6b141/conftool/extensions/README.md - if you want to look at the code, it's here [14:22:44] <_joe_> https://gerrit.wikimedia.org/r/#/c/operations/software/conftool/+/441396/ [14:23:33] Thanks _joe_ I started to take a look earlier and then got distracted [14:23:37] Enjoy your holidays :) [14:24:29] thank you, _joe_ [15:45:03] 10DBA, 10Patch-For-Review: Failover db1052 (s1) db primary master - https://phabricator.wikimedia.org/T197069 (10Marostegui) a:03Marostegui