[04:47:20] 10DBA, 10MediaWiki-General-or-Unknown, 10MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Krinkle) > Finally, run the sc... [06:23:29] 10DBA: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) [06:23:50] 10DBA: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) p:05Triage>03High [06:26:30] 10DBA: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) [06:36:44] 10DBA, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) [06:49:21] 10DBA, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) [07:11:27] 10DBA, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) [07:12:45] 10DBA, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) a:05Marostegui>03RobH pc1004, pc1005 and pc1006 are now fully ready for #dc-ops to... [07:13:27] 10DBA, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) Priority is high like {T209858} because these have a hard deadline on the lease expiration [07:14:09] 10DBA, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10Marostegui) [07:50:47] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change: Dropping page.page_counter on wmf databases - https://phabricator.wikimedia.org/T86338 (10Marostegui) @Bstorm I have finished the alter on s2 on the labs infra. Can you recreate the views for the `page` table for the following wikis?:... [08:29:14] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Database, 10Scoring-platform-team, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) [08:29:19] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change: Dropping page.page_counter on wmf databases - https://phabricator.wikimedia.org/T86338 (10Marostegui) [08:32:19] I'll do T85757 (drop column user_options) on the rest of s2 (sanitarium & master) [08:32:20] T85757: Dropping user.user_options on wmf databases - https://phabricator.wikimedia.org/T85757 [08:32:36] ok [08:55:09] have you seen my comment about db1063? [09:13:29] yes :( [09:13:46] but I was not sure if that applies for a master too [09:14:12] I think it is the same thing - it can be with smart errors for weeks without failing really [09:14:23] i can close the ticket and add to the T208323 [09:14:24] T208323: Predictive failures on disk S.M.A.R.T. status - https://phabricator.wikimedia.org/T208323 [09:14:29] if we see performance issues, we can fail it manually [09:14:33] and then get it replaced [09:14:47] kk [09:14:58] then I decline it [09:15:12] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Database, 10Scoring-platform-team, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) [09:15:15] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change: Dropping page.page_counter on wmf databases - https://phabricator.wikimedia.org/T86338 (10Marostegui) [09:16:09] 10DBA, 10Operations: Predictive failures on disk S.M.A.R.T. status - https://phabricator.wikimedia.org/T208323 (10Banyek) [09:16:19] 10DBA, 10Operations: Predictive failures on disk S.M.A.R.T. status - https://phabricator.wikimedia.org/T208323 (10Banyek) db1063 `name: Adapter #0 Virtual Drive: 0 (Target Id: 0) RAID Level: Primary-1, Secondary-0, RAID Level Qualifier-0 State: Optimal Number Of Drives per span: 2 Number of Spans: 6 Cur... [09:16:35] You can specify on the host disk which hosts are masters [09:17:00] note that db1065 is also a master and it is on that list [09:17:11] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change, 10User-Banyek: Dropping user.user_options on wmf databases - https://phabricator.wikimedia.org/T85757 (10Banyek) [09:24:17] I'll run the my schema change on db1066 but now not per database but with the dblist.s2 sile (on s6 I had no problems at all, and on s2 we have way more dbs) [09:27:01] ok [09:27:43] time PYTHONPATH=/home/banyek/wmfmariadbpy/ ./wmfmariadbpy/wmfmariadbpy/osc_host.py --method=ddl --host db1066.eqiad.wmnet --dblist mediawiki-config/dblists/s2.dblist --table user --no-replicate "DROP COLUMN IF EXISTS user_options" [09:28:16] looks good [09:28:23] on the way [09:33:34] 10DBA, 10JADE, 10Operations, 10TechCom-RFC, and 2 others: Introduce a new namespace for collaborative judgments about wiki entities - https://phabricator.wikimedia.org/T200297 (10Marostegui) >>! In T200297#4787352, @awight wrote: > @Marostegui Hello! I've added a few summary columns and indexes to the lin... [09:34:22] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Database, 10Scoring-platform-team, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) s5 eqiad progress [] labsdb1011 [] labsdb1010 [] labsdb1009 [x] dbstore1002 [] db1124 [] db1113 [] db1110 [] db110... [09:34:25] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change: Dropping page.page_counter on wmf databases - https://phabricator.wikimedia.org/T86338 (10Marostegui) s5 eqiad progress [] labsdb1011 [] labsdb1010 [] labsdb1009 [x] dbstore1002 [] db1124 [] db1113 [] db1110 [] db1102 [] db1100 [] db10... [09:34:32] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change: Dropping page.page_counter on wmf databases - https://phabricator.wikimedia.org/T86338 (10Marostegui) [09:34:49] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Database, 10Scoring-platform-team, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) [09:37:12] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change, 10User-Banyek: Dropping user.user_options on wmf databases - https://phabricator.wikimedia.org/T85757 (10Banyek) [10:03:42] 10DBA, 10Performance-Team: Increase parsercache keys TTL from 22 days back to 30 days - https://phabricator.wikimedia.org/T210992 (10Marostegui) [10:04:47] 10DBA, 10Performance-Team: Increase parsercache keys TTL from 22 days back to 30 days - https://phabricator.wikimedia.org/T210992 (10Marostegui) p:05Triage>03Normal @Krinkle can you confirm whether those are the two reverts we have to do? I have been looking around and I haven't found anything else to re... [10:05:31] 10DBA, 10Operations, 10Performance-Team: Increase parsercache keys TTL from 22 days back to 30 days - https://phabricator.wikimedia.org/T210992 (10Marostegui) [10:36:28] 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 10MW-1.32-notes (WMF-deploy-2018-05-22 (1.32.0-wmf.5)), and 2 others: Clean up indexes of wb_terms table - https://phabricator.wikimedia.org/T194273 (10Marostegui) We cannot alter `wb_terms` anymore. The table is almost 1TB on the master, and th... [11:16:33] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Banyek) [13:00:11] 10DBA, 10Operations, 10Availability (MediaWiki-MultiDC), 10Performance-Team (Radar): Investigate solutions for MySQL connection pooling - https://phabricator.wikimedia.org/T196378 (10jijiki) p:05Triage>03Normal [14:00:08] 10DBA, 10Lexicographical data, 10Wikidata, 10Datacenter-Switchover-2018, and 5 others: S8 replication issues leading to rows missing during eqiad -> codfw switch (Was: "A few lexemes disappeared") - https://phabricator.wikimedia.org/T206743 (10Banyek) on db1116 we still have the backup dir: `drwxr-xr-x 8... [14:02:39] 10DBA, 10Lexicographical data, 10Wikidata, 10Datacenter-Switchover-2018, and 5 others: S8 replication issues leading to rows missing during eqiad -> codfw switch (Was: "A few lexemes disappeared") - https://phabricator.wikimedia.org/T206743 (10Marostegui) If there is no space issues on that host, there is... [14:24:37] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review, 10Schema-change: Dropping page.page_counter on wmf databases - https://phabricator.wikimedia.org/T86338 (10Marostegui) >>! In T86338#4792917, @Marostegui wrote: > @Bstorm I have finished the alter on s2 on the labs infra. > Can you recreate the views f... [14:56:24] 10DBA, 10Epic, 10Tracking: Database tables to be dropped on Wikimedia wikis and other WMF databases (tracking) - https://phabricator.wikimedia.org/T54921 (10Anomie) [14:56:26] 10DBA: Drop table image_comment_temp on all wikis - https://phabricator.wikimedia.org/T209591 (10Anomie) 05stalled>03Open Should be good to go now. [15:00:04] TIL Aria is a MyISAM replacement for MyISAM [15:00:05] https://phabricator.wikimedia.org/P7879 [15:01:09] yeah [15:01:31] it is the mariadb implementation [15:14:20] I do the etherpad [15:14:40] thanks [15:35:29] 10DBA, 10MediaWiki-General-or-Unknown, 10MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Anomie) >>! In T208909#4792783... [15:52:57] 10DBA, 10MediaWiki-General-or-Unknown, 10MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10phuedx) PEBCAK error! Great! I... [16:15:31] 10DBA, 10Analytics, 10Analytics-Kanban, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Banyek) On the dbstore1002 host the following databases exists along the wikis (with sizes) | Schema | Size | Comment | |centralauth | 88G |... [16:20:01] 10DBA, 10Analytics, 10Analytics-Kanban, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Marostegui) Some comments: `ops` doesn't need to be migrated. I believe `datasets` isn't used anymore, but needs double checking. `centralau... [16:21:32] 10DBA, 10Analytics, 10Analytics-Kanban, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Banyek) If the tables mentioned in https://phabricator.wikimedia.org/T210478#4794524 could be moved together (no need to create the own 'sta... [17:41:02] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Milimetric) @Marostegui, we would like to go over plans for implementation during our Wednesday meeting. Is the... [17:42:22] banyek: ^ [17:43:28] . [17:44:02] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Marostegui) @Banyek @Bstorm ^ [17:44:55] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Banyek) @Bstrom when could we talk about the details? [18:06:03] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Banyek) We talked about the kick-off, and tomorrow we'll sync up about what we found. @Bstorm gives me a few SQ... [19:03:04] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Ottomata) @marostegui, I think these nodes will not have any 'heavy queries'. There won't be any huge joins etc. (unless I don't fully understand the materi... [19:16:13] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Marostegui) >>! In T210749#4795112, @Ottomata wrote: > @marostegui, I think these nodes will not have any 'heavy queries'. There won't be any huge joins etc... [19:17:08] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Ottomata) Ah, interesting ok. [19:30:41] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Milimetric) The queries that we'll be running on here will be of the form: `select [subset of fields] from [one of the tables listed below] where [timestamp... [19:40:29] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Marostegui) @Milimetric I think T210749#4795213 should go to T210693 as you are explaining what you need and kinda drafting an SQL. The problem with the cur... [19:46:15] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Marostegui) I checked the hosts @elukey mentioned at T210749#4789691 and those have even SATA disks, so not even SAS, so even slower. [19:47:06] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Milimetric) Got it. Luca asked me to comment here describing exactly what we need to do on the boxes. But if basic replication can't even run, I of course... [19:53:46] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Marostegui) >>! In T210749#4795321, @Milimetric wrote: > Got it. Luca asked me to comment here describing exactly what we need to do on the boxes. But if b... [19:56:50] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Milimetric) More details on requirements for Analytics. The queries that we'll be running will be of the form:... [19:58:22] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Milimetric) Ok, done and agreed. But instead of trying to find hardware that will keep up with replication, I'm asking if replication is necessary, could we... [20:12:34] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Marostegui) Thanks @milimetric, very useful information. Could I ask for an specific and concrete SQL? With the... [20:14:21] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Ottomata) @Marostegui while Dan answers your Q, I like his idea: Could the HDDs handle replication if we only r... [20:23:43] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Milimetric) Yes, all the templates for the queries are here, and they're easy to read, basic python templating:... [20:37:00] 10DBA, 10Analytics, 10Analytics-Kanban, 10Data-Services, and 2 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Marostegui) >>! In T210693#4795448, @Ottomata wrote: > @Marostegui while Dan answers your Q, I like his idea: C... [20:50:16] 10DBA, 10Operations, 10decommission, 10ops-codfw, 10Patch-For-Review: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10RobH) [20:56:21] 10DBA, 10Operations, 10decommission, 10ops-codfw, 10Patch-For-Review: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for pc2004.codfw.wmnet and performed the following actions: - Revoked Pu... [20:56:34] 10DBA, 10Operations, 10decommission, 10ops-codfw, 10Patch-For-Review: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for pc2005.codfw.wmnet and performed the following actions: - Revoked Pu... [20:56:45] 10DBA, 10Operations, 10decommission, 10ops-codfw, 10Patch-For-Review: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for pc2006.codfw.wmnet and performed the following actions: - Revoked Pu... [21:02:16] 10DBA, 10Operations, 10decommission, 10ops-codfw, 10Patch-For-Review: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10RobH) Switch ports for later removal once unracked: pc2004 asw-b-codfw:ge-5/0/35 pc2005 asw-c-codfw:ge-5/0/3 pc2006 asw-d-codfw:ge-... [21:03:00] 10DBA, 10Operations, 10decommission, 10ops-codfw, 10Patch-For-Review: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10RobH) [21:11:24] 10DBA, 10Operations, 10decommission, 10ops-codfw, 10Patch-For-Review: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10RobH) [21:15:57] 10DBA, 10Operations, 10decommission, 10ops-codfw: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10RobH) [21:16:34] 10DBA, 10Operations, 10decommission, 10ops-codfw: Decommission parsercache hosts: pc2004 pc2005 pc2006 - https://phabricator.wikimedia.org/T209858 (10RobH) a:05RobH>03Papaul @Papaul, This is now ready for you to take over, wipe disks, and set aside for lease return this month (December 2018). This is... [21:18:07] 10DBA, 10Operations, 10decommission, 10ops-codfw: Decommission parsercache hosts: pc2004 pc2005 pc2006 (Dec 2018 lease return) - https://phabricator.wikimedia.org/T209858 (10RobH) [21:23:04] 10DBA, 10Analytics, 10Data-Services, 10User-Elukey: Hardware for cloud db replicas for analytics usage - https://phabricator.wikimedia.org/T210749 (10Nuria) >So to sum up the hardware approach so far: the proposed hardware wouldn't be able to catch up with the current replication stream for all sections. I... [21:26:24] 10DBA, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for pc1004.eqiad.wmnet and performed... [21:26:35] 10DBA, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for pc1005.eqiad.wmnet and performed... [21:27:01] 10DBA, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for pc1006.eqiad.wmnet and performed... [21:34:59] 10DBA, 10Operations, 10decommission, 10ops-eqiad, 10Patch-For-Review: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10RobH) Switch ports noted for @cmjohnson to clear their descriptions once they are unracked: pc1004... [21:36:39] 10DBA, 10Operations, 10decommission, 10ops-eqiad: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10RobH) [21:37:33] 10DBA, 10Operations, 10decommission, 10ops-eqiad: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10RobH) [21:37:37] 10DBA, 10Operations, 10decommission, 10ops-eqiad: Decommission parsercache hosts: pc1004.eqiad.wmnet pc1005.eqiad.wmnet pc1006.eqiad.wmnet - https://phabricator.wikimedia.org/T210969 (10RobH) a:05RobH>03Cmjohnson [22:37:49] 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 10MW-1.32-notes (WMF-deploy-2018-05-22 (1.32.0-wmf.5)), and 2 others: Clean up indexes of wb_terms table - https://phabricator.wikimedia.org/T194273 (10Ladsgroup) >>! In T194273#4793529, @Marostegui wrote: > We cannot alter `wb_terms` anymore. >...