[03:34:28] I am looking for the means to export Wikimedia's database (portions) as a csv or search Wikimedia as a data-table [03:38:00] the wiki database replicas are available through labs [07:28:48] 07Blocked-on-schema-change, 10DBA: Apply change_tag and tag_summary primary key schema change to Wikimedia wikis - https://phabricator.wikimedia.org/T147166#2737367 (10Marostegui) I have applied this change to another host on S1.enwiki codfw - `db2062`: ``` MariaDB PRODUCTION s1 localhost enwiki > show create... [07:33:17] 07Blocked-on-schema-change, 10DBA, 10Wikidata, 07Schema-change: Drop eu_touched in production - https://phabricator.wikimedia.org/T144010#2737370 (10Marostegui) a:03Marostegui [08:01:22] 07Blocked-on-schema-change, 10DBA, 10Wikidata, 07Schema-change: Drop eu_touched in production - https://phabricator.wikimedia.org/T144010#2737391 (10Marostegui) A quick salt run: ``` sudo salt -C 'G@mysql_role:master and G@site:codfw and G@mysql_group:core' cmd.run 'find /srv/sqldata/ -name wbc_entity_us... [10:00:19] jynus: I know you are busy with dbproxy, just let me know when you are ready for the call, no rush, I have plenty of time :) [10:00:52] it is ok [10:00:59] but please give me 5 minutes [10:01:09] sure, as I said, no rush, just let me know :) [11:40:56] 10DBA: Test InnoDB compression - https://phabricator.wikimedia.org/T139055#2737645 (10Marostegui) a:05jcrespo>03Marostegui [11:42:15] 10DBA: DROP OAI-related tables - https://phabricator.wikimedia.org/T139342#2428747 (10jcrespo) a:05jcrespo>03None [11:57:24] 10DBA, 06Community-Tech, 10MediaWiki-extensions-CentralAuth: Add indices for local_user_id and global_user_id in production - https://phabricator.wikimedia.org/T148243#2718099 (10Marostegui) Hi, Thanks for the ticket. We (DBAs) are not completely sure about how to proceed on this ticket as we are missing s... [12:27:07] 10DBA, 06Operations: Puppetize tendril web user creation - https://phabricator.wikimedia.org/T148955#2737720 (10akosiaris) [13:27:41] 10DBA, 10MediaWiki-extensions-Linter: DBA review of Linter extension - https://phabricator.wikimedia.org/T148866#2735236 (10jcrespo) > -- error category > linter_cat VARCHAR(30) not null, This seems like a clear candidate to normalize on a separate table with a key, isn't it? linter_params seems ok, but ple... [13:57:45] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2737966 (10Marostegui) [14:12:53] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738014 (10Marostegui) I will start with codfw in this order and only one host at the time: `` ` dbstore2002.codfw.wmnet db2045.codfw.wmnet db2052.codfw.wmnet db2059.codfw.wmnet db2066.codfw.wmnet db2023.codfw.wmnet ``` [14:17:31] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738033 (10Marostegui) [14:31:26] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738056 (10jcrespo) What is the current status of commons? Because according to https://phabricator.wikimedia.org/diffusion/MW/browse/master/maintenance/tables.sql the indexes should be: ``` rev_id int unsigned NOT NULL PRIMARY KEY... [14:36:04] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738100 (10Marostegui) This is the S4 master now: ``` `rev_id` int(8) unsigned NOT NULL AUTO_INCREMENT, PRIMARY KEY (`rev_id`), KEY `rev_timestamp` (`rev_timestamp`), KEY `page_timestamp` (`rev_page`,`rev_timestamp`), KEY... [14:38:12] 10DBA, 13Patch-For-Review: Unify commonswiki.revision - https://phabricator.wikimedia.org/T147305#2738116 (10Marostegui) I just realised that db2019 - codfw's master was never altered. So it needs to gets its table changed to match the rest of the shard. I will alter it tomorrow morning. [14:39:42] ^ actually I will do it now, no need to do it tomorrow morning [14:42:46] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738139 (10jcrespo) @Marostegui so commons is missing `​CREATE INDEX /*i*/page_user_timestamp ON /*_*/revision (rev_page,rev_user,rev_timestamp);` and so is dewiki. I would wait for input that that is correct, and that index is inde... [14:44:08] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738159 (10Marostegui) Sounds good, let's wait for their input before proceeding so we can get it right in just one go. Thanks a lot for the help guys! [14:44:25] 10DBA: Rampant differences in indexes on enwiki.revision across the DB cluster - https://phabricator.wikimedia.org/T132416#2738166 (10jcrespo) [14:44:28] 10DBA, 13Patch-For-Review: Unify commonswiki.revision - https://phabricator.wikimedia.org/T147305#2738162 (10jcrespo) 05Resolved>03Open See also T148967#2738139 [14:52:23] 10DBA: DROP OAI-related tables - https://phabricator.wikimedia.org/T139342#2738205 (10Aklapper) [15:11:15] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738326 (10Anomie) I can confirm that the `page_user_timestamp` index is targeted by certain queries that may be generated from ApiQueryRevisions.php. For example, a request to https://en.wikipedia.org/w/api.php?action=query&titles=... [15:37:22] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738422 (10Marostegui) @Anomie thanks for the input! Then we should aim to get it done to in this task, so it would be adding the following to the original command I suggested: ``` #./software/dbtools/osc_host.sh --host=xxx --port=3... [15:37:25] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738423 (10jcrespo) @Anomie You are the best! @Marostegui This means we have to re-review commons and apply this to dewiki, if possible, with all indexes there. [15:37:57] jynus: good timing :p [15:38:57] take this as a long race, ok? [15:39:10] yeah, it will take months to get it sorted [15:39:39] to be fair, before you were here, I could not even think about doing it [15:40:27] I am glad to be able to help :) [15:40:56] and again, we both should work on this [15:41:39] it is otherwise too big to tackle [15:41:44] indeed [15:41:55] Well, at least we have now clear how S5 and S4 should look like :) [15:42:00] yes [15:42:04] which is not a small feat [15:42:21] we now know that our largest and more important table should look like [15:42:29] \o/ [15:43:09] hahahahaha [16:30:40] 10DBA, 10Cognate, 10Wikidata: Initial Cognate DB review - https://phabricator.wikimedia.org/T148988#2738687 (10Addshore) [16:31:15] jynus: FYI I just made ^^ for review of a single table in a new extension (I talked to you a few weeks ago about it) [16:31:29] If you think you need any more info in the ticket then give me a ping! [16:41:34] addshore, @ meeting, will look at it after it [16:46:55] 10DBA: Fix PK on S5 dewiki.revision - https://phabricator.wikimedia.org/T148967#2738775 (10Anomie) >>! In T148967#2738422, @Marostegui wrote: > ``` > #./software/dbtools/osc_host.sh --host=xxx --port=3306 --db=dewiki --table=revision --method=ddl --no-replicate "DROP index rev_id, DROP PRIMARY KEY, ADD PRIMARY K... [16:52:23] 10DBA, 13Patch-For-Review: Reimage dbstore2001 as jessie - https://phabricator.wikimedia.org/T146261#2738797 (10Marostegui) The replication thread went fine throughout the night. I have been stopping and starting it today for a few times as well as stopping MySQL and starting it. No problems at all, so I would... [17:00:08] great!