[00:04:27] 10DBA, 10MediaWiki-Database, 10MediaWiki-Documentation, 10ArchCom-RfC (ArchCom-Approved), and 3 others: Bump MediaWiki's minimum supported MySQL Version to 5.5.8 - https://phabricator.wikimedia.org/T161232#3446861 (10Legoktm) >>! In T161232#3437495, @Reedy wrote: >>>! In T161232#3436277, @jcrespo wrote: >>... [01:29:34] 10DBA, 10MediaWiki-Database, 10MediaWiki-Documentation, 10ArchCom-RfC (ArchCom-Approved), and 3 others: Bump MediaWiki's minimum supported MySQL Version to 5.5.8 - https://phabricator.wikimedia.org/T161232#3446946 (10Paladox) @Reedy we will want mariadb checks too unless it works with the mysql one? [05:39:09] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s1 - https://phabricator.wikimedia.org/T166204#3447125 (10Marostegui) db1073 is done: ``` root@neodymium:/home/marostegui# for i in `cat s1_tables`; do echo $i; mysql --skip-ssl -hdb1073 enwik... [05:40:37] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s1 - https://phabricator.wikimedia.org/T166204#3447126 (10Marostegui) [06:00:17] 10DBA, 10Tools: Killed very long transaction that was blocking the replica on labsdb1001 - https://phabricator.wikimedia.org/T133086#3447136 (10Marostegui) 05Open>03Resolved I am going to close this ticket as resolved as no further reports were done and looks like this was a one-time issue. We can reopen i... [07:14:51] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3447152 (10Marostegui) [07:17:25] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3447153 (10Marostegui) 05Open>03stalled s4 is done. So, only s1 and s4 primary masters are pending to be done. Unfortunately, due to the bug: https://ji... [07:23:45] morning! [07:24:25] eventlogging cleaner is still running on db1047, but it looks ok, so I'd proceed to run it also on dbstore1002 too. For the moment I'll target the 2014 data [07:24:51] elukey: Sounds good! Once you are done with a few tables, ping me and we can try to rebuild them and see how much space back we get [07:24:51] anything against it? If not I'll start this morning [07:24:57] yep! [07:24:59] go for it :) [07:42:02] started! [07:43:28] elukey: delete them all! :-P [07:43:29] cool! [07:43:32] XDDD [07:45:26] "sanitize" them all :D [07:45:26] 10DBA, 10MediaWiki-Parser, 10MediaWiki-Platform-Team, 10Patch-For-Review: WMF ParserCache disk space exhaustion - https://phabricator.wikimedia.org/T167784#3447180 (10Marostegui) pc1005 and pc1006 showed up on warning status on Icinga today because of disk space consumption and it is indeed growing quite w... [07:48:02] 10DBA, 10MediaWiki-Database, 10MediaWiki-Documentation, 10ArchCom-RfC (ArchCom-Approved), and 3 others: Bump MediaWiki's minimum supported MySQL Version to 5.5.8 - https://phabricator.wikimedia.org/T161232#3447181 (10jcrespo) What is the narrative? "5.5.X required. If you are restricted to 5.1 or 5.0, use... [08:23:34] 10DBA, 10MediaWiki-Parser, 10MediaWiki-Platform-Team, 10Patch-For-Review: WMF ParserCache disk space exhaustion - https://phabricator.wikimedia.org/T167784#3447221 (10jcrespo) I would say these^ are back on the table, unless someone can point to another explicit reason why this is happening again. [09:19:35] did labsdb1001 crash? [09:20:06] no that I know of [09:20:11] it is executing alters [09:20:16] why? [09:20:42] ok [09:20:50] it was a glitch on monitoring [09:20:54] | Uptime | 8200682 | [09:21:00] probably because extra load [09:21:01] yeah, those are heavy alters [09:21:06] no problem [09:21:08] :) [09:21:15] just sahw it as uptime 1 minute on tendril [09:22:24] sorry for the distraction [09:22:27] it is a glitch as you said, mysql uptime is what i pasted above [09:22:30] no distraction [09:22:39] I am already scared of those alters on the old labs hosts :) [09:22:39] I think tendril is actually ok [09:22:43] just monitoring [09:22:48] is getting slow [09:22:52] yeah [11:11:22] 10DBA, 10Operations, 10Pybal, 10Availability: Create a backend check for pybal to monitor the MySQL protocol being up - https://phabricator.wikimedia.org/T165677#3447471 (10ema) p:05Triage>03Normal [11:17:24] 10DBA, 10Operations, 10Pybal, 10Availability: Create a backend check for pybal to monitor the MySQL protocol being up - https://phabricator.wikimedia.org/T165677#3273473 (10ema) It might be worth looking into the built-in [[ http://twistedmatrix.com/documents/current/core/howto/rdbms.html | adbapi ]] for... [11:27:18] 10DBA, 10Wikidata: Migrate wb_terms to using prefixed entity IDs instead of numeric IDs - https://phabricator.wikimedia.org/T114903#3447531 (10daniel) [11:32:07] 10DBA, 10Wikidata: Migrate wb_terms to using prefixed entity IDs instead of numeric IDs - https://phabricator.wikimedia.org/T114903#3447549 (10daniel) [11:50:06] 10DBA, 10Cloud-Services, 10User-Urbanecm: Prepare and check storage layer for maiwikimedia - https://phabricator.wikimedia.org/T168788#3447607 (10Urbanecm) @Marostegui Just a notification, a few other users were created before a moment. [12:27:20] 10DBA, 10Cloud-Services, 10User-Urbanecm: Prepare and check storage layer for maiwikimedia - https://phabricator.wikimedia.org/T168788#3447668 (10Marostegui) Thanks - I am checking the sanitized tables, but so far look good. So I will check a few more things and then create the views [12:35:26] 10DBA, 10Cloud-Services, 10User-Urbanecm: Prepare and check storage layer for maiwikimedia - https://phabricator.wikimedia.org/T168788#3447688 (10Marostegui) 05Open>03Resolved I have created the views on all the labs servers [12:36:16] 10DBA, 10Cloud-Services, 10User-Urbanecm: Prepare and check storage layer for maiwikimedia - https://phabricator.wikimedia.org/T168788#3447691 (10Urbanecm) Thank you! [13:36:49] 10DBA: duplicate key problems - https://phabricator.wikimedia.org/T151029#3447932 (10jcrespo) 05Resolved>03Open From db2072 (`ENGINE=InnoDB row_format=COMPRESSED KEY_BLOCK_SIZE=8`): ``` change_tag... ERROR 1062 (23000) at line 1: Duplicate entry '963806525-mobile edit' for key 'ct_rc_id' linter... ERROR 10... [13:36:59] 10Blocked-on-schema-change, 10DBA, 10Wikimedia-Site-requests, 10Wikisource, and 4 others: Schema change for page content language - https://phabricator.wikimedia.org/T69223#3447934 (10jcrespo) [13:37:56] heads up to marostegui for reopening of T151029 [13:37:56] T151029: duplicate key problems - https://phabricator.wikimedia.org/T151029 [13:38:10] yeah, I saw that :( [13:38:49] we'll do the usual, and see what happens [13:39:40] let's see :( [13:40:11] they are not key tables [13:40:23] and not large ones, so that is good [13:42:42] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447964 (10Addshore) [13:43:35] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447813 (10Addshore) The table need to be replicated to the labs DB replicas in order to be accessible to Quarry. [13:44:20] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata, 10cloud-services-team: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447968 (10Marostegui) [13:46:14] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata, 10cloud-services-team: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447974 (10Lucas_Werkmeister_WMDE) I think I just found the right config file in puppet – can I just submit a Ge... [13:49:12] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata, 10cloud-services-team: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447984 (10Addshore) I dont see why not! [13:50:01] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata, 10cloud-services-team: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447813 (10Marostegui) Just to clarify, the table is already on labs hosts. What is missing is the view creation [13:50:30] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata, 10cloud-services-team: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447999 (10Marostegui) p:05Triage>03Normal [13:53:03] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata, 10cloud-services-team: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3448028 (10Lucas_Werkmeister_WMDE) Thanks, then it makes more sense that the change I’m about to submit only tou... [13:58:05] 10DBA, 10Wikibase-Quality, 10Wikibase-Quality-Constraints, 10Wikidata, and 2 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3448053 (10Marostegui) I would suggest you add at least @chasemp and @bd808 as reviewers [14:52:08] 10DBA, 10Operations: Global rename user - https://phabricator.wikimedia.org/T170941#3448297 (10Marostegui) [14:53:31] 10DBA, 10Operations: Global rename user - https://phabricator.wikimedia.org/T170941#3448274 (10Marostegui) I am happy to monitor the involved DBs here. In which TZ are you? I would prefer to do it tomorrow in the morning, what about 9UTC? [14:58:04] 10DBA, 10Operations: Global rename user - https://phabricator.wikimedia.org/T170941#3448322 (10Marostegui) This will be done 19th at 11UTC. [14:58:18] 10DBA, 10Operations: Global rename user - https://phabricator.wikimedia.org/T170941#3448325 (10Steinsplitter) >>! In T170941#3448297, @Marostegui wrote: > I am happy to monitor the involved DBs here. > In which TZ are you? I would prefer to do it tomorrow in the morning, what about 9UTC? I am around after 11... [15:20:08] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s1 - https://phabricator.wikimedia.org/T166204#3448369 (10Marostegui) labsdb1001 is almost done, just pending categorylinks table, which is suffering table metadata lock [15:20:39] 10DBA, 10Security-Team, 10WMF-Legal, 10Wikibase-Quality, and 4 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3448370 (10bd808) Adding #security-team and #wmf-legal for approval of exposing new data. If the data in this table is all available on-w... [15:21:58] 10DBA, 10Data-Services, 10Security-Team, 10WMF-Legal, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3448376 (10bd808) [15:24:28] 10DBA, 10MediaWiki-Database, 10MediaWiki-Documentation, 10ArchCom-RfC (ArchCom-Approved), and 3 others: Bump MediaWiki's minimum supported MySQL Version to 5.5.8 - https://phabricator.wikimedia.org/T161232#3448390 (10Reedy) >>! In T161232#3447181, @jcrespo wrote: > What is the narrative? "5.5.X required. I... [15:58:33] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s1 - https://phabricator.wikimedia.org/T166204#3448510 (10Marostegui) labsdb1001 is done: ``` # for i in `cat s1_tables`;do echo $i; mysql --skip-ssl -e "show create table $i\G" enwiki | grep... [15:58:37] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s1 - https://phabricator.wikimedia.org/T166204#3448512 (10Marostegui) [16:00:14] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s1 - https://phabricator.wikimedia.org/T166204#3288285 (10Marostegui) I am going to start the same alter table on labsdb1003 #cloud-services-team labsdb1001 took a lot less than I expected (on... [16:52:49] 10DBA, 10Analytics-Kanban: Inconsistent default charset for analytics slaves - https://phabricator.wikimedia.org/T170952#3448710 (10elukey) [20:58:12] 10DBA, 10Analytics-Kanban, 10Operations, 10Patch-For-Review, 10User-Elukey: Puppetize Piwik's Database and set up periodical backups - https://phabricator.wikimedia.org/T164073#3449972 (10Nuria) 05Open>03Resolved [21:06:35] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, and 2 others: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990#3450014 (10kaldari) [21:07:36] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, and 2 others: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990#3450032 (10kaldari) [21:15:13] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, and 2 others: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990#3450075 (10kaldari) [21:16:38] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, and 2 others: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990#3450014 (10kaldari) I see that T170925 already exists and is almost the same thing. Will follow up there and likely close this as a dup... [23:05:06] 10DBA, 10Wikimedia-Language-setup: Deletion of als Wiktionary, Wikibooks and Wikiquote? - https://phabricator.wikimedia.org/T169450#3450565 (10MF-Warburg) So is creation the redirection without deleting the databases a reasonable way forward? [23:25:50] 10DBA, 10Operations: Global rename of user Moros - https://phabricator.wikimedia.org/T170941#3450716 (10Aklapper)