[05:56:33] 10DBA, 10Operations, 10Goal, 10Patch-For-Review: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy - https://phabricator.wikimedia.org/T190704#4241894 (10Marostegui) [06:24:11] 10DBA, 10Operations, 10Goal, 10Patch-For-Review: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy - https://phabricator.wikimedia.org/T190704#4241913 (10Marostegui) [06:31:08] 10DBA, 10Operations, 10Goal, 10Patch-For-Review: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy - https://phabricator.wikimedia.org/T190704#4241917 (10Marostegui) [06:43:26] 10DBA, 10Operations, 10Goal, 10Patch-For-Review: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy - https://phabricator.wikimedia.org/T190704#4241923 (10Marostegui) [06:51:58] 10DBA, 10Operations, 10Goal, 10Patch-For-Review: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy - https://phabricator.wikimedia.org/T190704#4241926 (10Marostegui) [06:53:33] 10DBA, 10Operations, 10Goal, 10Patch-For-Review: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy - https://phabricator.wikimedia.org/T190704#4163218 (10Marostegui) db2094 and db2095 (sanitarium in codfw) have been moved to replicate under codfw sanitarium masters. The... [07:18:31] 10DBA, 10Operations, 10Goal, 10Patch-For-Review: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy - https://phabricator.wikimedia.org/T190704#4241957 (10Marostegui) [08:10:07] coul you give a quick look at https://gerrit.wikimedia.org/r/#/c/434920/ ? [08:13:06] yep [08:23:08] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4242013 (10Marostegui) [08:23:17] 10DBA, 10Multi-Content-Revisions, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4242014 (10Marostegui) [08:23:24] 10Blocked-on-schema-change, 10DBA: Make several mediawiki table fields unsigned ints on wmf databases - https://phabricator.wikimedia.org/T89737#4242015 (10Marostegui) [08:23:34] 10Blocked-on-schema-change, 10MediaWiki-Change-tagging, 10MediaWiki-Database, 10Wikidata-Ministry-Of-Magic: Schema change for ct_tag_id field to change_tag - https://phabricator.wikimedia.org/T195193#4242016 (10Marostegui) [09:11:51] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review: Schema change for refactored actor storage - https://phabricator.wikimedia.org/T188299#4242115 (10Marostegui) [09:13:00] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review: Schema change for refactored actor storage - https://phabricator.wikimedia.org/T188299#4210128 (10Marostegui) 05Open>03Resolved All done. Note there is still this ticket pending in regards to the ac... [09:13:39] 10DBA, 10Wikimedia-Site-requests: Global rename of Horcrux92 → Horcrux: supervision needed - https://phabricator.wikimedia.org/T195661#4242138 (101997kB) @Marostegui Can we start? [09:16:10] 10DBA, 10Wikimedia-Site-requests: Global rename of Horcrux92 → Horcrux: supervision needed - https://phabricator.wikimedia.org/T195661#4242140 (10Marostegui) Go for it [09:16:16] I'd like to double check one thing for debmonitor with you, if it's ok to have 'read committed' for isolation level and STRICT_ALL_TABLES for sql mode [09:16:23] Django docs seems to suggest those two ;) [09:19:01] 10DBA, 10Wikimedia-Site-requests: Global rename of Horcrux92 → Horcrux: supervision needed - https://phabricator.wikimedia.org/T195661#4242142 (101997kB) [[https://meta.wikimedia.org/wiki/Special:GlobalRenameProgress/Horcrux|Progress]] [09:25:18] ( https://gerrit.wikimedia.org/r/#/c/436244/ for reference) [09:29:53] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4242182 (10Marostegui) For the first interation of this alter, I am going to be altering s5 codfw. This will be done on... [09:30:09] 10DBA, 10Multi-Content-Revisions, 10Patch-For-Review, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4242183 (10Marostegui) For the first interation of this alter, I am going to be altering s5 codfw. This will be done on a per host ba... [09:30:14] 10Blocked-on-schema-change, 10MediaWiki-Change-tagging, 10MediaWiki-Database, 10Patch-For-Review, 10Wikidata-Ministry-Of-Magic: Schema change for ct_tag_id field to change_tag - https://phabricator.wikimedia.org/T195193#4242184 (10Marostegui) For the first interation of this alter, I am going to be alter... [09:30:55] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Make several mediawiki table fields unsigned ints on wmf databases - https://phabricator.wikimedia.org/T89737#4242185 (10Marostegui) For the first interation of this alter, I am going to be altering s5 codfw. This will be done on a per host basis, to see... [09:33:16] 10DBA, 10Wikimedia-Site-requests: Global rename of Horcrux92 → Horcrux: supervision needed - https://phabricator.wikimedia.org/T195661#4242192 (101997kB) Successfully finished. Thanks! [09:33:29] 10DBA, 10Wikimedia-Site-requests: Global rename of Horcrux92 → Horcrux: supervision needed - https://phabricator.wikimedia.org/T195661#4242194 (101997kB) 05Open>03Resolved a:031997kB [09:53:16] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4242284 (10Marostegui) Primary masters will need to be failed over (or done once eqiad is passive) as changing a column... [09:56:34] 10DBA, 10Multi-Content-Revisions, 10Patch-For-Review, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4242305 (10Marostegui) Primary masters will need to be failed over (or done once eqiad is passive) as changing a column data type isn... [09:58:13] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Make several mediawiki table fields unsigned ints on wmf databases - https://phabricator.wikimedia.org/T89737#4242306 (10Marostegui) Primary masters will need to be failed over (or done once eqiad is passive) as changing a column data type isn't an inpla... [09:59:11] 10DBA, 10Operations, 10Epic: DB meta task for next DC failover issues - https://phabricator.wikimedia.org/T189107#4242311 (10Marostegui) [09:59:16] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4242310 (10Marostegui) [09:59:35] 10DBA, 10Operations, 10Epic: DB meta task for next DC failover issues - https://phabricator.wikimedia.org/T189107#4031557 (10Marostegui) [09:59:40] 10DBA, 10Multi-Content-Revisions, 10Patch-For-Review, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4242322 (10Marostegui) [09:59:48] 10DBA, 10Operations, 10Epic: DB meta task for next DC failover issues - https://phabricator.wikimedia.org/T189107#4031557 (10Marostegui) [09:59:51] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Make several mediawiki table fields unsigned ints on wmf databases - https://phabricator.wikimedia.org/T89737#4242324 (10Marostegui) [10:03:20] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4242327 (10Marostegui) a:03Marostegui [10:03:38] 10DBA, 10Multi-Content-Revisions, 10Patch-For-Review, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4242329 (10Marostegui) a:05Anomie>03Marostegui [10:03:48] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Make several mediawiki table fields unsigned ints on wmf databases - https://phabricator.wikimedia.org/T89737#4242331 (10Marostegui) a:03Marostegui [10:04:22] 10Blocked-on-schema-change, 10MediaWiki-Change-tagging, 10MediaWiki-Database, 10Patch-For-Review, 10Wikidata-Ministry-Of-Magic: Schema change for ct_tag_id field to change_tag - https://phabricator.wikimedia.org/T195193#4242334 (10Marostegui) a:03Marostegui [10:05:06] if you get bored, it would be nice to test the instant feature on the 8.0 host [10:05:50] oh! Indeed! [10:05:55] I will! [10:07:22] I am filing right now a bug for lack of documentation [10:12:28] send me the url when done :) [10:19:52] https://bugs.mysql.com/bug.php?id=91074 [10:24:21] subscribed! [10:24:43] 10DBA, 10Patch-For-Review: Setup database logical backups on eqiad - https://phabricator.wikimedia.org/T192358#4242378 (10jcrespo) All misc hosts are backed up (except m4 and tendril), maybe some extra s* hosts could be done, too. [10:54:28] 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 10MW-1.32-release-notes (WMF-deploy-2018-05-22 (1.32.0-wmf.5)), and 2 others: Clean up indexes of wb_terms table - https://phabricator.wikimedia.org/T194273#4242472 (10jcrespo) Still ongoing: ``` Wikibase\Lib\Store\Sql\TermSqlIndex::getMatchingT... [16:38:21] 10DBA, 10Operations, 10ops-eqiad, 10Patch-For-Review: db1065 storage crash - https://phabricator.wikimedia.org/T195444#4243358 (10Marostegui) p:05Triage>03Normal >>! In T195444#4230827, @jcrespo wrote: > db1065 storage has been rebuilt and data cloned to it again. However, there is a smart error on the... [18:05:03] 10DBA, 10MediaWiki-User-management, 10Anti-Harassment (AHT Sprint 21/22): Draft a proposal for granular blocks table schema(s), submit for DBA review - https://phabricator.wikimedia.org/T193449#4243634 (10kaldari) [18:06:41] 10DBA, 10MediaWiki-User-management, 10Anti-Harassment (AHT Sprint 21/22): Draft a proposal for granular blocks table schema(s), submit for DBA review - https://phabricator.wikimedia.org/T193449#4169871 (10kaldari) FWIW, I prefer option A if it seems feasible from a performance POV. Tagged for DBA input. [18:15:22] 10DBA, 10Multi-Content-Revisions, 10Patch-For-Review, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4243687 (10Anomie) >>! In T192926#4242305, @Marostegui wrote: > Primary masters will need to be failed over (or done once eqiad is pa... [18:22:26] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4243745 (10Anomie) >>! In T191316#4242284, @Marostegui wrote: > Primary masters will need to be failed over (or done onc... [18:23:10] 10DBA, 10Multi-Content-Revisions, 10Patch-For-Review, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4243751 (10Marostegui) I guess that CHANGE https://gerrit.wikimedia.org/r/c/393929/14/maintenance/archives/patch-drop-ar_text.sql is... [18:27:16] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4243767 (10Marostegui) Same as: T192926#4243751 that CHANGE is being interpreted as a data change by mariadb: ``` MariaD... [18:29:44] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4243784 (10Anomie) According to https://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html#online-ddl-col... [18:34:44] 10DBA, 10Multi-Content-Revisions, 10Patch-For-Review, 10Schema-change: Schema change to drop archive.ar_text and archive.ar_flags - https://phabricator.wikimedia.org/T192926#4243814 (10Anomie) Copying for clarity from T191316#4243784: Ah, it's not complaining about a data type change, it's complaining abo... [18:36:32] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4243827 (10Marostegui) It can be enabled on a session: ``` MariaDB [test2]> set SQL_MODE=strict_trans_tables; Query OK,... [18:44:21] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Schema change to make archive.ar_rev_id NOT NULL - https://phabricator.wikimedia.org/T191316#4243840 (10Marostegui) As we are depooling the replicas anyways so we can leave it there, but, we can just as enable it... [18:48:12] 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review, 10Schema-change: Fix WMF schemas to not break when comment store goes WRITE_NEW - https://phabricator.wikimedia.org/T187089#4243857 (10Anomie) [18:52:06] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review: Schema change for refactored actor storage - https://phabricator.wikimedia.org/T188299#4243865 (10Anomie) Thanks for reminding me about that, I might have forgotten that the image table was being done i... [19:12:32] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review: Schema change for refactored actor storage - https://phabricator.wikimedia.org/T188299#4243963 (10Bstorm) Are we at the point that I should deploy the change here? https://gerrit.wikimedia.org/r/#/c/431... [19:14:32] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-Platform-Team (MWPT-Q4-Apr-Jun-2018), 10Patch-For-Review: Schema change for refactored actor storage - https://phabricator.wikimedia.org/T188299#4244000 (10Anomie) Probably not, since `img_actor` doesn't exist yet. [20:15:17] 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 10MW-1.32-release-notes (WMF-deploy-2018-05-22 (1.32.0-wmf.5)), and 2 others: Clean up indexes of wb_terms table - https://phabricator.wikimedia.org/T194273#4244201 (10Ladsgroup) >>! In T194273#4242472, @jcrespo wrote: > Still ongoing: > ``` > W... [20:45:58] 10DBA, 10MediaWiki-User-management, 10Anti-Harassment (AHT Sprint 21/22): Draft a proposal for granular blocks table schema(s), submit for DBA review - https://phabricator.wikimedia.org/T193449#4244294 (10MaxSem) Honestly, A feels more manageable. One question: do you intend to have more than one options row... [20:54:22] 10DBA, 10MediaWiki-User-management, 10Anti-Harassment (AHT Sprint 21/22): Draft a proposal for granular blocks table schema(s), submit for DBA review - https://phabricator.wikimedia.org/T193449#4244309 (10dmaza) >>! In T193449#4244294, @MaxSem wrote: > Honestly, A feels more manageable. One question: do you... [20:58:22] 10DBA, 10MediaWiki-User-management, 10Anti-Harassment (AHT Sprint 21/22): Draft a proposal for granular blocks table schema(s), submit for DBA review - https://phabricator.wikimedia.org/T193449#4244315 (10dbarratt) >>! In T193449#4244309, @dmaza wrote: > Now that you said `options`. They are not really optio... [21:05:03] 10DBA, 10ContentTranslation, 10Data-Services: Consider adding CX tables to replica - https://phabricator.wikimedia.org/T196020#4244356 (10Framawiki) [21:13:31] 10DBA, 10ContentTranslation, 10Data-Services: Consider adding ContentTranslation (CX) tables to wiki replicas - https://phabricator.wikimedia.org/T196020#4244389 (10bd808) [21:29:27] Anyone here know anything about https://phabricator.wikimedia.org/T191109 ? [22:00:06] 10DBA, 10MediaWiki-Platform-Team, 10Structured-Data-Commons, 10Wikidata, and 2 others: Deploy MCR storage layer - https://phabricator.wikimedia.org/T174044#4244508 (10CCicalese_WMF)