[05:11:36] 10DBA: Recompress special slaves across eqiad and codfw - https://phabricator.wikimedia.org/T235599 (10Marostegui) [05:27:53] 10Blocked-on-schema-change, 10DBA: Schema change to rename user_newtalk indexes - https://phabricator.wikimedia.org/T234066 (10Marostegui) [05:28:08] 10Blocked-on-schema-change, 10DBA, 10Core Platform Team: Schema change for refactored actor and comment storage - https://phabricator.wikimedia.org/T233135 (10Marostegui) [05:28:23] 10Blocked-on-schema-change, 10DBA, 10Wikidata: Schema change on production for increase the size of wbt_text_in_lang.wbxl_language - https://phabricator.wikimedia.org/T237120 (10Marostegui) [05:37:29] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) [05:37:30] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) >>! In T238370#5673010, @Andrew wrote: > I applied all those changes to labtestwiki Thank you! [06:12:11] 10DBA: Switchover s6 primary database master db1061 -> db1131 - 19th Nov 06:00 - 06:30 UTC - https://phabricator.wikimedia.org/T235469 (10Marostegui) 05Open→03Resolved This was done successfully. Read only starts: 06:01:23Read only stops: 06:02:04 Total read only time: 41 seconds [06:12:14] 10DBA, 10Operations: Decommission db1061-db1073 - https://phabricator.wikimedia.org/T217396 (10Marostegui) [06:18:52] 10DBA: Decommission db1061.eqiad.wmnet - https://phabricator.wikimedia.org/T238624 (10Marostegui) [06:19:09] 10DBA: Decommission db1061.eqiad.wmnet - https://phabricator.wikimedia.org/T238624 (10Marostegui) p:05Triage→03Normal [06:20:06] 10DBA, 10Operations: Decommission db1061-db1073 - https://phabricator.wikimedia.org/T217396 (10Marostegui) [06:36:33] 10DBA, 10Patch-For-Review: Decommission db2061.codfw.wmnet - https://phabricator.wikimedia.org/T238526 (10Marostegui) [06:44:07] marostegui: 41 seconds is pretty nice :) [06:44:08] 10DBA, 10Patch-For-Review: Decommission db2061.codfw.wmnet - https://phabricator.wikimedia.org/T238526 (10Marostegui) [06:47:17] 10DBA: Switchover s6 primary database master db1061 -> db1131 - 19th Nov 06:00 - 06:30 UTC - https://phabricator.wikimedia.org/T235469 (10Marostegui) [10:26:15] 10DBA, 10Operations, 10Patch-For-Review, 10Puppet, 10User-jbond: Extend Puppet CA Expiry date - https://phabricator.wikimedia.org/T236277 (10Volans) @jbond just to be on the safe side and to verify the theory, if possible make a quick test that the new cert in the CR is able to verify exiting puppet node... [10:29:31] 10DBA, 10Operations, 10Patch-For-Review, 10Puppet, 10User-jbond: Extend Puppet CA Expiry date - https://phabricator.wikimedia.org/T236277 (10jbond) @volans cheers also re keeping the old cert around, for what its worth it will remain in the git history [10:30:38] marostegui: hi, do you have a momentito? [10:31:00] hauskater: Better tomorrow if your request isn't urgent [10:31:35] Nope, just an advance notice that today/tomorrow there's going to be a BIG rename (ca. 1,5 million edits) [10:32:11] hauskater: ah yes, I am aware, this one, right?: https://phabricator.wikimedia.org/T238512 [10:32:16] Considering jynus' statement that this shouldn't be a problem anymore there's consensus among the renamers to do it [10:33:00] Yup, that one. I was not aware they filed a task [10:33:02] Perfect [10:33:07] Thanks for the heads up [10:33:09] Better safe than sorry is my motto [10:33:16] definitely, appreciate it! [10:33:20] :) [11:49:07] 10DBA, 10Operations, 10Patch-For-Review, 10Puppet, 10User-jbond: Extend Puppet CA Expiry date - https://phabricator.wikimedia.org/T236277 (10jbond) I have tested a puppet run and made sure the debmonitor client works with the new CA. Are there any other service worth validating. specificity dose this c... [11:56:32] 10DBA, 10Operations, 10Patch-For-Review, 10Puppet, 10User-jbond: Extend Puppet CA Expiry date - https://phabricator.wikimedia.org/T236277 (10Volans) The debmonitor test didn't test much as the debmonitor client sends the puppet client cert (not the CA) and it's the server that validates it with the CA.... [12:09:34] 10DBA, 10Patch-For-Review: decommission db1067.eqiad.wmnet - https://phabricator.wikimedia.org/T238297 (10Marostegui) [13:44:48] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) [13:48:08] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) [13:48:31] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) `labswiki` is done: ` root@db1133.eqiad.wmnet[labswiki]> ALTER TABLE /*_*/oauth_accepted_consumer -> ADD oaac_oauth_version TINYINT NOT NULL DEF... [13:51:24] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) `foundationwiki` done directly on the master with replication enabled: ` root@db1123.eqiad.wmnet[foundationwiki]> ALTER TABLE /*_*/oauth_accepted_con... [13:51:54] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) [13:56:23] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) `mediawikiwiki` done directly on the master with replication enabled: ` root@db1123.eqiad.wmnet[mediawikiwiki]> ALTER TABLE /*_*/oauth_accepted_consu... [13:56:31] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) [13:59:36] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) `metawiki` done directly on the master with replication enabled: ` root@db1062.eqiad.wmnet[metawiki]> ALTER TABLE /*_*/oauth_accepted_consumer ->... [13:59:44] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) [13:59:59] 10Blocked-on-schema-change, 10DBA, 10CPT Initiatives (OAuth 2.0): Apply schema changes for OAuth 2.0 - https://phabricator.wikimedia.org/T238370 (10Marostegui) 05Open→03Resolved a:03Marostegui This is all done [14:10:21] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (10Sotiale) The main account went well. (looks fine) I will handle bot account at 14:20 by UTC. [14:14:44] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (10Marostegui) Thanks for the heads up. [14:20:59] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (10Sotiale) >>! In T238512#5675217, @Marostegui wrote: > Thanks for the heads up. Thanks for caring. Can we proceed now? [14:21:16] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (10Marostegui) >>! In T238512#5675233, @Sotiale wrote: >>>! In T238512#5675217, @Marostegui wrote: >> Thanks for the heads up. > > Thanks f... [14:28:16] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (101997kB) https://meta.wikimedia.org/wiki/Special:GlobalRenameProgress?username=Lamchuhan-bot already finished, in case you guys still need... [14:29:00] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (10Marostegui) Excellent - thank you. All good from our side too. [14:30:06] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (10Sotiale) I was so surprised that it was taken care of so quickly. Thank you very much for your attention!! [14:31:25] 10DBA, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 10Operations: Scheduled renaming of two Wikimedia user accounts - https://phabricator.wikimedia.org/T238512 (101997kB) 05Open→03Resolved a:03Sotiale [14:35:37] 10DBA, 10Operations, 10Patch-For-Review, 10Puppet, 10User-jbond: Extend Puppet CA Expiry date - https://phabricator.wikimedia.org/T236277 (10jbond) Thanks Ricardo, I have now validated all the certificates in the private repo ` lines=7 puppetmaster1001 ~ % openssl x509 -in new_ca.pem -noout -subject -d... [14:42:18] 10DBA, 10Operations, 10Patch-For-Review, 10Puppet, 10User-jbond: Extend Puppet CA Expiry date - https://phabricator.wikimedia.org/T236277 (10Ottomata) Hm, uh oh, @elukey we might want to recreate all those certs with a really long expiry :) > i noticed that all of the folders under /srv/private/modules/...