[00:03:04] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Jdforrester-WMF) Did we ever do a stand-alone upcoming EOL announcement/reminder? Closest I can find for REL1_27 is https://lists.wikimedia.org/pipermail/mediawiki... [00:03:17] 10Release-Engineering-Team (Deployment Training Requests), 10User-xSavitar: Deployment training request for **xSavitar** - https://phabricator.wikimedia.org/T281309 (10thcipriani) a:05thciprianiβ†’03xSavitar ## {icon clock-o} After training Hey @xSavitar thanks for attending the late night training session!... [00:07:41] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Jdforrester-WMF) Here's a quick draft: ` Subject: MediaWiki 1.31 will soon be End-of-Life This is a reminder that MediaWiki 1.31 will soon reach its End-of-Life... [00:08:26] (03CR) 10Jforrester: [C: 04-2] "The plan is to do this in June." [integration/config] - 10https://gerrit.wikimedia.org/r/683031 (https://phabricator.wikimedia.org/T281294) (owner: 10Jforrester) [00:23:13] 10MediaWiki-Releasing, 10MW-1.36-notes, 10MW-1.36-release: Release 1.36.0-rc.0 - https://phabricator.wikimedia.org/T279462 (10Jdforrester-WMF) [00:23:15] 10MediaWiki-Releasing, 10MW-1.36-notes, 10MW-1.36-release: Write release announcement for 1.36.0-rc.0 - https://phabricator.wikimedia.org/T279463 (10Jdforrester-WMF) 05Openβ†’03Resolved [00:23:59] 10MediaWiki-Releasing, 10MW-1.36-notes, 10MW-1.36-release: Release 1.36.0-rc.0 - https://phabricator.wikimedia.org/T279462 (10Jdforrester-WMF) 05Openβ†’03Resolved a:03thcipriani https://lists.wikimedia.org/pipermail/mediawiki-announce/2021-April/000275.html [00:24:00] 10MediaWiki-Releasing, 10MW-1.36-notes, 10MW-1.36-release, 10Platform Team Workboards (Initiatives): Release MW 1.36.0 - https://phabricator.wikimedia.org/T279455 (10Jdforrester-WMF) [01:33:13] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO, 10VisualEditor, 10Editing-team (Tracking), and 2 others: Firefox CI tests keep failing in VE with Firefox 68 - https://phabricator.wikimedia.org/T240955 (10Krinkle) .... and it's affecting VisualEditor as well. I'll merge these tasks and re-... [01:34:11] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO, 10VisualEditor, 10Editing-team (Tracking), and 2 others: Firefox CI tests keep failing in VE with Firefox 68 - https://phabricator.wikimedia.org/T240955 (10Krinkle) [01:34:13] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10OOjs, 10Utilities-UnicodeJS: Unit tests for oojs/core and unicodejs repositories unable to launch Firefox in Docker - https://phabricator.wikimedia.org/T280935 (10Krinkle) [01:34:20] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO, 10VisualEditor, 10Editing-team (Tracking), and 2 others: Firefox CI tests keep failing in VE with Firefox 68 - https://phabricator.wikimedia.org/T240955 (10Krinkle) 05Resolvedβ†’03Open [01:58:53] 10Continuous-Integration-Infrastructure, 10Cloud-Services: Stop using integration/composer and then archive the repo - https://phabricator.wikimedia.org/T249949 (10Jdforrester-WMF) [02:00:07] 10Continuous-Integration-Infrastructure, 10Cloud-Services: Stop using integration/composer and then archive the repo - https://phabricator.wikimedia.org/T249949 (10Jdforrester-WMF) [02:01:19] 10Continuous-Integration-Infrastructure, 10Cloud-Services: Stop using integration/composer and then archive the repo - https://phabricator.wikimedia.org/T249949 (10Jdforrester-WMF) @Legoktm, is there a nice way to stop extdist still using this? Same for WMCS folk and the Toolforge usage. I can manually re-bui... [02:01:34] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Reedy) More usually it's been tacked onto another email that was being sent at the right time - https://lists.wikimedia.org/pipermail/mediawiki-announce/2020-Septe... [02:03:13] (03PS1) 10Jforrester: Upgrade composer to 1.10.22, per CVE-2021-29472 [integration/composer] - 10https://gerrit.wikimedia.org/r/683764 [02:03:29] (03PS2) 10Jforrester: Upgrade composer to 1.10.22, per CVE-2021-29472 [integration/composer] - 10https://gerrit.wikimedia.org/r/683764 (https://phabricator.wikimedia.org/T249949) [02:05:44] 10Continuous-Integration-Infrastructure, 10Cloud-Services, 10Patch-For-Review: Stop using integration/composer and then archive the repo - https://phabricator.wikimedia.org/T249949 (10Legoktm) We can switch to the packaged version of composer as part of {T277249}. [02:06:28] (03PS2) 10Jforrester: dockerfiles: [node10] Switch OS from stretch to buster and mostly cascade [integration/config] - 10https://gerrit.wikimedia.org/r/674185 (https://phabricator.wikimedia.org/T278203) [02:17:52] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Jdforrester-WMF) That means that we'll announce the final release as such with no prior notice. Sending something now would give people three months' notice to get... [02:25:14] (03CR) 10Krinkle: [C: 03+1] "Do you know off-hand what this does to Chromium and Firefox versions? I believe at least Chromium will get a significant bump which will n" [integration/config] - 10https://gerrit.wikimedia.org/r/674185 (https://phabricator.wikimedia.org/T278203) (owner: 10Jforrester) [02:26:38] (03CR) 10Jforrester: "> Patch Set 2: Code-Review+1" [integration/config] - 10https://gerrit.wikimedia.org/r/674185 (https://phabricator.wikimedia.org/T278203) (owner: 10Jforrester) [02:29:04] (03CR) 10Jforrester: "> Patch Set 2:" [integration/config] - 10https://gerrit.wikimedia.org/r/674185 (https://phabricator.wikimedia.org/T278203) (owner: 10Jforrester) [02:35:21] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [node10] Switch OS from stretch to buster and mostly cascade [integration/config] - 10https://gerrit.wikimedia.org/r/674185 (https://phabricator.wikimedia.org/T278203) (owner: 10Jforrester) [02:36:22] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO, 10VisualEditor, 10Editing-team (Tracking), and 2 others: Firefox CI tests keep failing in VE with Firefox 68 - https://phabricator.wikimedia.org/T240955 (10Jdforrester-WMF) [02:36:34] (03Merged) 10jenkins-bot: dockerfiles: [node10] Switch OS from stretch to buster and mostly cascade [integration/config] - 10https://gerrit.wikimedia.org/r/674185 (https://phabricator.wikimedia.org/T278203) (owner: 10Jforrester) [02:37:29] !log Docker: Publishing node10 images based on buster T278203 T240955 [02:37:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [02:37:33] T240955: Firefox CI tests keep failing in VE with Firefox 68 - https://phabricator.wikimedia.org/T240955 [02:37:33] T278203: Migrate all CI jobs from stretch to buster or later and drop stretch testing support - https://phabricator.wikimedia.org/T278203 [03:01:38] Krinkle: Do you know off-hand if we actually use the chromedriver package inside quibble? [03:03:53] James_F: errr.. I dont know anymore. A lot of stuff has changed there. [03:03:59] Yeah. [03:04:01] * James_F shrugs. [03:04:10] I think core now uses devtools [03:04:20] but I don't know if that has been applies to all/enough extensions [03:04:22] For some reason the 'chromedriver' alias for 'chromium-driver' has somehow been unpublished from stretch, so the builds fail. [03:04:27] ack. [03:04:30] or whether non-mw stuff needs it (e.g. karma might care) [03:04:37] I don't think so though [03:04:41] Oh, yeah, good point. Hard to tell. [03:04:57] I'll build explicitly on chromium-driver rather than drop it for now, then. [03:19:37] cool [03:19:46] yeah so long as the command has the same name, the pckage name should be fine to change [03:19:59] * James_F nods. [03:20:45] https://packages.debian.org/stretch/chromium-driver [03:20:50] https://packages.debian.org/stretch/chromedriver [03:20:57] looks like it was an alias for 1 release [03:21:07] under jessie it was only 'chromedriver' [03:21:15] then an alias, then no more [03:21:23] I guess someone liked it better that way :P [03:21:26] Yeah. [03:22:18] * Krinkle sings in high pitch 🎢 "Istanbul was Constantinople" [03:22:25] alirght, goodnight [03:22:26] o/ [03:22:38] Good night. :-) [03:44:20] (03PS5) 10Jforrester: jjb: [integration-quibble-apache-fullrun] Switch to buster [integration/config] - 10https://gerrit.wikimedia.org/r/680423 (https://phabricator.wikimedia.org/T252432) [03:45:14] (03CR) 10Jforrester: [C: 03+2] jjb: [integration-quibble-apache-fullrun] Switch to buster [integration/config] - 10https://gerrit.wikimedia.org/r/680423 (https://phabricator.wikimedia.org/T252432) (owner: 10Jforrester) [03:46:17] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Reedy) Yeah. It was more an answer to your "Did we ever do a stand-alone upcoming EOL announcement/reminder?" April is nearly over. May starts soon... So even end... [03:46:22] (03Merged) 10jenkins-bot: jjb: [integration-quibble-apache-fullrun] Switch to buster [integration/config] - 10https://gerrit.wikimedia.org/r/680423 (https://phabricator.wikimedia.org/T252432) (owner: 10Jforrester) [03:46:30] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Reedy) [03:47:29] (03PS1) 10Jforrester: dockerfiles: Drop quibble-stretch-php72-apache, unused [integration/config] - 10https://gerrit.wikimedia.org/r/683778 (https://phabricator.wikimedia.org/T252432) [03:48:20] (03CR) 10Jforrester: [C: 03+2] dockerfiles: Drop quibble-stretch-php72-apache, unused [integration/config] - 10https://gerrit.wikimedia.org/r/683778 (https://phabricator.wikimedia.org/T252432) (owner: 10Jforrester) [03:49:28] (03Merged) 10jenkins-bot: dockerfiles: Drop quibble-stretch-php72-apache, unused [integration/config] - 10https://gerrit.wikimedia.org/r/683778 (https://phabricator.wikimedia.org/T252432) (owner: 10Jforrester) [03:59:45] (03PS2) 10Jforrester: Zuul: Update T226945 comment now we've dropped PHP70/71 support [integration/config] - 10https://gerrit.wikimedia.org/r/683033 [03:59:47] (03PS2) 10Jforrester: jjb: [quibble] Drop PHP70 and PHP71 testing [integration/config] - 10https://gerrit.wikimedia.org/r/683032 [03:59:49] (03PS1) 10Jforrester: Zuul: [mediawiki/core] Drop PHP70/71 testing for REL1_31 [integration/config] - 10https://gerrit.wikimedia.org/r/683779 [03:59:51] (03PS1) 10Jforrester: dockerfiles: Drop quibble-stretch-php7{0,1}, unused [integration/config] - 10https://gerrit.wikimedia.org/r/683780 [04:41:34] (03PS1) 10Jforrester: dockerfiles: [quibble-stretch] Rebuild manually on ci-stretch directly [integration/config] - 10https://gerrit.wikimedia.org/r/683783 [04:43:10] (03CR) 10Jforrester: "This fails to build, for some reason:" [integration/config] - 10https://gerrit.wikimedia.org/r/683783 (owner: 10Jforrester) [05:22:29] 10Beta-Cluster-Infrastructure, 10ContentTranslation, 10Wikimedia-Site-requests, 10Language-Team (Language-2021-April-June), 10MW-1.37-notes (1.37.0-wmf.4; 2021-05-04): Beta cluster Special:ContentTranslation Error: Unsupported operand types - https://phabricator.wikimedia.org/T276656 (10Nikerabbit) https... [07:39:43] 10Release-Engineering-Team, 10GerritBot, 10Developer Productivity, 10Regression: Gerritbot turns "+" into space, thus breaking most Gerrit URLs - https://phabricator.wikimedia.org/T280197 (10hashar) The change for T93331 is to wrap the abandoning reason with a Phabricator literal block marker: `%%%` . The... [08:11:58] 10Release-Engineering-Team, 10GerritBot, 10Developer Productivity, 10Regression: Gerritbot turns "+" into space, thus breaking most Gerrit URLs - https://phabricator.wikimedia.org/T280197 (10hashar) I went to elevate a bunch of logging level in Gerrit. Loggers can be found using: ` name=gerrit logging ls |... [08:20:36] 10Release-Engineering-Team, 10GerritBot, 10Developer Productivity, 10Patch-For-Review, 10Regression: Gerritbot turns "+" into space, thus breaking most Gerrit URLs - https://phabricator.wikimedia.org/T280197 (10hashar) Fun thing, @Paladox raised that on Google Closure Template mailing list at https://gro... [08:20:40] 10Release-Engineering-Team, 10GerritBot, 10Developer Productivity, 10Patch-For-Review, 10Regression: Gerritbot turns "+" into space, thus breaking most Gerrit URLs - https://phabricator.wikimedia.org/T280197 (10hashar) a:03hashar [08:39:25] (03CR) 10Addshore: "Looks awesome, but I'll let you releng folks handle this patch unless I'm poked :)" [integration/config] - 10https://gerrit.wikimedia.org/r/683753 (https://phabricator.wikimedia.org/T248779) (owner: 10Jeena Huneidi) [08:55:55] 10Beta-Cluster-Infrastructure, 10ContentTranslation, 10Wikimedia-Site-requests, 10Language-Team (Language-2021-April-June), 10MW-1.37-notes (1.37.0-wmf.4; 2021-05-04): Beta cluster Special:ContentTranslation Error: Unsupported operand types - https://phabricator.wikimedia.org/T276656 (10Pginer-WMF) 05Op... [10:13:47] 10Beta-Cluster-Infrastructure, 10SRE: Possible to run writes (e.g. UPDATE) on Beta Cluster replica - https://phabricator.wikimedia.org/T110115 (10Majavah) Instead of tracking the current primary in hiera/etc, I'd propose to just set `read_only = 1` by default on all beta database servers. The replicas should b... [10:19:03] I'm experimenting with envoy/cfssl on deployment-mediawiki11 for T206158, shouldn't cause downtime but if something breaks let me know [10:19:04] T206158: Investigate setting up HTTPS directly on beta appservers - https://phabricator.wikimedia.org/T206158 [10:49:04] 10Release-Engineering-Team (Deployment Training Requests), 10User-xSavitar: Deployment training request for **xSavitar** - https://phabricator.wikimedia.org/T281309 (10xSavitar) Thank you very much @thcipriani. The training was excellent and I'll for sure attend about 2 - 4 more sessions. I've requested shell... [11:06:07] 10Beta-Cluster-Infrastructure: Investigate setting up HTTPS directly on beta appservers - https://phabricator.wikimedia.org/T206158 (10Majavah) Now that CFSSL is a thing, I took a look at this as the certs are much easier to generate. I added the following hiera to deployment-mediawiki11: `lang=yaml,lines=10 pro... [11:51:08] (03CR) 10Lars Wirzenius: [C: 03+1] "LGTM, but I don't know this config enough to +2." [integration/config] - 10https://gerrit.wikimedia.org/r/683753 (https://phabricator.wikimedia.org/T248779) (owner: 10Jeena Huneidi) [11:51:38] 10Beta-Cluster-Infrastructure: Investigate setting up HTTPS directly on beta appservers - https://phabricator.wikimedia.org/T206158 (10Majavah) >>! In T206158#7048731, @Majavah wrote: > So Envoy will only be installed if `profile::envoy::ensure` is present, but if that's set to present you need to have at least... [11:51:43] (03CR) 10Lars Wirzenius: [C: 03+2] fix: improve scap update-wikiversions error reporting [tools/scap] - 10https://gerrit.wikimedia.org/r/683744 (owner: 10Ahmon Dancy) [11:52:27] (03Merged) 10jenkins-bot: fix: improve scap update-wikiversions error reporting [tools/scap] - 10https://gerrit.wikimedia.org/r/683744 (owner: 10Ahmon Dancy) [11:55:21] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Add tooling to pgp-public-keys.git to make using and maintaining it easier - https://phabricator.wikimedia.org/T279051 (10LarsWirzenius) https://gerrit.wikimedia.org/r/c/pgp-public-keys/+/683840 [12:37:10] !log force reboot deployment-cache-text06, not letting me to log in, this will disrupt beta cluster availability [12:37:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:53:19] 10Beta-Cluster-Infrastructure, 10User-Majavah: Investigate setting up HTTPS directly on beta appservers - https://phabricator.wikimedia.org/T206158 (10Majavah) a:03Majavah So this is now working on deployment-mediawiki11 with the help of jbond to get CFSSL working with envoy. Next step is to get caches and o... [14:05:56] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments: 1.37.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T278347 (10LarsWirzenius) 05Openβ†’03Resolved Train seems to have been OK since yesterady, closing task. [14:08:29] 10phan, 10Upstream: Analyze why phan does not report a possible undeclared variable on short-circuit code with pass-by-ref variable - https://phabricator.wikimedia.org/T281524 (10Daimona) Not fixed on phan master, see [[https://phan.github.io/demo/?code=%3C%3Fphp%0A%0Afunction+getUnclosedTags%28+%24message+%29... [14:13:30] !log deployment-cache-text: trying out useusing HTTPS for backend traffic to deployment-mediawiki11 T206158 [14:13:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:13:33] T206158: Investigate setting up HTTPS directly on beta appservers - https://phabricator.wikimedia.org/T206158 [14:15:41] !log revert above as it's not working, T206158 [14:15:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:21:32] !log add profile::pki::client to all deployment-prep instances to trust deployment-prep cfssl certificates, already deployed on production [14:21:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:22:08] 10Release-Engineering-Team, 10GerritBot, 10Developer Productivity, 10Regression: Gerritbot turns "+" into space, thus breaking most Gerrit URLs - https://phabricator.wikimedia.org/T280197 (10hashar) Tried again and there is a server side error: ` name=TRACE com.googlesource.gerrit.plugins.its.base.workflow... [14:23:18] (03PS1) 10Arlolra: Point Parsoid's documentation to main [integration/docroot] - 10https://gerrit.wikimedia.org/r/683902 (https://phabricator.wikimedia.org/T254804) [14:23:41] 10Project-Admins: Request Project (cfssl-pki) for pki tasks - https://phabricator.wikimedia.org/T281371 (10jbond) [14:41:33] 10Release-Engineering-Team, 10GerritBot, 10Developer Productivity, 10Patch-For-Review, 10Regression: Gerritbot turns "+" into space, thus breaking most Gerrit URLs - https://phabricator.wikimedia.org/T280197 (10hashar) My message had `Abandonning ...` , the template thus has `%%%Abandonning` and I guess... [15:01:02] (03CR) 10Subramanya Sastry: [C: 03+1] Point Parsoid's documentation to main [integration/docroot] - 10https://gerrit.wikimedia.org/r/683902 (https://phabricator.wikimedia.org/T254804) (owner: 10Arlolra) [15:02:10] 10Beta-Cluster-Infrastructure, 10SRE-swift-storage: Swift uses http in deployment-prep, https in production - https://phabricator.wikimedia.org/T277990 (10Majavah) Now that we have cfssl I created a CNAME for a svc domain name and configured an envoy proxy in front of Swift. This means that it's now available... [15:59:10] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Release Pipeline: Implement pipeline config validation - https://phabricator.wikimedia.org/T225335 (10dduvall) a:03dduvall [16:22:41] Hi, is import broken when importing with templates (even if the file is only 300 kb? Tried a few ways but can't get it to work... [16:42:52] (03Abandoned) 10Arlolra: Point Parsoid's documentation to main [integration/docroot] - 10https://gerrit.wikimedia.org/r/683902 (https://phabricator.wikimedia.org/T254804) (owner: 10Arlolra) [17:02:14] 10Beta-Cluster-Infrastructure, 10Jade, 10MediaWiki-ContentHandler, 10Patch-For-Review, and 2 others: Beta cluster: The content model 'JadeJudgment' is not registered - https://phabricator.wikimedia.org/T247476 (10DannyS712) >>! In T247476#7047673, @Ladsgroup wrote: > I can't reproduce the error anymore giv... [17:11:06] (03PS1) 10Ahmon Dancy: branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) [17:11:15] (03CR) 10jerkins-bot: [V: 04-1] branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) (owner: 10Ahmon Dancy) [17:12:59] (03PS2) 10Ahmon Dancy: branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) [17:14:39] (03CR) 10jerkins-bot: [V: 04-1] branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) (owner: 10Ahmon Dancy) [17:22:59] (03PS3) 10Ahmon Dancy: branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) [17:23:02] James_F: thcipriani: It end of Node 10 is near. E.g. https://github.com/nodejs/node/issues/35367#issuecomment-699600236 [17:23:17] we can expect to start seeing flies dropping from the sky pretty soon now in the coming weeks/months [17:23:46] people making minor auto-updates and breaking our CI etc. [17:23:51] (03CR) 10jerkins-bot: [V: 04-1] branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) (owner: 10Ahmon Dancy) [17:26:02] 10MediaWiki-Releasing, 10MW-1.36-notes, 10MW-1.36-release: Finish up RELEASE-NOTES for 1.36.0 - https://phabricator.wikimedia.org/T279461 (10Jdforrester-WMF) a:03Jdforrester-WMF [17:27:57] Krinkle: Right. I guess I should prepare for node14 builds? [17:28:28] https://phabricator.wikimedia.org/T267888 [17:28:39] maybe SRE will not decline this time for it to be a component deb. [17:28:43] now that it is EOL [17:28:48] well, I guess debian supports for a little longer [17:29:20] but we can also go back to how it was with node 4 and not use official packages :/ [17:29:39] Indeed. :-( [17:36:40] (03PS4) 10Ahmon Dancy: branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) [17:38:02] (03CR) 1020after4: [C: 03+1] branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) (owner: 10Ahmon Dancy) [17:40:28] (03CR) 10Brennen Bearnes: [C: 03+2] branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) (owner: 10Ahmon Dancy) [17:41:34] (03Merged) 10jenkins-bot: branch.py: Convert deleted wmf branches to tags [tools/release] - 10https://gerrit.wikimedia.org/r/683934 (https://phabricator.wikimedia.org/T250095) (owner: 10Ahmon Dancy) [17:44:34] 10Beta-Cluster-Infrastructure, 10Jade, 10MediaWiki-ContentHandler, 10Patch-For-Review, and 2 others: Beta cluster: The content model 'JadeJudgment' is not registered - https://phabricator.wikimedia.org/T247476 (10Ladsgroup) Is that error related to jade and its content model? It doesn't look like it TBH [17:49:11] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10MediaWiki-Release-Tools, 10Patch-For-Review, 10User-brennen: branch.py: Convert deleted wmf branches to tags - https://phabricator.wikimedia.org/T250095 (10dancy) 05Openβ†’03Resolved [17:49:15] 10Beta-Cluster-Infrastructure, 10Jade, 10MediaWiki-ContentHandler, 10Patch-For-Review, and 2 others: Beta cluster: The content model 'JadeJudgment' is not registered - https://phabricator.wikimedia.org/T247476 (10DannyS712) >>! In T247476#7049438, @Ladsgroup wrote: > Is that error related to jade and its c... [18:13:52] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10MediaWiki-Release-Tools, 10Patch-For-Review, 10User-brennen: branch.py: Convert deleted wmf branches to tags - https://phabricator.wikimedia.org/T250095 (10greg) >>! In T250095#7049451, @Krinkle wrote: >>>! In T250095#7044227, @brennen wrote: >>> What value do... [18:13:59] James_F: I tried to suggest a way forward at https://phabricator.wikimedia.org/T267891#7049504 [18:15:39] legoktm: I'm OK with node 12 personally, sure. It means twice the migrations (10->12, 12->14) but theoretically each will be less bumpy. [18:17:02] legoktm: And yes, we're more cautious in production dependencies but I expect e.g. bluebird might well drop node 10 support today/tomorrow. [18:18:23] yeah, I'm really not sure about that [18:18:32] maybe services move to bullseye? ;) [18:18:36] I mean… [18:18:45] The point of the pipeline is that services can move independently. [18:18:49] So… yes? :-) [18:18:56] But not your job, and not mine either. :-D [18:21:48] 10Release-Engineering-Team (Doing), 10GitLab-Test, 10GitLab (Initialization), 10User-brennen: Investigate whether issues, operations, wikis, etc. can be disabled globally on GitLab - https://phabricator.wikimedia.org/T264231 (10brennen) [18:24:02] (03PS1) 10Ahmon Dancy: feat: Fail fast if non-existent version is passed to scap prep [tools/scap] - 10https://gerrit.wikimedia.org/r/683943 [18:25:24] (03PS1) 10Jforrester: jjb: Switch node jobs to buster images [integration/config] - 10https://gerrit.wikimedia.org/r/683945 (https://phabricator.wikimedia.org/T278203) [18:30:11] heh [18:30:14] legoktm: actually, do you know what node version debian will use for their next release and when that would come out? [18:30:28] Krinkle: nodejs12 is in bullseye [18:30:29] maybe I'm wrong about us having to do a custom version at all if we wait a few months [18:30:34] and I'm working on creating the bullseye image right now [18:30:36] yeah but I mean 14 [18:30:54] https://phabricator.wikimedia.org/T267891#7049542 [18:31:05] bookworm comes after bullseye, probably in June 2023? [18:31:14] idk if it'll ship with 14 or something later [18:31:50] wow, that's a long time out [18:31:51] right [18:32:00] and we don't use debian node 10 today, do we? [18:32:04] afaik we use a component deb [18:32:17] we do [18:32:53] was that a switch, or do I misremember [18:33:13] https://apt-browser.toolforge.org/stretch-wikimedia/component/node10/ was a backport of the buster packages for stretch [18:33:22] maybe we switched from stretch + component node 10 to buster + deb node 10? [18:33:45] right, so it caught up and it just stayed on the same node version under the deb upgrade [18:33:45] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/683945 (https://phabricator.wikimedia.org/T278203) (owner: 10Jforrester) [18:33:50] https://gerrit.wikimedia.org/r/plugins/gitiles/operations/docker-images/production-images/+/refs/heads/master/images/nodejs10/buster/Dockerfile.template [18:34:00] right [18:34:08] but what do you expect we'll do next year after node 12 goes EOL [18:34:17] would we go back to a component until 2023? [18:34:19] Our component on stretch is 10.12.2 [18:34:23] The deb is 10.24.0 [18:34:41] So I'm running around fixing repos that "cleverly" pin their built dependencies to their node version. [18:35:24] (03Merged) 10jenkins-bot: jjb: Switch node jobs to buster images [integration/config] - 10https://gerrit.wikimedia.org/r/683945 (https://phabricator.wikimedia.org/T278203) (owner: 10Jforrester) [18:37:11] Krinkle: maybe. [18:37:30] My understanding is that nodejs in Debian isn't really security supported anyways because v8 has no/limited security support [18:37:46] Yeah. :-( [18:38:07] https://salsa.debian.org/debian/debian-security-support/-/commit/169aea4b8a0df8b9f45eb944e2f1d57404cfd16f [18:38:11] maybe it is supported now [18:38:21] Oh, huh. [18:38:49] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=931376#20 [18:39:29] in which case the nodejs10 EOL is only an ecosystem probelm (everyone dropping support), not really a security one [18:40:09] Yeah, like PHP 7.2. [18:40:47] except the js community gets andrenaline rush when they get to drop a version from their .travis.yml file and feel good. [18:41:31] Krinkle: Let's not start throwing stones, given that there's no 7.2 & 8.0 compatible version of several of our upstream PHP dependencies. [18:41:50] it's not perfect, but php-parallel-lint did just restore php 5.3 compat :P [18:42:01] They did? Huh. Elastic won't. [18:42:14] Production dependency issues are more problematic. [18:42:25] Elastic? speaking of throwing stones... [18:42:42] :-) [18:43:16] php 8.0 came a bit sooner than I think a lot of people expected, pandemic probably didn't help. [18:43:22] Fair. [18:43:39] I tried to build it when it was in beta and it broke, so put it out of my mind. [18:43:47] A few weeks later and it was released. [18:43:51] Surprise! [18:44:01] anyway, bottom line, staying on node 10 will be hard for us, stuff will stop working and packages will likely not be willing to pin dependencies to avoid breaking changes indirectly to support node 10. [18:45:06] I [18:45:21] I'll leave it up for SRE to decide their investments. I'd be happy with a supported node 12 image, very happy. [18:45:48] 0de9c83e95c0eb9a7e339bdf70f3f854834893498862c480c4f619a6a0281b24 wikimedia-buster.pub.gpg [18:45:48] 0de9c83e95c0eb9a7e339bdf70f3f854834893498862c480c4f619a6a0281b24 wikimedia-stretch.pub.gpg [18:45:50] :thinking: [18:45:55] Umm. [18:46:20] yeah, really threw me off. it's just the same key because we don't change keys for each release [18:46:29] Oh! [18:46:31] I kept trying to find the bullseye key and came up empty :p [18:46:40] If we think doing deb-node12 work now is worth the savings now compared to doing more work next year, or if we think it's likely that next year deb will have a node14 package for us, so be it! [18:46:51] legoktm: cp wikimedia-buster.pub.gpg wikimedia-bullseye.pub.gpg [18:47:02] But also we don't want to steal legoktm's time from other SRE work. [18:47:22] Majavah: nooooooo. I did rm wikimedia-stretch.pub.gpg && mv wikimedia-buster.pub.gpg wikimedia.pub.gpg [18:47:37] well we need a bullseye image anyways, so I'm happy to do that [18:48:30] heh, we also need that for toolforge images and I was hoping someone else would create a need for those so I would not need to bug you [18:49:40] It's all coming together. [18:49:54] CI and Toolforge users unite to make yet more work for SRE! [18:49:57] Oh, wait, that sounds bad. [18:50:55] this is the second thing this week i've found like that [18:51:09] we had redis-{trusty,jessie,stretch,buster}.conf [18:51:19] All identical? [18:51:21] jessie, stretch, buster were all identical [18:51:28] Ha. [18:52:51] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-brennen: 1.37.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281145 (10Jdlrobson) [18:55:14] What, me, premature? :-) [18:55:18] (03PS1) 10Jforrester: dockerfiles: Provide ci-bullseye based on wikimedia-bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/683973 [18:55:37] (03CR) 10Jforrester: [C: 04-1] "Let's give lego a second." [integration/config] - 10https://gerrit.wikimedia.org/r/683973 (owner: 10Jforrester) [18:55:38] the best part was your IRC message came before wikibugs [18:55:46] * James_F grins. [18:57:24] legoktm: BTW, is https://gerrit.wikimedia.org/r/c/integration/config/+/644367 good to go? [18:57:30] It's been sitting around for a while. [18:57:58] oh, yes probably [18:58:17] I assume Daimona hasn't broken their repo in the meantime [18:58:32] Want me to deploy? [18:58:41] please :) [18:59:07] (03PS4) 10Jforrester: docker: Switch back to Daimona's repo for phan-taint-check demo [integration/config] - 10https://gerrit.wikimedia.org/r/644367 (https://phabricator.wikimedia.org/T257301) (owner: 10Legoktm) [18:59:23] (03CR) 10Jforrester: [C: 03+2] docker: Switch back to Daimona's repo for phan-taint-check demo [integration/config] - 10https://gerrit.wikimedia.org/r/644367 (https://phabricator.wikimedia.org/T257301) (owner: 10Legoktm) [19:02:23] James_F: btw, when I started disabling FF in a few repos, I knew upsream mozila said FF78esr would support docker, I just had no idea it was already in the next debian channel [19:02:28] I assumed it'd be until 2023 [19:02:35] * James_F grins. [19:02:36] but loks like it made it into the current stable after alll [19:02:44] Yeah, I knew it was updated but surprised how much. [19:03:53] Debian follows the latest ESR [19:04:36] 10LibUp: libraryupgrader missed some repos due to Generic.Files.LineLength.TooLong - https://phabricator.wikimedia.org/T203830 (10Umherirrender) Happen on canary right now: https://libraryupgrader2.wmcloud.org/logs2/256097 That breaks the whole upgrade process until manual handling. Not sure if there is a good... [19:05:43] 10LibUp: libraryupgrader missed some repos due to Generic.Files.LineLength.TooLong - https://phabricator.wikimedia.org/T203830 (10Jdforrester-WMF) I think it's reasonable if there's only a handful of repos and we work through them manually. [19:16:44] (03CR) 10Jforrester: [C: 03+2] "…" [integration/config] - 10https://gerrit.wikimedia.org/r/644367 (https://phabricator.wikimedia.org/T257301) (owner: 10Legoktm) [19:17:12] Krinkle: BTW, I've updated all CI, so OOjs and UnicodeJS will have FF 78 too. [19:18:29] (03Merged) 10jenkins-bot: docker: Switch back to Daimona's repo for phan-taint-check demo [integration/config] - 10https://gerrit.wikimedia.org/r/644367 (https://phabricator.wikimedia.org/T257301) (owner: 10Legoktm) [19:19:33] James_F: what you're saying is that the email I sent out last night contained a statement that only remained true for less than 12 hours :P [19:19:59] "Fresh is now in sync with CI again" [19:20:06] :-D [19:20:09] Sorry. [19:20:37] legoktm: Will you want a bullseye puppet test run? [19:21:54] !log Docker: Publishing mediawiki-phan-taint-check-demo:0.1.1 for T257301 [19:21:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:21:57] T257301: Create a web demo for phan taint-check - https://phabricator.wikimedia.org/T257301 [19:30:49] see https://phabricator.wikimedia.org/T281596#7049780 [19:30:54] going to figure out what to do after lunch [19:31:33] Fun. [19:37:37] (03PS2) 10Jforrester: jjb: [mw-tools-phan-demos-publish] Bump image to 0.1.1 [integration/config] - 10https://gerrit.wikimedia.org/r/644368 (owner: 10Legoktm) [19:37:43] (03PS3) 10Jforrester: jjb: [mw-tools-phan-demos-publish] Bump image to 0.1.1 [integration/config] - 10https://gerrit.wikimedia.org/r/644368 (owner: 10Legoktm) [19:38:31] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/644368 (owner: 10Legoktm) [19:39:37] (03Merged) 10jenkins-bot: jjb: [mw-tools-phan-demos-publish] Bump image to 0.1.1 [integration/config] - 10https://gerrit.wikimedia.org/r/644368 (owner: 10Legoktm) [19:48:38] (03PS1) 10Jforrester: Revert "dockerfiles: [quibble-{stretch,buster}-php72] Drop mediawiki-coverage.sh, unused" [integration/config] - 10https://gerrit.wikimedia.org/r/683955 [19:51:17] (03CR) 10Jforrester: [C: 03+2] Revert "dockerfiles: [quibble-{stretch,buster}-php72] Drop mediawiki-coverage.sh, unused" [integration/config] - 10https://gerrit.wikimedia.org/r/683955 (owner: 10Jforrester) [19:52:23] (03Merged) 10jenkins-bot: Revert "dockerfiles: [quibble-{stretch,buster}-php72] Drop mediawiki-coverage.sh, unused" [integration/config] - 10https://gerrit.wikimedia.org/r/683955 (owner: 10Jforrester) [19:56:10] James_F: The domain name mediawiki-coverage.sh is available for purchase :P [19:56:40] Too kind. [20:07:51] (03PS1) 10Ahmon Dancy: Generate a VERSION file in log dir [integration/config] - 10https://gerrit.wikimedia.org/r/683987 [20:09:02] (03CR) 10Thcipriani: [C: 03+2] Generate a VERSION file in log dir [integration/config] - 10https://gerrit.wikimedia.org/r/683987 (owner: 10Ahmon Dancy) [20:10:37] (03Merged) 10jenkins-bot: Generate a VERSION file in log dir [integration/config] - 10https://gerrit.wikimedia.org/r/683987 (owner: 10Ahmon Dancy) [20:13:15] !log Updating docker-pkg files on contint primary for https://gerrit.wikimedia.org/r/c/integration/config/+/683987 [20:13:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:18:09] (03PS2) 10Jforrester: dockerfiles: [quibble-stretch] Rebuild manually on ci-stretch directly [integration/config] - 10https://gerrit.wikimedia.org/r/683783 [20:18:13] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [quibble-stretch] Rebuild manually on ci-stretch directly [integration/config] - 10https://gerrit.wikimedia.org/r/683783 (owner: 10Jforrester) [20:21:26] (03CR) 10Jforrester: [C: 03+2] setuptools_scm 6.0.0 no more support python 3.5 [integration/quibble] - 10https://gerrit.wikimedia.org/r/676452 (owner: 10Hashar) [20:23:30] (03CR) 10Jforrester: [C: 03+2] "This is needed for the latest re-build of quibble images on stretch. :-(" [integration/quibble] - 10https://gerrit.wikimedia.org/r/676452 (owner: 10Hashar) [20:26:41] (03CR) 10jerkins-bot: [V: 04-1] setuptools_scm 6.0.0 no more support python 3.5 [integration/quibble] - 10https://gerrit.wikimedia.org/r/676452 (owner: 10Hashar) [20:34:03] 10Continuous-Integration-Infrastructure, 10Quibble: Release quibble 0.0.47 so we can build quibble stretch images again (py3.5 dependency) - https://phabricator.wikimedia.org/T281607 (10Jdforrester-WMF) [20:58:24] 10LibUp: libraryupgrader missed some repos due to Generic.Files.LineLength.TooLong - https://phabricator.wikimedia.org/T203830 (10Umherirrender) 05Openβ†’03Declined That sounds okay to me, too. [21:01:29] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-brennen: Explore tools for remote pairing - https://phabricator.wikimedia.org/T240484 (10brennen) Yak Shaving Day notes: **Tried**: https://replit.com/ - There's a free tier with 500 megs storage, 500 megs memory - Docker containers on Google cloud... [21:12:02] (03PS1) 10Ahmon Dancy: Bump train-deploy-notes job's image version to 0.0.6-s3 [integration/config] - 10https://gerrit.wikimedia.org/r/683991 [21:12:27] (03CR) 10Ahmon Dancy: [C: 03+2] Bump train-deploy-notes job's image version to 0.0.6-s3 [integration/config] - 10https://gerrit.wikimedia.org/r/683991 (owner: 10Ahmon Dancy) [21:12:39] (03CR) 10Ahmon Dancy: [C: 03+2] "already deployed, so self-approving" [integration/config] - 10https://gerrit.wikimedia.org/r/683991 (owner: 10Ahmon Dancy) [21:13:46] (03Merged) 10jenkins-bot: Bump train-deploy-notes job's image version to 0.0.6-s3 [integration/config] - 10https://gerrit.wikimedia.org/r/683991 (owner: 10Ahmon Dancy) [21:14:33] 10Project-Admins, 10Jade, 10Machine-Learning-Team: Archive Jade project - https://phabricator.wikimedia.org/T281430 (10calbon) Awesome, thanks Peachy88 [21:15:43] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Readers-Web-Backlog: MinervaNeue legacy ruby browser tests need to be updated to work on buster/ruby2.5 - https://phabricator.wikimedia.org/T280901 (10Jdlrobson) a:03ovasileva [21:56:08] 10Release-Engineering-Team (Doing), 10SRE, 10GitLab (Initialization), 10User-brennen: SSH Access of Git data in GitLab - https://phabricator.wikimedia.org/T276148 (10Sergey.Trofimovsky.SF) Here it is, requesting settings review: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/gitlab-ansible/+/re...