[00:08:55] (03PS2) 10Hslater: Use bluespice template for DrawioEditor & PageCheckout [integration/config] - 10https://gerrit.wikimedia.org/r/1132780 [03:27:30] 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible, 07Wikimedia-production-error: Intermittant "PHP Notice: Undefined variable: namespace" during beta-scap-sync-world - https://phabricator.wikimedia.org/T378046#10697072 (10Krinkle) 05Openβ†’03Resolved a:03Krinkle [03:27:32] 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible, 07Wikimedia-production-error: Intermittant "PHP Notice: Undefined variable: namespace" during beta-scap-sync-world - https://phabricator.wikimedia.org/T378046#10697075 (10Krinkle) a:05Krinkleβ†’03None [07:06:38] (03CR) 10Hashar: utils: script to manually launch jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [07:25:39] 10Phabricator, 06DBA: Prepare a database test for m3 - https://phabricator.wikimedia.org/T390034#10697318 (10Marostegui) @brennen can you describe a bit exactly what you need? Do you need just a database to access from any phabX host to perform the schema change upgrade? Would you perform that manually? I am t... [07:28:30] (03CR) 10Krinkle: utils: script to manually launch jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [07:29:26] Krinkle: there were some more dependencies added yesterday but I guess they will be caught when your change is rebased [07:29:41] I think about running the script today and see what happens [07:29:45] hashar: ack. I'm looking now at potentially doing a manual build in Jenkins UI for one or two repos [07:29:52] okay that would be great [07:30:01] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ArticlePlaceholder/+/1131926 > https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php74-noselenium/84837/rebuild/parameterized [07:30:03] I feel it is easier if I babysit the script given the code certainly has defect here and there [07:30:04] I don't see this variable there [07:30:15] I'm guessing the default is to generate it at runtime? [07:30:36] the job in Jenkins only has the Zuul parameter and some that are explicitly defined [07:30:36] or is it passed by zuul in a hidden way? [07:30:55] https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php74-noselenium/84837/parameters/ [07:30:57] ah I see it there [07:30:57] but Zuul/Gearman can inject arbitrary parameters to the build, and we even had to enable a feature flag to allow that in Jenkins (else it would reject the build) [07:31:05] right, so only some are given edit right in UI [07:31:08] the rebuild plugin does not know about those parameters cause they are not in the config [07:31:16] I see. ok [07:31:22] well, I'll just wait for your run then [07:31:32] the way to do is it to use the zuul client to reenqueue a change in a pipeline for a given change/project etc [07:31:36] zuul enqueue ... [07:31:51] but that acts on the whole change and triggers the whole set of jobs associated with that project/pipeline [07:33:07] I will see whether I can rebase :) [07:35:36] (03PS2) 10Hashar: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) [07:35:36] (03PS3) 10Hashar: zuul: Remove uncontrolled recursion in extra deps in MW jobs (take 2) [integration/config] - 10https://gerrit.wikimedia.org/r/1132543 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [07:38:04] hashar: given I did a lot of clean up after the automatic steps, it seems to rebase cleanly :) [07:38:13] indeed [07:38:17] to reduce the diff as much as possible in terms of whitespace and comments [07:52:41] Krinkle: I have started it, we will see :) [08:05:34] 10Gerrit (Gerrit 3.10): Upgrade to Gerrit 3.10.5 - https://phabricator.wikimedia.org/T390666 (10hashar) 03NEW [08:06:11] 10Gerrit (Gerrit 3.10), 07Upstream: Inverted search operators are not autocompleted - https://phabricator.wikimedia.org/T388672#10697425 (10hashar) [08:28:21] (03PS3) 10Hashar: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) [08:31:45] (03CR) 10Hashar: "I have made some adjustments to improve logging and I have started the job in a screen on `contint1002.wikimedia.org`. The log is written" [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [08:34:38] 10Continuous-Integration-Config, 10Math, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Allow control over which extra extensions are installed (Math REL1_43 jobs exceed 60min timeout) - https://phabricator.wikimedia.org/T389998#10697459 (10hashar) The script is running and once completed the log f... [08:40:21] (03CR) 10Hashar: [C:03+2] Use bluespice template for DrawioEditor & PageCheckout [integration/config] - 10https://gerrit.wikimedia.org/r/1132780 (owner: 10Hslater) [08:41:30] Krinkle: when adding bluespice template, that implicitly adds BlueSpiceFoundation as a dependency and that would need to be reflected in your change overhauling the dependencies. So I think we would need to rerun your script one last time before merging [08:41:31] (03Merged) 10jenkins-bot: Use bluespice template for DrawioEditor & PageCheckout [integration/config] - 10https://gerrit.wikimedia.org/r/1132780 (owner: 10Hslater) [08:41:52] the script is running [08:42:15] (03CR) 10Hashar: [C:03+2] Add VisualEditorPlus as a dependency to AIEditingAssistant [integration/config] - 10https://gerrit.wikimedia.org/r/1132776 (owner: 10Hslater) [08:43:55] (03Merged) 10jenkins-bot: Add VisualEditorPlus as a dependency to AIEditingAssistant [integration/config] - 10https://gerrit.wikimedia.org/r/1132776 (owner: 10Hslater) [08:46:28] (03CR) 10Hashar: [C:03+2] "deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/1132776 (owner: 10Hslater) [08:46:29] (03CR) 10Hashar: [C:03+2] "deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/1132780 (owner: 10Hslater) [08:48:59] (03CR) 10Hashar: [C:03+2] build: Updating eslint-config-wikimedia to 0.29.1 [integration/docroot] - 10https://gerrit.wikimedia.org/r/1132208 (owner: 10Libraryupgrader) [10:00:43] (03CR) 10Hashar: "I am fine moving it down to 20 minutes instead of the proposed 15 minutes, that will avoid unwanted timeout which I think would be mislead" [integration/config] - 10https://gerrit.wikimedia.org/r/1130325 (owner: 10Jforrester) [11:14:16] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T386218#10698015 (10Michael) [11:31:37] 11:30:36 PHP Fatal error: Uncaught Error: Call to undefined function mwsInitComponents() in /workspace/src/extensions/BlueSpiceFoundation/src/Foundation.php:28 [11:31:44] and I don't know where that function comes from :) [11:32:11] oh that is composer bummer [11:32:16] so yeah my script fails :) [11:35:09] Yippee, build fixed! [11:35:09] Project beta-code-update-eqiad build #541705: 09FIXED in 2 min 8 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/541705/ [11:51:03] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686 (10Atieno) 03NEW [11:54:44] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698184 (10Atieno) [11:57:29] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698208 (10Atieno) All tasks in the project's blocker column added here. https://phabricator.wikimedia.org/tag/mw-1.44-release/ [12:09:39] 10MediaWiki-Releasing, 05MW-1.44-release: Announce 1.44 "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T390692 (10Atieno) 03NEW [12:09:54] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698327 (10Atieno) [12:09:55] 10MediaWiki-Releasing, 05MW-1.44-release: Announce 1.44 "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T390692#10698326 (10Atieno) [12:12:57] 10Continuous-Integration-Config, 10MediaWiki-Releasing, 05MW-1.44-release: Configure CI to support REL1_43 - https://phabricator.wikimedia.org/T390695 (10Atieno) 03NEW [12:13:09] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698356 (10Atieno) [12:13:10] 10Continuous-Integration-Config, 10MediaWiki-Releasing, 05MW-1.44-release: Configure CI to support REL1_43 - https://phabricator.wikimedia.org/T390695#10698355 (10Atieno) [12:13:42] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T386218#10698360 (10Michael) [12:15:14] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Branch REL1_44 for MediaWiki and all extensions and skins - https://phabricator.wikimedia.org/T390696 (10Atieno) 03NEW [12:15:25] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698381 (10Atieno) [12:15:26] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Branch REL1_44 for MediaWiki and all extensions and skins - https://phabricator.wikimedia.org/T390696#10698380 (10Atieno) [12:15:42] 10Continuous-Integration-Config, 10MediaWiki-Releasing, 05MW-1.44-release: Configure CI to support REL1_44 - https://phabricator.wikimedia.org/T390695#10698382 (10Atieno) [12:17:01] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Release MW 1.43.0-rc.0 - https://phabricator.wikimedia.org/T390697 (10Atieno) 03NEW [12:17:12] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Release MW 1.43.0-rc.0 - https://phabricator.wikimedia.org/T390697#10698395 (10Atieno) [12:17:14] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698396 (10Atieno) [12:17:36] 10MediaWiki-Releasing, 05MW-1.44-release: Announce 1.44 "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T390692#10698398 (10Jdforrester-WMF) The nominal date for the branch would normally be at the same time as wmf.26, //i.e.// on 22 April (see https://www.mediawiki.or... [12:17:41] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Release MW 1.44.0-rc.0 - https://phabricator.wikimedia.org/T390697#10698403 (10Bugreporter) [12:20:30] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Write release announcement for 1.44.0-rc.0 - https://phabricator.wikimedia.org/T390699 (10Atieno) 03NEW [12:21:21] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Write release announcement for 1.44.0-rc.0 - https://phabricator.wikimedia.org/T390699#10698443 (10Atieno) [12:21:23] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698444 (10Atieno) [12:22:42] 10GitLab (Pipeline Services Migration🐀), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 3 others: move query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10698452 (10Jelto) >>! In T350793#10693429, @karapayneWMDE wrote: > Hey @Jelto , EM for wikidata here. Things a... [12:23:29] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Update CREDITS in MW master - https://phabricator.wikimedia.org/T390700 (10Atieno) 03NEW [12:23:31] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release: Update CREDITS in MW master - https://phabricator.wikimedia.org/T390700#10698464 (10Atieno) [12:23:34] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698465 (10Atieno) [12:28:20] (03PS4) 10Hashar: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) [12:31:19] Yippee, build fixed! [12:31:20] Project beta-update-databases-eqiad build #83680: 09FIXED in 11 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/83680/ [12:32:47] 10Gerrit (Gerrit 3.10): Upgrade to Gerrit 3.10.5 - https://phabricator.wikimedia.org/T390666#10698514 (10Paladox) This should fix https://phabricator.wikimedia.org/T355816 and I think this will fix https://phabricator.wikimedia.org/T355259 [12:33:41] 10MediaWiki-Releasing, 05MW-1.44-release: Announce 1.44 "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T390692#10698518 (10MSantos) @Jdforrester-WMF it was unintended side-effect of organizing the checklist after the PHP upgrade effort to keep both aligned in the futu... [12:34:32] (03PS3) 10Jforrester: jjb: [mwcore-phpunit-coverage-patch] lower timeout from 60 to 20 mins [integration/config] - 10https://gerrit.wikimedia.org/r/1130325 [12:34:38] (03PS4) 10Jforrester: jjb: [mwcore-phpunit-coverage-patch] lower timeout from 60 to 20 mins [integration/config] - 10https://gerrit.wikimedia.org/r/1130325 [12:35:39] (03CR) 10Jforrester: "> I am fine moving it down to 20 minutes instead of the proposed 15 minutes, that will avoid unwanted timeout which I think would be misle" [integration/config] - 10https://gerrit.wikimedia.org/r/1130325 (owner: 10Jforrester) [12:36:14] (03PS4) 10Hashar: jjb: [quibble*] lower timeout from 60 to 30 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/1130326 (https://phabricator.wikimedia.org/T389536) (owner: 10Jforrester) [12:36:39] 10MediaWiki-Releasing, 05MW-1.44-release: Announce 1.44 "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T390692#10698547 (10Jdforrester-WMF) Ack. [12:37:14] 10Continuous-Integration-Infrastructure, 10CampaignEvents, 10Campaigns-Product-Team (Campaign-Tools-Current-Sprint), 07ci-test-error: api-testing CI failure: Timeout of 5000ms exceeded in "before all" hook in "POST /campaignevents/v0/event_registration" - https://phabricator.wikimedia.org/T389863#10698555 (... [12:37:32] 10MediaWiki-Releasing, 05MW-1.44-release: Announce 1.44 "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T390692#10698561 (10Jdforrester-WMF) [12:37:35] (03CR) 10Hashar: "I have missed we can't run the same set of jobs on every single repo (`quibble-vendor-mysql-php81-noselenium` and `quibble-vendor-mysql-ph" [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [12:40:33] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698572 (10Aklapper) @Atieno: Most of the subtasks do not block the process of releasing MW 1.44.0 so I'm going to remove them. Subtasks should //technically// block r... [12:41:35] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10698577 (10Aklapper) [12:49:59] 10MediaWiki-Releasing, 05MW-1.44-release: Announce 1.44 "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T390692#10698682 (10MSantos) 05Openβ†’03Resolved [12:56:18] (03PS5) 10Hashar: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) [13:22:04] 10Gerrit, 06Release-Engineering-Team: Change Gerrit default submit strategy to 'Rebase if Necessary' and allowing content merge - https://phabricator.wikimedia.org/T390719 (10hashar) 03NEW [13:22:55] (03PS1) 10Hashar: Set submit strategy to rebase if necessary with contentmerge [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1133140 (https://phabricator.wikimedia.org/T390719) [14:50:12] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2025-04-01 - https://phabricator.wikimedia.org/T390737 (10brennen) 03NEW [14:50:41] (03open) 10brennen: update submodules for 2025-04-01 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/67 (https://phabricator.wikimedia.org/T390737) [15:05:54] (03merge) 10brennen: update submodules for 2025-04-01 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/67 (https://phabricator.wikimedia.org/T390737) [15:08:36] (03update) 10dancy: spiderpig: Render progress reports [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/730 [15:08:36] (03update) 10dancy: JobCard.vue: Use font-weight: bold for the job status [repos/releng/scap] (master-I0e61cabaaacb47aebb064a04e8cc30de7261d717) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/736 (https://phabricator.wikimedia.org/T383835) [15:08:36] (03open) 10dancy: JobCard.vue: Use font-weight: bold for the job status [repos/releng/scap] (master-I0e61cabaaacb47aebb064a04e8cc30de7261d717) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/736 (https://phabricator.wikimedia.org/T383835) [15:08:43] (03update) 10dancy: spiderpig: Render progress reports [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/730 [15:08:44] (03update) 10dancy: JobCard.vue: Use font-weight: bold for the job status [repos/releng/scap] (master-I0e61cabaaacb47aebb064a04e8cc30de7261d717) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/736 (https://phabricator.wikimedia.org/T383835) [15:10:20] (03update) 10dancy: spiderpig: Render progress reports [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/730 [15:11:41] (03update) 10dancy: JobCard.vue: Use font-weight: bold for the job status [repos/releng/scap] (master-I0e61cabaaacb47aebb064a04e8cc30de7261d717) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/736 (https://phabricator.wikimedia.org/T383835) [15:36:29] 10Phabricator (2025-04-01), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2025-04-01 - https://phabricator.wikimedia.org/T390737#10699843 (10Aklapper) 05Openβ†’03Resolved No other tasks tagged with this milestone, thus quick list of what got deployed: * Project Reports... [15:38:14] 10Phabricator (2025-04-01), 10Release-Engineering-Team (Doing 😎): Project Reports: Time span selector applies to a single option on that page - https://phabricator.wikimedia.org/T388997#10699850 (10Aklapper) [16:01:20] 10Phabricator (2025-03-25), 10Release-Engineering-Team (Doing 😎), 10Wikimedia-Phabricator-Extensions, 07Technical-Debt: Remove "Burnup Graph" project menu item and custom ProjectBurnupGraphProfileMenuItem code - https://phabricator.wikimedia.org/T388664#10699975 (10Aklapper) Of course 17 projects had c... [16:04:55] 10Phabricator (phabricator-next), 07Regression: Incorrect "Tasks completed this week" number on Project pages - https://phabricator.wikimedia.org/T390756 (10Aklapper) 03NEW p:05Triageβ†’03High [16:24:20] 10Phabricator (phabricator-next), 07Regression: Incorrect "Tasks completed this week" number on Project pages - https://phabricator.wikimedia.org/T390756#10700070 (10Aklapper) As I've changed nothing in the last months calculating this number in https://phabricator.wikimedia.org/source/phabricator/browse/wmf%2... [16:44:35] (03PS1) 10Reedy: layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate [integration/config] - 10https://gerrit.wikimedia.org/r/1133184 (https://phabricator.wikimedia.org/T390754) [16:48:22] (03CR) 10Subramanya Sastry: [C:03+1] layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate [integration/config] - 10https://gerrit.wikimedia.org/r/1133184 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [16:49:15] (03CR) 10Reedy: [C:03+2] layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate [integration/config] - 10https://gerrit.wikimedia.org/r/1133184 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [16:50:41] (03CR) 10Jforrester: "What do we think changed that means this is necessary?" [integration/config] - 10https://gerrit.wikimedia.org/r/1133184 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [16:51:08] (03Merged) 10jenkins-bot: layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate [integration/config] - 10https://gerrit.wikimedia.org/r/1133184 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [16:52:31] (03CR) 10Reedy: [C:03+2] "https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php83/ and https://integration.wikimedia.org/ci/job/quibble-composer-mysql" [integration/config] - 10https://gerrit.wikimedia.org/r/1133184 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [16:52:59] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1133184 T390754 [16:53:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:53:01] T390754: Parsoid CI failing with Error: Class "Pimple\Container" not found - https://phabricator.wikimedia.org/T390754 [17:07:00] it seems like a large percentage of CI jobs are failing because NPM is for some reason blocking all requests with the word "camel" in the path (so requests for the "decamelize" package are failing) [17:07:33] this one is definitely composer-y failures [17:11:38] I'm also seeing the CI failures [17:13:01] Dreamy_Jazz: which ones? :P [17:13:22] Well. The NPM one :D https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php74-selenium/82531/console [17:13:46] 18:09:12 npm error 403 403 Forbidden - GET https://registry.npmjs.org/decamelize [17:13:46] 18:09:12 npm error 403 In most cases, you or one of your dependencies are requesting [17:13:46] 18:09:12 npm error 403 a package version that is forbidden by your security policy, or [17:13:46] 18:09:12 npm error 403 on a server you do not have access to. [17:13:50] https://registry.npmjs.org/decamelize wfm... [17:13:58] some sort of rate limiting for CI? [17:14:25] as I said npm was just blocking all requests with 'camel' in the path. now fixed though, so those should go away after a recheck [17:15:26] Thanks. I'll try a recheck. [17:15:40] Wonder why this was happening? [17:15:47] Seems a random word to block based on [17:15:56] i'm curious too [17:16:29] https://status.npmjs.org/incidents/hdtkrsqp134s [17:16:57] Could always be some engineer trying some kind of April fools joke [17:17:06] But it backfiring as it went to prod :D [17:17:26] > We are currently investigating reports of intermittent failures when viewing and installing packages scoped to certain keywords. [17:17:29] heh, that seems likely [17:17:44] I wonder if camel is being caught by the scunthorpe problem... [17:18:02] Oh that's a good point. [17:18:24] i tested and 'camel' was the exact word getting blocked, which resulted in the 'decamelize' package failing to install [17:33:00] CI seems a bit backed up [17:35:14] Just a tiny bit. And the first patch in line is failing selenium, too, so brace for the queue invalidation. Joy! [17:35:59] Oh it actually passed on the retry. Maybe I jinxed it :P [17:46:43] 10Phabricator (Upstream), 07Upstream: Exception: "Call to phutil_nonempty_string() expected null or a string, got: int" in PhabricatorRepository.php (due to Importing) - https://phabricator.wikimedia.org/T347483#10700461 (10Aklapper) The culprit is `$map[$branch] = $branch_head;` in https://we.phorge.it/source... [18:00:28] 10Continuous-Integration-Config, 10Math, 10MediaWiki-Platform-Team (Radar), 10MW-1.44-notes (1.44.0-wmf.24; 2025-04-08), 13Patch-For-Review: Allow control over which extra extensions are installed (Math REL1_43 jobs exceed 60min timeout) - https://phabricator.wikimedia.org/T389998#10700497 (10hashar) The... [18:02:11] 10Diffusion, 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Unify gitlab migration scripts - https://phabricator.wikimedia.org/T346191#10700509 (10thcipriani) a:05jeenaβ†’03None [18:02:27] 10Diffusion, 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Unify repository archive scripts - https://phabricator.wikimedia.org/T346191#10700512 (10thcipriani) [18:02:46] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Move phabricator/deployments repository from Diffusion to GitLab/Gerrit - https://phabricator.wikimedia.org/T321380#10700515 (10thcipriani) [18:02:47] 10Diffusion, 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Unify repository archive scripts - https://phabricator.wikimedia.org/T346191#10700514 (10thcipriani) [18:28:15] 06Release-Engineering-Team, 06Security-Team, 07Security: Split PrivateSettings into config and business logic - https://phabricator.wikimedia.org/T390773 (10Tgr) 03NEW [18:32:19] (03PS1) 10Reedy: zuul: Remove IPReputation from ext_deps on REL1_39 [integration/config] - 10https://gerrit.wikimedia.org/r/1133209 (https://phabricator.wikimedia.org/T390772) [18:33:39] (03CR) 10CI reject: [V:04-1] zuul: Remove IPReputation from ext_deps on REL1_39 [integration/config] - 10https://gerrit.wikimedia.org/r/1133209 (https://phabricator.wikimedia.org/T390772) (owner: 10Reedy) [18:33:49] Yippee, build fixed! [18:33:49] Project mediawiki-core-doxygen build #9364: 09FIXED in 15 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/9364/ [18:36:00] (03PS2) 10Reedy: zuul: Remove IPReputation from ext_deps on REL1_39 [integration/config] - 10https://gerrit.wikimedia.org/r/1133209 (https://phabricator.wikimedia.org/T390772) [18:36:05] hackhackhack [18:37:49] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Release Pipeline (Blubber): Cache buildkitd layers locally relative to the runner in blubber's pipeline - https://phabricator.wikimedia.org/T320746#10700701 (10thcipriani) 05Openβ†’03Declined Declining since the task descri... [18:37:51] (03CR) 10Reedy: [C:03+2] zuul: Remove IPReputation from ext_deps on REL1_39 [integration/config] - 10https://gerrit.wikimedia.org/r/1133209 (https://phabricator.wikimedia.org/T390772) (owner: 10Reedy) [18:39:08] (03Merged) 10jenkins-bot: zuul: Remove IPReputation from ext_deps on REL1_39 [integration/config] - 10https://gerrit.wikimedia.org/r/1133209 (https://phabricator.wikimedia.org/T390772) (owner: 10Reedy) [18:40:57] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1133209 T390772 [18:41:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:41:00] T390772: CI broken on MW 1.39 - https://phabricator.wikimedia.org/T390772 [18:43:25] 10Scap: scap has confusing warning message when deploying a patch with a cross-repo dependency - https://phabricator.wikimedia.org/T387798#10700723 (10thcipriani) [18:44:38] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Scap: scap backport non_config_is_in_production() check doesn't work right for VisualEditor/VisualEditor repo - https://phabricator.wikimedia.org/T353720#10700730 (10thcipriani) 05Openβ†’03Resolved Change merged for this one, haven't heard about it happe... [18:45:09] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Scap: Introduce new workflow for relevant dependencies calculation - https://phabricator.wikimedia.org/T371611#10700734 (10thcipriani) a:05jeenaβ†’03None [18:45:17] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Scap: Backport: implement new criteria for relevant dependencies - https://phabricator.wikimedia.org/T365146#10700737 (10thcipriani) a:05jeenaβ†’03None [18:51:05] (03PS1) 10Reedy: Revert "zuul: Update WikimediaEvents dependencies" [integration/config] - 10https://gerrit.wikimedia.org/r/1133214 (https://phabricator.wikimedia.org/T390754) [18:51:09] (03PS2) 10Reedy: Revert "zuul: Update WikimediaEvents dependencies" [integration/config] - 10https://gerrit.wikimedia.org/r/1133214 (https://phabricator.wikimedia.org/T390754) [19:18:30] (03CR) 10Jforrester: "We can't merge this without breaking CI in the other direction, unless https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEven" [integration/config] - 10https://gerrit.wikimedia.org/r/1133214 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [19:27:53] (03PS1) 10Reedy: parameter_functions.py: Don't load WebAuthn on any non master branch [integration/config] - 10https://gerrit.wikimedia.org/r/1133221 [19:28:36] Krinkle: I got the builds done in ~ 3 hours and a half which is not too bad. I am not sure where to paste the results :b [19:28:58] (03PS2) 10Reedy: parameter_functions.py: Don't load WebAuthn on any non master branch [integration/config] - 10https://gerrit.wikimedia.org/r/1133221 (https://phabricator.wikimedia.org/T390754) [19:29:24] (03CR) 10Reedy: [C:03+2] parameter_functions.py: Don't load WebAuthn on any non master branch [integration/config] - 10https://gerrit.wikimedia.org/r/1133221 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [19:30:43] (03Merged) 10jenkins-bot: parameter_functions.py: Don't load WebAuthn on any non master branch [integration/config] - 10https://gerrit.wikimedia.org/r/1133221 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [19:31:19] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1133221 T390754 [19:31:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:31:21] T390754: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754 [19:33:58] https://gist.github.com/hashar/c6b06159056cb6d0ea5cc43fafdaae5b ! [19:35:39] 12:33:14 1) SpecialPageFatalTest::testSpecialPageDoesNotFatal with data set "UserBoard" ('UserBoard') [19:35:39] 12:33:14 Error: Class 'Title' not found [19:35:41] poor Title [19:41:28] (03Abandoned) 10Reedy: Revert "zuul: Update WikimediaEvents dependencies" [integration/config] - 10https://gerrit.wikimedia.org/r/1133214 (https://phabricator.wikimedia.org/T390754) (owner: 10Reedy) [19:54:01] (03PS1) 10Reedy: Revert "layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate" [integration/config] - 10https://gerrit.wikimedia.org/r/1133231 [19:54:08] (03PS2) 10Reedy: Revert "layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate" [integration/config] - 10https://gerrit.wikimedia.org/r/1133231 [20:07:13] (03CR) 10Reedy: [C:03+2] Revert "layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate" [integration/config] - 10https://gerrit.wikimedia.org/r/1133231 (owner: 10Reedy) [20:08:59] (03Merged) 10jenkins-bot: Revert "layout.yaml: Remove quibble-composer-mysql-php8[32] from parsoid gate" [integration/config] - 10https://gerrit.wikimedia.org/r/1133231 (owner: 10Reedy) [20:09:29] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1133231 [20:09:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:18:03] should i be worried when several jobs in gate-and-submit-wmf have been queued for 13 minutes at this point? [20:19:40] dunno why it's just the phan jobs... [20:20:10] the wikilove patch has quibble-vendor-mysql-php74-noselenium in queue as well [20:23:09] zuul error being logged: https://phabricator.wikimedia.org/P74551 [20:23:12] Reedy: https://phabricator.wikimedia.org/P74550 you've broken it I think [20:23:23] synchronized! [20:24:10] bah [20:24:25] (03PS1) 10Majavah: zuul: Fix Phan jobs on repos without WebAuthn [integration/config] - 10https://gerrit.wikimedia.org/r/1133238 [20:24:29] ^ should fix it [20:24:49] (03CR) 10Reedy: [C:03+2] zuul: Fix Phan jobs on repos without WebAuthn [integration/config] - 10https://gerrit.wikimedia.org/r/1133238 (owner: 10Majavah) [20:24:50] thanks [20:25:46] dancy: my paste number was lower! [20:25:58] lol [20:26:03] (03Merged) 10jenkins-bot: zuul: Fix Phan jobs on repos without WebAuthn [integration/config] - 10https://gerrit.wikimedia.org/r/1133238 (owner: 10Majavah) [20:26:28] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1133238 [20:26:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:26:57] ok, now the jobs are running at least [20:26:58] eugh [20:27:39] Not that webauthn actually has many tests... [20:27:48] I suspect we shouldn't be removing it on wmf/ branches either :) [20:28:53] (03PS1) 10Reedy: zuul: Swap the branch check to specific release branches for WebAuthn removal [integration/config] - 10https://gerrit.wikimedia.org/r/1133240 [20:28:54] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Migrate Quibble images from bullseye to bookworm - https://phabricator.wikimedia.org/T362705#10701235 (10Jdforrester-WMF) [20:28:55] 10Continuous-Integration-Infrastructure, 10MediaWiki-libs-Rdbms: Raise Quibble jobs' tested version of MariaDB to 10.6 - https://phabricator.wikimedia.org/T366646#10701234 (10Jdforrester-WMF) [20:28:55] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate Quibble images from buster to bullseye - https://phabricator.wikimedia.org/T335766#10701236 (10Jdforrester-WMF) [20:29:12] (03PS1) 10Jforrester: Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package [integration/config] - 10https://gerrit.wikimedia.org/r/1133241 (https://phabricator.wikimedia.org/T366646) [20:30:01] (03CR) 10CI reject: [V:04-1] zuul: Swap the branch check to specific release branches for WebAuthn removal [integration/config] - 10https://gerrit.wikimedia.org/r/1133240 (owner: 10Reedy) [20:30:33] (03PS2) 10Jforrester: zuul: Swap the branch check to specific release branches for WebAuthn removal [integration/config] - 10https://gerrit.wikimedia.org/r/1133240 (owner: 10Reedy) [20:30:36] (03CR) 10Jforrester: [C:03+2] zuul: Swap the branch check to specific release branches for WebAuthn removal [integration/config] - 10https://gerrit.wikimedia.org/r/1133240 (owner: 10Reedy) [20:32:07] (03Merged) 10jenkins-bot: zuul: Swap the branch check to specific release branches for WebAuthn removal [integration/config] - 10https://gerrit.wikimedia.org/r/1133240 (owner: 10Reedy) [20:32:18] I worked so hard on the message wrapping for https://gerrit.wikimedia.org/r/c/integration/config/+/1133241 and gerrit just ruins it, sob. [20:32:38] (03CR) 10Jforrester: [C:03+2] Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package [integration/config] - 10https://gerrit.wikimedia.org/r/1133241 (https://phabricator.wikimedia.org/T366646) (owner: 10Jforrester) [20:34:01] (03Merged) 10jenkins-bot: Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package [integration/config] - 10https://gerrit.wikimedia.org/r/1133241 (https://phabricator.wikimedia.org/T366646) (owner: 10Jforrester) [20:34:17] !log Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package, for T366646 [20:34:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:34:19] T366646: Raise Quibble jobs' tested version of MariaDB to 10.6 - https://phabricator.wikimedia.org/T366646 [20:34:28] Oh no. [20:34:57] My ssh config isn't working for contint? [20:40:40] contint dns cached? [20:41:17] No, it matches the known hosts. [20:41:24] Did my access get taken away? [20:41:43] based on the auth log, you seem to be trying to log in as `jamesforrester` and not `jforrester`? [20:42:29] I get `debug1: Authenticating to bast1003.wikimedia.org:22 as 'jforrester'` in ssh when I pass -v [20:43:01] I think this is just outcome of my OpenSsh upgrade from Apple overnight. [20:43:01] yeah you can log in to a bastion just fine, but it's using the wrong credentials to log in to contint1002 [20:43:26] I'll slap an extra `User jforrester` on my proxyjump [20:44:09] Yup, that fixed it. [20:44:16] Probably worth some wider engineering-all type heads up if it was macOS 15.4... [20:44:27] I guess I should edit https://wikitech.wikimedia.org/wiki/SRE/Production_access#Setting_up_your_SSH_config to match? [20:45:16] that snippet already has a username set in 'Match host ="*.wikimedia.org,*.wmnet"', wonder why that doesn't work [20:45:21] Yeah, probably. Anyone else with 15.4 confirm I'm not just a fool? [20:45:29] taavi: I was surprised that second one was needed. [20:46:12] !log Zuul: Swap the branch check to specific release branches, for T390754 etc. [20:46:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:46:15] T390754: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754 [20:46:54] also i wonder why that uses 'Match' and not just 'Host' [20:47:27] I imagine no-one write our current specimen ssh config from scratch, but it built over years and people just add/adjust as needed. [20:48:12] that config could be made a lot simpler if it wouldn't make a special 'bast' alias for a bastion [20:48:55] Mine is even worse, as I have four different entries there for bast, three commented out at a time, so I can be fast wherever I am. [20:49:14] 1003 for home, 3007 for Europe, 4005 for SF, 5004 for Asia. [20:49:25] Maybe I should have all of them? Eh. [20:52:58] one day i'll get to publishing the idea in my head on how we could at least introduce site-specific CNAMEs for the current bastion to individual hosts in configs relatively easily [21:14:59] taavi: I asked in ops-l a few years ago and that got declined [21:15:07] namely due to the ssh host key varying [21:16:16] I end up with config such as: [21:16:16] Host deployment.wmnet [21:16:16] Hostname deployment.eqiad.wmnet [21:16:16] HostKeyAlias deploy1003.eqiad.wmnet [21:16:47] Host *.codfw.wmnet [21:16:47] ProxyJump bast2003.wikimedia.org [21:17:12] and a default to bast1003 [21:17:35] so whenever the hosts change (which happens every two years or so), I update the config [21:17:59] or every six months for the deployment.wmnet entry (due to the switchover) [21:18:14] but yes, DNS entries would be nice to have [21:19:02] * hashar sleeps [21:28:43] taavi: some history from 2019. see the comments back then. https://gerrit.wikimedia.org/r/c/operations/dns/+/489103 [21:29:18] because that sounded just like https://gerrit.wikimedia.org/r/c/operations/dns/+/489103/2/templates/wikimedia.org [21:30:37] 10Continuous-Integration-Infrastructure: Find and document how to debug a NodeJS process on CI/Docker - https://phabricator.wikimedia.org/T390125#10701486 (10hashar) [21:30:40] 10Continuous-Integration-Infrastructure, 10Testing Support, 10ci-test-error (WMF-deployed Build Failure), 10MW-1.44-notes (1.44.0-wmf.23; 2025-04-01), and 2 others: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#10701485 (10hashar) [21:32:30] 10Continuous-Integration-Infrastructure, 10Testing Support, 10ci-test-error (WMF-deployed Build Failure), 10MW-1.44-notes (1.44.0-wmf.23; 2025-04-01), and 2 others: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#10701489 (10hashar)... [22:35:44] 10MediaWiki-Releasing, 10MediaWiki-Vendor, 05MW-1.44-release: Prune /vendor for REL1_44 - https://phabricator.wikimedia.org/T390792 (10ABreault-WMF) 03NEW [22:40:41] 10MediaWiki-Releasing, 05MW-1.44-release: Add REL1_44 to on-wiki documentation as the development snapshot - https://phabricator.wikimedia.org/T390793 (10ABreault-WMF) 03NEW [22:41:23] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10701671 (10ABreault-WMF) [22:41:24] 10MediaWiki-Releasing, 05MW-1.44-release: Add REL1_44 to on-wiki documentation as the development snapshot - https://phabricator.wikimedia.org/T390793#10701670 (10ABreault-WMF) [22:43:08] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.44-release: Add REL1_43 to ExtensionDistributor as the development snapshot - https://phabricator.wikimedia.org/T390794 (10ABreault-WMF) 03NEW [22:43:43] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.44-release: Add REL1_43 to ExtensionDistributor as the development snapshot - https://phabricator.wikimedia.org/T390794#10701688 (10ABreault-WMF) [22:43:46] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10701689 (10ABreault-WMF) [22:44:28] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.44-release: Add REL1_43 to ExtensionDistributor as the development snapshot - https://phabricator.wikimedia.org/T390794#10701691 (10ABreault-WMF) [22:44:51] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.44-release: Add REL1_44 to ExtensionDistributor as the development snapshot - https://phabricator.wikimedia.org/T390794#10701693 (10ABreault-WMF) [22:47:04] 10MediaWiki-Releasing, 05MW-1.44-release: Release MW 1.44.0-rc.1 (if needed) - https://phabricator.wikimedia.org/T390795 (10ABreault-WMF) 03NEW [22:47:50] 10MediaWiki-Releasing, 05MW-1.44-release: Release MW 1.44.0-rc.1 (if needed) - https://phabricator.wikimedia.org/T390795#10701705 (10ABreault-WMF) [22:47:50] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10701706 (10ABreault-WMF) [22:51:16] 10MediaWiki-Releasing, 07Documentation, 05MW-1.44-notes, 05MW-1.44-release: Finish up RELEASE-NOTES for 1.44.0 - https://phabricator.wikimedia.org/T390796 (10ABreault-WMF) 03NEW [22:52:02] 10MediaWiki-Releasing, 07Documentation, 05MW-1.44-notes, 05MW-1.44-release: Finish up RELEASE-NOTES for 1.44.0 - https://phabricator.wikimedia.org/T390796#10701720 (10ABreault-WMF) [22:52:03] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10701721 (10ABreault-WMF) [22:54:11] 10MediaWiki-Releasing, 05MW-1.44-release: Write and send release announcement for 1.44.0 - https://phabricator.wikimedia.org/T390797 (10ABreault-WMF) 03NEW [22:54:22] 10MediaWiki-Releasing, 05MW-1.44-release: Write and send release announcement for 1.44.0 - https://phabricator.wikimedia.org/T390797#10701735 (10ABreault-WMF) [22:54:23] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10701736 (10ABreault-WMF) [22:55:35] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.44-release: Mark REL1_44 in ExtensionDistributor as a stable release - https://phabricator.wikimedia.org/T390798 (10ABreault-WMF) 03NEW [22:56:09] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.44-release: Mark REL1_44 in ExtensionDistributor as a stable release - https://phabricator.wikimedia.org/T390798#10701763 (10ABreault-WMF) [23:10:47] 10MediaWiki-Releasing, 05MW-1.44-notes, 05MW-1.44-release, 05Release: Release MediaWiki 1.44.0 - https://phabricator.wikimedia.org/T390686#10701797 (10ABreault-WMF) [23:19:24] dancy: hej hej, did the train succeed or is it blocked? Asking because https://versions.toolforge.org shows still .22 for group0 [23:21:48] I don't see any reverts.. [23:22:38] https://github.com/wikimedia/operations-mediawiki-config/blob/master/wikiversions.json says 143 and https://github.com/wikimedia/operations-mediawiki-config/blob/master/dblists/group0.dblist 143... [23:24:23] andre: pihwiki was closed... [23:24:39] Right... [23:25:06] it was in group2, it's now in group0 [23:25:06] https://github.com/wikimedia/operations-mediawiki-config/commit/69ba4384154a0f0a02a4e49c835162c127eaf9a5 [23:25:18] but it wasn't switched version... [23:25:22] So hence the numbers aren't right [23:25:51] It's just that I expect https://versions.toolforge.org to display "1.44.0-wmf.23" for group0 at this stage, not "1.44.0-wmf.23 / 1.44.0-wmf.22" [23:25:55] I know [23:25:57] ah yeah [23:26:11] makes sense [23:26:23] someone does probably need to sync up pihwiki specifically [23:26:50] annoying, but shouldn't be a big deal [23:43:57] andre: fixeded [23:44:06] Thanks!