[01:28:56] 10Phabricator, 10corto: Increase trusted volunteer's visibility into production incidents - https://phabricator.wikimedia.org/T426137 (10Soda) 03NEW [02:11:27] 10MediaWiki-Releasing, 07Documentation, 05MW-1.46-release: Finish up RELEASE-NOTES for 1.46.0 - https://phabricator.wikimedia.org/T423270#11915550 (10Bugreporter) https://phabricator.wikimedia.org/source/mediawiki/browse/REL1_46/RELEASE-NOTES-1.46#L18 is outdated too since the lowest version that upgrade is... [03:08:56] 10Gerrit: Can't use 2nd ssh - https://phabricator.wikimedia.org/T426139 (10Mroik) 03NEW [06:30:17] 06Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T423911#11915708 (10Aklapper) Removing T425988 again as a train blocker per T425988#11915352, Logstash looks calm since about 23:45 UTC. Th... [06:30:23] 06Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T423911#11915709 (10Aklapper) [08:22:37] 10Gerrit: Can't use 2nd ssh - https://phabricator.wikimedia.org/T426139#11916036 (10Aklapper) I assume this is about https://gerrit.wikimedia.org/r/settings/#SSHKeys ? What's the output of `ssh -vvv mroik@gerrit.wikimedia.org -p29418` ? [08:52:36] (03PS2) 10Hslater: Zuul: [mediawiki/extensions/ChatBot] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/1284725 [08:52:38] (03PS2) 10Hslater: Zuul: [mediawiki/extensions/ChatIntegration] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1284783 [08:58:19] Hi all! I am trying to set up a build pipeline on gitlab that publishes a docker image when a tag is pushed. But it's failing: https://gitlab.wikimedia.org/repos/mediawiki/services/smokepy/-/pipelines [08:59:46] I already have this working on the redioscope repo, but on the new smokepy repo, I get "There has been a timeout failure or the job got stuck. Check your timeout limits or try again" [09:00:45] I vaguely recall something about needing to configure a "worker" for the redioscope repo... Can anyone here point me in the right direction? What am I missing? [09:00:45] [09:10:30] 06Release-Engineering-Team, 06ServiceOps new, 05FY2025-26 KR 5.1, 06MediaWiki-Platform-Team (Kanban Board), and 3 others: api-gateway: run make test in CI - https://phabricator.wikimedia.org/T424824#11916234 (10daniel) >>! In T424824#11915027, @SLong-WMF wrote: > Hey Daniel! It's a little unclear to me wha... [09:41:42] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 06collaboration-services: Self-build and publish buildkit helper images - https://phabricator.wikimedia.org/T321316#11916376 (10Jelto) Thank you very much for the detailed explanation! > I believe that any legacy docker build based build (so, docker b... [09:47:03] 06Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 10Scap, 10MW-on-K8s, 06ServiceOps new: Add helm rollback functionality to scap - https://phabricator.wikimedia.org/T371069#11916398 (10MLechvien-WMF) @RLazarus given Scott's comment and yours, it sounds like this should move to Backlog (though the discussion... [09:47:31] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 19 - https://phabricator.wikimedia.org/T426164 (10Jelto) 03NEW [11:07:51] (03update) 10hoo: Default XDEBUG_CONFIG to client_host=xdebug://gateway [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/644 [12:01:16] 06Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T423911#11917044 (10Dreamy_Jazz) [12:32:40] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/WikiLambda] Drop AF and SB deps down to phan-only [integration/config] - 10https://gerrit.wikimedia.org/r/1286889 (https://phabricator.wikimedia.org/T423180) [12:35:12] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/WikiLambda] Drop AF and SB deps down to phan-only [integration/config] - 10https://gerrit.wikimedia.org/r/1286889 (https://phabricator.wikimedia.org/T423180) (owner: 10Jforrester) [12:37:03] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikiLambda] Drop AF and SB deps down to phan-only [integration/config] - 10https://gerrit.wikimedia.org/r/1286889 (https://phabricator.wikimedia.org/T423180) (owner: 10Jforrester) [12:37:36] !log Zuul: [mediawiki/extensions/WikiLambda] Drop AF and SB deps down to phan-only, for T423180 [12:37:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:37:39] T423180: Allow Wikifunctions-generated references to have links to non-Wikimedia sites (but not in general Wikifunctions output) - https://phabricator.wikimedia.org/T423180 [12:37:59] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/ChatIntegration] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1284783 (owner: 10Hslater) [12:38:07] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/ChatBot] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/1284725 (owner: 10Hslater) [12:38:14] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/Springboard] Add AdminLinks Phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1286499 (owner: 10Fomafix) [12:40:52] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/ChatIntegration] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1284783 (owner: 10Hslater) [12:40:53] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/ChatBot] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/1284725 (owner: 10Hslater) [12:41:02] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Springboard] Add AdminLinks Phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1286499 (owner: 10Fomafix) [12:42:08] ! Zuul: [mediawiki/extensions/ChatIntegration] Add dependency on VisualEditor [12:42:11] Bah. [12:42:14] !log Zuul: [mediawiki/extensions/ChatIntegration] Add dependency on VisualEditor [12:42:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:42:30] !log Zuul: [mediawiki/extensions/ChatBot] Add dependencies on VisualEditor and BlueSpiceFoundation [12:42:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:42:43] !log Zuul: [mediawiki/extensions/Springboard] Add AdminLinks Phan dependency [12:42:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:44:23] 10Beta-Cluster-Infrastructure, 10Pywikibot, 07Pywikibot-tests: Find a way for pywikibot GitHub Actions to avoid IP range blocks of Microsoft Azure hosted runners - https://phabricator.wikimedia.org/T399485#11917526 (10Xqt) @bd808: Thank you fro the SOCKS proposal (see also https://www.mediawiki.org/wiki/Manu... [13:51:01] 06Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 10Wiki-Setup (Delete / Redirect): Close TestCommons from production - https://phabricator.wikimedia.org/T213295#11917582 (10UndueMarmot) Any Commons-imported templates on testwiki fail because the JsonConfig extension there still insists on pulling data from... [13:52:42] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 19 - https://phabricator.wikimedia.org/T426164#11917590 (10Jdforrester-WMF) [14:05:09] 06Release-Engineering-Team, 10ChangeProp, 06Data-Engineering, 10EventStreams, and 15 others: Migrate node-based services in production to node22 - https://phabricator.wikimedia.org/T393434#11917643 (10Jdforrester-WMF) [14:05:15] 06Release-Engineering-Team, 10Math, 10Mathoid, 10Texvcjs: update mathoid to node22 - https://phabricator.wikimedia.org/T408665#11917645 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF [14:10:46] 10Gerrit: Can't use 2nd ssh - https://phabricator.wikimedia.org/T426139#11917661 (10Mroik) ` debug1: OpenSSH_10.3p1, OpenSSL 3.6.2 7 Apr 2026 debug3: Running on Linux 7.0.3-arch1-2 #1 SMP PREEMPT_DYNAMIC Fri, 01 May 2026 15:49:22 +0000 x86_64 debug3: Started with: ssh -vvv mroik@gerrit.wikimedia.org -p29418 -i .... [14:16:09] 06Release-Engineering-Team, 06cloud-services-team, 10Toolforge: [toolforge_deploy_mr] Branches can only be tested for 24 hours, otherwise fail with CI too old. - https://phabricator.wikimedia.org/T408999#11917699 (10aputhin) p:05Triage→03Low [14:22:23] (03Abandoned) 10Jforrester: build: Upgrade eslint-config-wikimedia to 0.31.0 and make pass [performance/fresnel] - 10https://gerrit.wikimedia.org/r/1176753 (owner: 10Jforrester) [14:46:22] FIRING: PuppetAgentNoResources: No Puppet resources found on instance deployment-mx04 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [14:53:47] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work: Buildkit v0.30.0 released - https://phabricator.wikimedia.org/T426212 (10dancy) 03NEW [15:26:36] (03open) 10trueg: wdqs-backend-init: Docker image which prime the Qlever index for the WDQS deployment. [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/169 (https://phabricator.wikimedia.org/T425702) [16:46:07] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 10VPS-project-devtools, and 2 others: devtools: Create new Phab/Phorge test instance on: Debian Trixie, Debian Bookworm - https://phabricator.wikimedia.org/T424055#11918551 (10Arnoldokoth) There is an issue running `scap` from the deploy... [17:21:10] (03open) 10dancy: README.md: Remove completed todos [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/170 [17:21:14] (03update) 10dancy: README.md: Remove completed todos [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/170 [17:21:53] (03merge) 10dancy: README.md: Remove completed todos [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/170 [17:22:01] (03open) 10dancy: README.md: Add Processing access requests section [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/171 [17:22:05] (03update) 10dancy: README.md: Add Processing access requests section [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/171 [17:25:30] (03update) 10dancy: README.md: Add Processing access requests section [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/171 [17:26:08] (03update) 10dancy: README.md: Add Processing access requests section [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/171 [17:33:02] (03update) 10dancy: wdqs-backend-init: Docker image which prime the Qlever index for the WDQS deployment. [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/169 (https://phabricator.wikimedia.org/T425702) (owner: 10trueg) [17:33:34] (03merge) 10dancy: wdqs-backend-init: Docker image which prime the Qlever index for the WDQS deployment. [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/169 (https://phabricator.wikimedia.org/T425702) (owner: 10trueg) [18:14:28] (03open) 10dancy: web/src/state.js: Update a comment [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1184 [18:14:32] (03update) 10dancy: web/src/state.js: Update a comment [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1184 [18:17:01] (03merge) 10dancy: web/src/state.js: Update a comment [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1184 [18:30:55] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 10VPS-project-devtools, and 2 others: devtools: Create new Phab/Phorge test instance on: Debian Trixie, Debian Bookworm - https://phabricator.wikimedia.org/T424055#11919020 (10Arnoldokoth) P92512 I've copied the error output here. [18:56:50] hashar: I'd like to merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1282006/9, let me know when you're available to supervise [19:07:35] andrewbogott: hash.ar is on vacation this week [19:07:46] ok! it'll keep [19:08:07] okie doke, thanks! [19:12:34] (03open) 10dancy: config: Always close cfg_file in load() [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1185 [19:12:39] (03update) 10dancy: config: Always close cfg_file in load() [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1185 [19:15:11] (03merge) 10dancy: config: Always close cfg_file in load() [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1185 [19:18:01] (03open) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 [19:18:05] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 [19:19:01] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:19:11] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:21:36] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:23:45] (03CR) 10Jeena Huneidi: [C:03+2] jjb: [catalyst-daily-VisualEditor] Add in Zuul parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1285895 (https://phabricator.wikimedia.org/T424411) (owner: 10Vaughn Walters) [19:25:25] (03Merged) 10jenkins-bot: jjb: [catalyst-daily-VisualEditor] Add in Zuul parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1285895 (https://phabricator.wikimedia.org/T424411) (owner: 10Vaughn Walters) [19:25:25] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:25:30] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:26:22] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:26:29] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:32:22] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [19:49:47] 06Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 07Epic, 06Quality-and-Test-Engineering-Team (Test Infrastructure): Pretrain (nÊe Group -1) QTE validation environment - https://phabricator.wikimedia.org/T369112#11919368 (10bd808) [19:49:48] 06Release-Engineering-Team (Doing 😎): How MediaWiki code gets deployed and things we are changing - https://phabricator.wikimedia.org/T424780#11919369 (10bd808) [19:50:59] 06Release-Engineering-Team (Doing 😎): How MediaWiki code gets deployed and things we are changing - https://phabricator.wikimedia.org/T424780#11919370 (10bd808) [19:53:25] 06Release-Engineering-Team (Doing 😎): How MediaWiki code gets deployed and things we are changing - https://phabricator.wikimedia.org/T424780#11919383 (10bd808) I lost a week to a post-Hackathon COVID-19 infection. As a result I have not yet made time to clean up the slide deck and publish it on meta. I will try... [20:33:46] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily-VisualEditor] Add daily run [integration/config] - 10https://gerrit.wikimedia.org/r/1287020 (https://phabricator.wikimedia.org/T424441) [20:42:57] (03PS2) 10Vaughn Walters: jjb: [catalyst-daily-VisualEditor] Add daily run and ZUUL_PIPELINE [integration/config] - 10https://gerrit.wikimedia.org/r/1287020 (https://phabricator.wikimedia.org/T424441) [20:48:09] (03CR) 10Vaughn Walters: "This is ready for review now. The is building and tests passing, so adding in the cron. Castor-save was failing downstream on "ZUUL_PIPE" [integration/config] - 10https://gerrit.wikimedia.org/r/1287020 (https://phabricator.wikimedia.org/T424441) (owner: 10Vaughn Walters) [20:51:45] (03CR) 10Vaughn Walters: "*This is building" [integration/config] - 10https://gerrit.wikimedia.org/r/1287020 (https://phabricator.wikimedia.org/T424441) (owner: 10Vaughn Walters) [21:17:28] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [21:17:33] (03update) 10dancy: scap prep: Add foss_violations support [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1186 (https://phabricator.wikimedia.org/T403829) [22:07:25] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 06collaboration-services: Self-build and publish buildkit helper images - https://phabricator.wikimedia.org/T321316#11920063 (10dduvall) >>! In T321316#11916376, @Jelto wrote: > Thank you very much for the detailed explanation! You're welcome! However... [22:14:20] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 06collaboration-services: Self-build and publish buildkit helper images - https://phabricator.wikimedia.org/T321316#11920068 (10dduvall) Oh boy, there are many instances of old Blubber usage: [ ] Striker (1 files) ** [[https://gerrit.wikimedia.org/g/l... [22:33:30] Project mediawiki-core-doxygen build #20499: 04FAILURE in 15 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/20499/ [22:44:22] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 13Patch-For-Review: puppetize setup of new zuul VMs - https://phabricator.wikimedia.org/T395938#11920116 (10Dzahn) 05Resolved→03Open [23:33:01] Yippee, build fixed! [23:33:01] Project mediawiki-core-doxygen build #20500: 09FIXED in 14 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/20500/