[06:01:30] (03PS1) 10Kevin Bazira: inference-services: Add CI pipeline jobs for tts model-server [integration/config] - 10https://gerrit.wikimedia.org/r/1307680 (https://phabricator.wikimedia.org/T430536) [06:01:57] (03open) 10santhosh: performance: improve cli startup time [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/652 (https://phabricator.wikimedia.org/T431234 https://phabricator.wikimedia.org/T431235) [06:03:05] (03update) 10santhosh: performance: improve cli startup time [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/652 (https://phabricator.wikimedia.org/T431234 https://phabricator.wikimedia.org/T431235) [06:04:47] (03update) 10santhosh: performance: improve cli startup time [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/652 (https://phabricator.wikimedia.org/T431234 https://phabricator.wikimedia.org/T431235) [07:10:18] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 13Patch-For-Review: zuul-client encrypt fails to retrieve public key - https://phabricator.wikimedia.org/T431003#12088758 (10hashar) Another thing I found on Thursday evening is our CDN rejects `zuul-client` requests to the Zuu... [07:39:20] (03CR) 10Bartosz Wójtowicz: [C:03+1] inference-services: Add CI pipeline jobs for tts model-server [integration/config] - 10https://gerrit.wikimedia.org/r/1307680 (https://phabricator.wikimedia.org/T430536) (owner: 10Kevin Bazira) [07:48:58] 06Release-Engineering-Team, 06Infrastructure-Foundations: Add tox-uv support to the tox-v{3,4} Docker images - https://phabricator.wikimedia.org/T421348#12088861 (10elukey) @Lferreira Hi! I noticed that Tyler is on sabbatical, can I ping to you for this task? [07:57:19] 10GitLab: Reset MFA on gitlab - https://phabricator.wikimedia.org/T431245 (10OKryva-WMF) 03NEW [08:03:27] (03PS1) 10Kosta Harlan: Zuul: Add WikimediaAntiAbuse extension [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) [08:05:06] (03CR) 10CI reject: [V:04-1] Zuul: Add WikimediaAntiAbuse extension [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) (owner: 10Kosta Harlan) [08:06:08] (03CR) 10Jforrester: "As this is headed to Wikimedia production, add the template now (with a comment linking to the task). (Or put it in the non-Wikimedia-prod" [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) (owner: 10Kosta Harlan) [08:07:37] (03PS2) 10Kosta Harlan: Zuul: Add WikimediaAntiAbuse extension [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) [08:07:44] (03PS3) 10Kosta Harlan: Zuul: Add WikimediaAntiAbuse extension [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) [08:08:21] (03PS4) 10Kosta Harlan: Zuul: Add WikimediaAntiAbuse extension [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) [08:09:37] (03CR) 10Jforrester: [C:03+2] Zuul: Add WikimediaAntiAbuse extension [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) (owner: 10Kosta Harlan) [08:11:19] (03Merged) 10jenkins-bot: Zuul: Add WikimediaAntiAbuse extension [integration/config] - 10https://gerrit.wikimedia.org/r/1307695 (https://phabricator.wikimedia.org/T431023) (owner: 10Kosta Harlan) [08:11:35] !log Zuul: Add WikimediaAntiAbuse extension, for T431023 [08:11:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:11:38] T431023: Create and deploy Extension:WikimediaAntiAbuse - https://phabricator.wikimedia.org/T431023 [08:13:02] (03update) 10kharlan: make-release: Start branching WikimediaAntiAbuse [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/252 (https://phabricator.wikimedia.org/T431023) [08:13:05] (03open) 10kharlan: make-release: Start branching WikimediaAntiAbuse [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/252 (https://phabricator.wikimedia.org/T431023) [08:15:11] hi, who would be able to review/merge https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/252 ? [08:15:20] James_F: thanks for merging the CI patch! [08:15:46] Of course. :-) [08:30:24] (03merge) 10jforrester: make-release: Start branching WikimediaAntiAbuse [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/252 (https://phabricator.wikimedia.org/T431023) (owner: 10kharlan) [10:09:28] (03CR) 10Hashar: [C:03+2] "I have deployed the new jobs:" [integration/config] - 10https://gerrit.wikimedia.org/r/1307680 (https://phabricator.wikimedia.org/T430536) (owner: 10Kevin Bazira) [10:09:56] (03CR) 10Hashar: [C:03+2] phan: Add CentralAuth as EventBus phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1307228 (https://phabricator.wikimedia.org/T430967) (owner: 10Ottomata) [10:12:07] (03Merged) 10jenkins-bot: inference-services: Add CI pipeline jobs for tts model-server [integration/config] - 10https://gerrit.wikimedia.org/r/1307680 (https://phabricator.wikimedia.org/T430536) (owner: 10Kevin Bazira) [10:12:57] (03Merged) 10jenkins-bot: phan: Add CentralAuth as EventBus phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1307228 (https://phabricator.wikimedia.org/T430967) (owner: 10Ottomata) [10:13:17] (03CR) 10Hashar: [C:03+2] "I have reloaded Zuul." [integration/config] - 10https://gerrit.wikimedia.org/r/1307680 (https://phabricator.wikimedia.org/T430536) (owner: 10Kevin Bazira) [10:14:42] (03CR) 10Hashar: [C:03+2] "deployed and I have retriggered a run on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/1307221" [integration/config] - 10https://gerrit.wikimedia.org/r/1307228 (https://phabricator.wikimedia.org/T430967) (owner: 10Ottomata) [13:16:48] hashar: Do you remember the experiment b.d808 did earlier this year to check if rolling out missing extension i18n would break scap, which showed we didn't need to worry too much about the timing nowadays? k.ostajh has a new extension to put the i18n loading into production immediately, but he (reasonably!) wants to see the evidence that this doesn't break the world any more. [14:36:22] 06Release-Engineering-Team (Radar), 10QS-Test-Automation, 07Epic: [EPIC] Create Critical User Journey test automation for CI - https://phabricator.wikimedia.org/T422555#12090736 (10zeljkofilipin) a:03zeljkofilipin [14:42:12] James_F: http://phabricator.wikimedia.org/T411516 [14:42:18] https://gerrit.wikimedia.org/r/c/1225023 [14:42:29] http://phabricator.wikimedia.org/T411516 [14:42:34] sigh [14:42:35] https://phabricator.wikimedia.org/T411516#11550236 [14:42:39] >The train ran for a week and continued to run in the next week without any identified problems. I think we can call this experiment a success and proceed to the cleanup phase where we update docs on mediawiki.org to stop telling people that they need to rush to get their extension/skin on the train early. [14:49:08] Reedy: So… it does work? Or rather, the errors get silently swallowed? [14:49:18] Same thing? :P [14:49:23] * James_F coughs. [15:19:41] 10Phabricator, 10Test Kitchen, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07OKR-Work: Allow egress from test kitchen to the phabricator API - https://phabricator.wikimedia.org/T430024#12090923 (10brouberol) [15:20:28] (03merge) 10dancy: requirements: Bump vulnerable dependencies to address Dependabot alerts [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1214 [15:20:44] (03update) 10dancy: package-lock.json: Updates from npm audit fix [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1215 [15:23:20] 10GitLab, 06Release-Engineering-Team (Radar), 06collaboration-services: Update gitlab cookbooks and processes - https://phabricator.wikimedia.org/T430655#12090933 (10LSobanski) p:05Triage→03High [15:24:01] 10GitLab, 06Release-Engineering-Team (Radar), 06collaboration-services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12090938 (10LSobanski) {T430655} should be considered a blocker for the final change. [15:24:16] (03merge) 10dancy: package-lock.json: Updates from npm audit fix [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1215 [15:24:59] 10GitLab, 06Release-Engineering-Team (Radar), 06collaboration-services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12090941 (10LSobanski) [15:25:48] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 10observability: Enable Zuul OpenTelemetry tracing - https://phabricator.wikimedia.org/T430900#12090944 (10LSobanski) p:05Triage→03Low [15:28:41] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 10observability: Enable Zuul OpenTelemetry tracing - https://phabricator.wikimedia.org/T430900#12090950 (10LSobanski) Is the work here to configure Zuul, configure the receiving system (Jaeger) or both of the above? [15:29:18] (03open) 10dancy: Release 4.272.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1216 [15:31:38] (03merge) 10dancy: Release 4.272.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1216 [15:51:12] (03approved) 10brennen: add repos/sre/etherpad to Trusted Runners [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/180 (owner: 10jelto) [16:02:11] 10Continuous-Integration-Infrastructure (Zuul upgrade), 13Patch-For-Review: Zuul web API calls are cached - https://phabricator.wikimedia.org/T430462#12091083 (10Dzahn) Well, a patch to disable caching has also been uploaded. Feel free to review it. [16:40:32] (03approved) 10aghirelli: Rules: allow valid media types and handle polymorphic properties [repos/ci-tools/wikimedia-spectral-ruleset] - 10https://gitlab.wikimedia.org/repos/ci-tools/wikimedia-spectral-ruleset/-/merge_requests/14 (https://phabricator.wikimedia.org/T428149) (owner: 10milazg) [16:40:54] (03merge) 10aghirelli: Rules: allow valid media types and handle polymorphic properties [repos/ci-tools/wikimedia-spectral-ruleset] - 10https://gitlab.wikimedia.org/repos/ci-tools/wikimedia-spectral-ruleset/-/merge_requests/14 (https://phabricator.wikimedia.org/T428149) (owner: 10milazg) [16:52:20] 06Release-Engineering-Team, 10Cloud-Services, 06collaboration-services, 06Data-Persistence, and 7 others: codfw: rack B5 maintenance - https://phabricator.wikimedia.org/T430918#12091218 (10Dzahn) [16:53:54] 06Release-Engineering-Team, 10Cloud-Services, 06collaboration-services, 06Data-Persistence, and 7 others: codfw: rack B5 maintenance - https://phabricator.wikimedia.org/T430918#12091221 (10Dzahn) re: contint2002 - This is not the main CI server, that is currently contint1002, but it is still used to a cert... [16:54:40] 06Project-Admins, 10WikimediaCustomizations: Split WikimediaCustomizations Phabricator project into subprojects - https://phabricator.wikimedia.org/T428727#12091223 (10Aklapper) 05Open→03Stalled This welcomes more consensus. Discussss! :D [17:11:43] 10Phabricator, 10Test Kitchen, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07OKR-Work: Allow egress from test kitchen to the phabricator API - https://phabricator.wikimedia.org/T430024#12091264 (10brouberol) 05In progress→03Resolved [17:18:57] (03open) 10dancy: requirements: Bump fastapi and starlette to fix CVE-2025-62727 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1217 [17:20:03] (03update) 10dancy: requirements: Bump fastapi and starlette to fix CVE-2025-62727 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1217 [17:20:06] (03update) 10dancy: requirements: Bump fastapi and starlette to fix CVE-2025-62727 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1217 [17:27:21] 10Continuous-Integration-Infrastructure (Zuul upgrade), 13Patch-For-Review: Zuul web API calls are cached - https://phabricator.wikimedia.org/T430462#12091408 (10hashar) > https://gerrit.wikimedia.org/r/c/operations/puppet/+/1306353 cache: turn off caching for zuul.wikimedia.org Indeed, as I said on IRC: the... [19:18:54] (03merge) 10dancy: requirements: Bump fastapi and starlette to fix CVE-2025-62727 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1217 [19:31:49] (03open) 10dancy: Dockerfile: Bump base bookworm image tag to 20260705 [repos/releng/buildkit] (wmf/build) - 10https://gitlab.wikimedia.org/repos/releng/buildkit/-/merge_requests/81 (https://phabricator.wikimedia.org/T416707) [19:34:39] (03merge) 10dancy: Dockerfile: Bump base bookworm image tag to 20260705 [repos/releng/buildkit] (wmf/build) - 10https://gitlab.wikimedia.org/repos/releng/buildkit/-/merge_requests/81 (https://phabricator.wikimedia.org/T416707) [19:51:55] (03open) 10dancy: staging/staging.tfvars: Use buildkitd wmf-v0.31.1 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/619 (https://phabricator.wikimedia.org/T429988) [19:53:11] (03merge) 10dancy: staging/staging.tfvars: Use buildkitd wmf-v0.31.1 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/619 (https://phabricator.wikimedia.org/T429988) [19:55:51] (03open) 10dancy: production/prod.tfvars: Use buildkitd wmf-v0.31.1 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/620 (https://phabricator.wikimedia.org/T429988) [19:57:51] (03merge) 10dancy: production/prod.tfvars: Use buildkitd wmf-v0.31.1 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/620 (https://phabricator.wikimedia.org/T429988) [20:02:01] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Ensure PipelineLib works on new Jenkins host - https://phabricator.wikimedia.org/T423968#12091958 (10Dzahn) Has this been resolved? Asking because we tried to switch-over before a... [20:06:40] !log Updated buildkitd to v0.31.1 in gitlab-cloud-runners (T429988) [20:06:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:06:44] T429988: Buildkit v0.31.1 released - https://phabricator.wikimedia.org/T429988 [20:09:27] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work, 13Patch-For-Review: Buildkit v0.31.1 released - https://phabricator.wikimedia.org/T429988#12091995 (10dancy) 05Open→03In progress p:05Triage→03Low a:03dancy [20:45:22] 10WikimediaDebug, 06MW-Interfaces-Team, 06ServiceOps new, 10ServiceOps-Mediawiki, 10ServiceOps-SharedInfra: WikimediaDebug makes api.wikimedia.org always 301 redirect - https://phabricator.wikimedia.org/T430600#12092178 (10Krinkle) [23:12:36] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily-Math] Add Math job [integration/config] - 10https://gerrit.wikimedia.org/r/1307868 (https://phabricator.wikimedia.org/T427008) [23:14:58] (03CR) 10Vaughn Walters: "This is ready for review" [integration/config] - 10https://gerrit.wikimedia.org/r/1307868 (https://phabricator.wikimedia.org/T427008) (owner: 10Vaughn Walters) [23:42:20] 06Release-Engineering-Team, 10Cloud-Services, 06collaboration-services, 06Data-Persistence, and 7 others: codfw: rack B5 maintenance - https://phabricator.wikimedia.org/T430918#12092634 (10Eevans)