[00:56:40] 10GitLab (Project Migration), 06Release-Engineering-Team: Create new GitLab project group: lua - https://phabricator.wikimedia.org/T423762#11837316 (10tstarling) @ssastry says on Slack that `repos/lua` might be too generic, and suggests `wikilua` or `luamodules`. I would prefer either `lua` or `produnto`. Prod... [02:56:17] (03CR) 10Subramanya Sastry: "recheck" [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1218841 (owner: 10Subramanya Sastry) [02:57:30] (03PS2) 10Subramanya Sastry: Bump puppeteer to latest version [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1218841 [03:08:42] 10GitLab (Project Migration), 06Release-Engineering-Team: Create new GitLab project group: lua - https://phabricator.wikimedia.org/T423762#11837364 (10ssastry) Sounds good. Works for me. [03:14:48] 10Beta-Cluster-Infrastructure: 2FA removal and request for sysop and interfaceadmin on zh.wikipedia.beta.wmcloud.org - https://phabricator.wikimedia.org/T423830 (10amanojaku) 03NEW [04:34:47] 10Beta-Cluster-Infrastructure: 2FA removal and request for sysop and interfaceadmin on zh.wikipedia.beta.wmcloud.org - https://phabricator.wikimedia.org/T423830#11837454 (10Bugreporter) [04:53:57] 10Beta-Cluster-Infrastructure: 2FA removal and request for sysop and interfaceadmin on zh.wikipedia.beta.wmcloud.org - https://phabricator.wikimedia.org/T423830#11837461 (10amanojaku) >>! In T423830#11837453, @Bugreporter wrote: > Do you still have access to the "Vanished user 1929210" account? You should use th... [07:51:43] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, and 2 others: gerrit: Adapt timeouts to avoid 502 errors in CI jobs - https://phabricator.wikimedia.org/T421827#11837765 (10ABran-WMF) I've tried increasing `initial_connection_window_size` and `initial_stream_window_size`, but stil... [09:28:40] (03update) 10kamila: Draft: Change base image to use ICU 72 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/239 (https://phabricator.wikimedia.org/T419049 https://phabricator.wikimedia.org/T421913) [09:28:50] (03update) 10kamila: Change base image to use ICU 72 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/239 (https://phabricator.wikimedia.org/T419049 https://phabricator.wikimedia.org/T421913) [09:30:26] (03merge) 10kamila: Change base image to use ICU 72 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/239 (https://phabricator.wikimedia.org/T419049 https://phabricator.wikimedia.org/T421913) [10:21:56] We, currently working on a Wikibase feature that is supposed to be prototyped on top of (but not in) REL1_45, we're wondering how much work it would be to get CI like on REL1_45 set up for such a feature branch. [10:33:24] is it intentional that mwext-codehealth-master-non-voting only seems to run one build at a time? postmerge in zuul is pretty backlogged at the moment, as a result of that AFAICT [11:28:55] Hi folks. I'm getting console errors in chrome on gerrit. Firefox seems to be fine. https://usercontent.irccloud-cdn.com/file/1t13HhYd/image.png [11:29:48] ha. now it's working [11:29:52] might have been my network [12:05:43] 10Scap: scap security-check fails with memoryview: a bytes-like object is required, not 'str' - https://phabricator.wikimedia.org/T423862 (10Zabe) 03NEW [12:25:05] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, and 2 others: gerrit: Adapt timeouts to avoid 502 errors in CI jobs - https://phabricator.wikimedia.org/T421827#11838692 (10ABran-WMF) I've tried to disable http2 on envoy but it also triggered: https://integration.wikimedia.org/ci/... [12:44:25] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review, 07Sustainability (Incident Followup): Move Gerrit data out of root partition - https://phabricator.wikimedia.org/T333143#11838723 (10Jelto) [12:50:47] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review, 07Sustainability (Incident Followup): Move Gerrit data out of root partition - https://phabricator.wikimedia.org/T333143#11838740 (10Jelto) There is an active incinga alert for the migrated spare host: ` PROCS CRITICAL:... [13:05:17] (03CR) 10Jforrester: [C:04-2] "Hey, yes, it's a really slow thing to manually test – I'll need to land, then work through each of the ~30 extensions that directly or ind" [integration/config] - 10https://gerrit.wikimedia.org/r/1248853 (owner: 10Jforrester) [13:10:34] 06Project-Admins: Create project tag for Quibble Local - https://phabricator.wikimedia.org/T423867 (10zeljkofilipin) 03NEW [13:10:48] 06Project-Admins: Create project tag for Quibble Local - https://phabricator.wikimedia.org/T423867#11838837 (10zeljkofilipin) [13:29:26] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, and 2 others: gerrit: Adapt timeouts to avoid 502 errors in CI jobs - https://phabricator.wikimedia.org/T421827#11838973 (10ABran-WMF) After checking Envoy's logs, I'll disable http2 again, Envoy was not fully reloaded for the previ... [13:36:31] 06Project-Admins, 06Release-Engineering-Team (Doing 😎): Create project tag for Quibble Local - https://phabricator.wikimedia.org/T423867#11838996 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project #Quibble-Local has been created: https://phabricator.wikimedia.org/project/view/8655/... [13:37:11] 06Project-Admins: Allow to search tasks about MediaWiki core and core only (create MediaWiki umbrella project?) - https://phabricator.wikimedia.org/T76942#11839000 (10Aklapper) 05Open→03Declined [13:39:15] andre: re T76942, i had a wild idea that came to mind at some point regarding a possible solution to that, but i didnt get to fully writing it down yet, or working out if it'd actually be feasible in practice (and might not be able to for a bit of time). i'll probably comment on that (now-declined) task if/when i get to actually writing that idea down :p [13:39:15] T76942: Allow to search tasks about MediaWiki core and core only (create MediaWiki umbrella project?) - https://phabricator.wikimedia.org/T76942 [13:40:26] A_smart_kitten: eh, sure :) I think we're just too late. Lots of tasks also only tagged with MW-Core "team" projects which changed with every reorg about every 2y [13:45:37] (03open) 10damilare: Add image for Fundraising Fraud Prediction service [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/101 [13:52:10] (03CR) 10Zfilipin: [C:03+2] documentation: Use tmpfs locally to mimic production [integration/quibble] - 10https://gerrit.wikimedia.org/r/1271698 (https://phabricator.wikimedia.org/T423127) (owner: 10Phedenskog) [13:58:29] 06Project-Admins, 06Release-Engineering-Team (Doing 😎), 10Quibble-Local: Create project tag for Quibble Local - https://phabricator.wikimedia.org/T423867#11839144 (10zeljkofilipin) [14:09:08] (03CR) 10CI reject: [V:04-1] documentation: Use tmpfs locally to mimic production [integration/quibble] - 10https://gerrit.wikimedia.org/r/1271698 (https://phabricator.wikimedia.org/T423127) (owner: 10Phedenskog) [14:22:40] 10Beta-Cluster-Infrastructure, 06Data-Engineering, 06MW-Interfaces-Team, 10WMF-JobQueue, 10Event-Platform: Jobs are not being processed in beta, April 2026 edition - https://phabricator.wikimedia.org/T423615#11839257 (10Daimona) FTR, I checked the archived logs to see about when this broke. The relevant... [14:30:52] 10Fresh, 10Gerrit, 06collaboration-services: Wikimedia gerrit load management 429s break fresh-install - https://phabricator.wikimedia.org/T421726#11839311 (10Jelto) I can't see the patches above and don't have the full context, but I think the user agent in https://requestctl.wikimedia.org/pattern/ua/gerrit... [14:37:06] 10Fresh, 10Gerrit, 06collaboration-services: Wikimedia gerrit load management 429s break fresh-install - https://phabricator.wikimedia.org/T421726#11839367 (10ABran-WMF) >>! In T421726#11839311, @Jelto wrote: > I can't see the patches above and don't have the full context, but I think the user agent in https... [14:39:10] 06Project-Admins: Create project tag for HTML Tags extension - https://phabricator.wikimedia.org/T423882 (10Yaron_Koren) 03NEW [14:51:51] 06Release-Engineering-Team (Radar), 06Infrastructure-Foundations, 06SRE: New base images without mirrors.wikimedia.org - https://phabricator.wikimedia.org/T423622#11839483 (10Jdforrester-WMF) 05Open→03In progress a:03Jdforrester-WMF [14:56:34] (03PS1) 10Jforrester: Docker: Bump ci-bullseye/-bookworm/-trixie for mirrors.wm.org removal [integration/config] - 10https://gerrit.wikimedia.org/r/1275432 (https://phabricator.wikimedia.org/T423622) [14:59:28] (03CR) 10Jforrester: [C:03+2] Docker: Bump ci-bullseye/-bookworm/-trixie for mirrors.wm.org removal [integration/config] - 10https://gerrit.wikimedia.org/r/1275432 (https://phabricator.wikimedia.org/T423622) (owner: 10Jforrester) [15:00:15] 06Project-Admins, 06Release-Engineering-Team (Doing 😎): Create project tag for HTML Tags extension - https://phabricator.wikimedia.org/T423882#11839528 (10Aklapper) 05Open→03Resolved a:03Aklapper Thank you! Requested public project #mediawiki-extensions-html_tags has been created: https://phabricato... [15:01:15] (03Merged) 10jenkins-bot: Docker: Bump ci-bullseye/-bookworm/-trixie for mirrors.wm.org removal [integration/config] - 10https://gerrit.wikimedia.org/r/1275432 (https://phabricator.wikimedia.org/T423622) (owner: 10Jforrester) [15:03:14] !log Docker: Bump ci-bullseye/-bookworm/-trixie for mirrors.wm.org removal, T423622 [15:03:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:03:17] T423622: New base images without mirrors.wikimedia.org - https://phabricator.wikimedia.org/T423622 [15:06:02] (03CR) 10C. Scott Ananian: [C:03+2] Bump puppeteer to latest version [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1218841 (owner: 10Subramanya Sastry) [15:06:14] 10Fresh, 10Gerrit, 06collaboration-services: Wikimedia gerrit load management 429s break fresh-install - https://phabricator.wikimedia.org/T421726#11839579 (10sbassett) >>! In T421726#11839311, @Jelto wrote: > I can't see the patches above and don't have the full context, but I think the user agent in https:... [15:06:40] (03Merged) 10jenkins-bot: Bump puppeteer to latest version [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1218841 (owner: 10Subramanya Sastry) [15:09:08] We are currently working on a Wikibase feature that is supposed to be prototyped on top of (but not in) REL1_45. We were wondering how much work it would be to get CI equivalent to REL1_45 set up for such a feature branch (meaning that it tests with core 1.45 also). [15:19:53] (03CR) 10Zfilipin: documentation: Use tmpfs locally to mimic production [integration/quibble] - 10https://gerrit.wikimedia.org/r/1271698 (https://phabricator.wikimedia.org/T423127) (owner: 10Phedenskog) [15:20:00] (03CR) 10Zfilipin: [C:03+2] documentation: Use tmpfs locally to mimic production [integration/quibble] - 10https://gerrit.wikimedia.org/r/1271698 (https://phabricator.wikimedia.org/T423127) (owner: 10Phedenskog) [15:33:59] 06Release-Engineering-Team, 06Wikimedia Enterprise: Trusted Gitlab Runner Access [Access Request] - https://phabricator.wikimedia.org/T423893 (10LDlulisa-WMF) 03NEW [15:36:24] (03CR) 10CI reject: [V:04-1] documentation: Use tmpfs locally to mimic production [integration/quibble] - 10https://gerrit.wikimedia.org/r/1271698 (https://phabricator.wikimedia.org/T423127) (owner: 10Phedenskog) [15:42:30] 10Gerrit, 06collaboration-services, 07Documentation, 07Sustainability (Incident Followup): Update and improve operation runbooks and documentation for Gerrit - https://phabricator.wikimedia.org/T423601#11839881 (10LSobanski) a:03ABran-WMF [15:42:41] 10Gerrit, 06collaboration-services, 07Documentation, 07Sustainability (Incident Followup): Update and improve operation runbooks and documentation for Gerrit - https://phabricator.wikimedia.org/T423601#11839883 (10LSobanski) p:05Triage→03High [15:48:51] 10GitLab (CI & Job Runners), 06collaboration-services, 07IPv6: IPv6 non-functional in GitLab CI environments - https://phabricator.wikimedia.org/T423310#11839914 (10taavi) (Partial) duplicate of {T403746}? [15:51:48] 10GitLab (CI & Job Runners), 06collaboration-services, 07IPv6: Support IPv6 on WMCS hosted runners - https://phabricator.wikimedia.org/T403746#11839940 (10Jelto) [16:04:06] 06Project-Admins, 06Release-Engineering-Team (Doing 😎): Create project tag for HTML Tags extension - https://phabricator.wikimedia.org/T423882#11840097 (10Yaron_Koren) Thank you also! [16:25:42] 10GitLab (Account Approval), 06Release-Engineering-Team, 10Tool-gitlab-account-approval: GitLab Account Approval Bot is failing - https://phabricator.wikimedia.org/T423909 (10brennen) 03NEW [16:29:52] 10GitLab (Account Approval), 06Release-Engineering-Team, 10Tool-gitlab-account-approval: GitLab Account Approval Bot is failing - https://phabricator.wikimedia.org/T423909#11840237 (10brennen) From logs: ` 2026-04-20T16:27:13Z glaab.unhandled CRITICAL: Uncaught exception Traceback (most recent call last):... [16:40:24] 10GitLab (Account Approval), 06Release-Engineering-Team, 10Tool-gitlab-account-approval: GitLab Account Approval Bot is failing - https://phabricator.wikimedia.org/T423909#11840355 (10brennen) Looks like this has been happening on (nearly?) every run since 2026-04-18 12:43 UTC. Running a dry-run per the `RE... [16:55:07] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Update to Phorge/Arcanist upstream (2026.xx) - https://phabricator.wikimedia.org/T410849#11840442 (10Aklapper) [17:01:31] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:02:03] (03open) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:03:29] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:03:35] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:03:58] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:04:04] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:20:38] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:22:47] 10GitLab (Account Approval), 06Release-Engineering-Team, 10Tool-gitlab-account-approval: GitLab Account Approval Bot is failing - https://phabricator.wikimedia.org/T423909#11840555 (10bd808) p:05Triage→03High a:03bd808 [17:23:16] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [17:38:09] 10GitLab (Account Approval), 06Release-Engineering-Team, 10Tool-gitlab-account-approval: GitLab Account Approval Bot is failing - https://phabricator.wikimedia.org/T423909#11840604 (10bd808) 05Open→03Resolved I forgot to add the task id when logging [[https://sal.toolforge.org/log/qu3yq50BffdvpiTrPwa... [17:47:55] PROBLEM - jenkins_service_running on contint1003 is CRITICAL: PROCS CRITICAL: 0 processes with regex args .*/bin/java .*-jar /usr/share/java/jenkins.war https://wikitech.wikimedia.org/wiki/Jenkins [17:48:55] RECOVERY - jenkins_service_running on contint1003 is OK: PROCS OK: 1 process with regex args .*/bin/java .*-jar /usr/share/java/jenkins.war https://wikitech.wikimedia.org/wiki/Jenkins [17:56:38] (03PS1) 10Jforrester: jjb: Bump ci-bullseye/-bookworm/-trixie images for mirrors.wm.org removal [integration/config] - 10https://gerrit.wikimedia.org/r/1275486 (https://phabricator.wikimedia.org/T423622) [17:58:50] 10Scap: scap security-check fails with memoryview: a bytes-like object is required, not 'str' - https://phabricator.wikimedia.org/T423862#11840744 (10dancy) p:05Triage→03Low a:03dancy [18:08:54] (03open) 10dancy: Add repos/wme/pageviews-data-transfer to projects.json [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/165 (https://phabricator.wikimedia.org/T423893) [18:08:57] (03update) 10dancy: Add repos/wme/pageviews-data-transfer to projects.json [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/165 (https://phabricator.wikimedia.org/T423893) [18:09:03] (03PS1) 10Aude: Zuul: [mediawiki/extensions/WikimediaCustomizations] Add CentralAuth dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1275487 (https://phabricator.wikimedia.org/T416949) [18:09:43] (03PS2) 10Aude: Zuul: [mediawiki/extensions/WikimediaCustomizations] Add CentralAuth dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1275487 (https://phabricator.wikimedia.org/T420548) [18:10:06] (03merge) 10dancy: Add repos/wme/pageviews-data-transfer to projects.json [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/165 (https://phabricator.wikimedia.org/T423893) [18:11:28] 06Release-Engineering-Team, 06Wikimedia Enterprise, 13Patch-For-Review: Trusted Gitlab Runner Access [Access Request] - https://phabricator.wikimedia.org/T423893#11840844 (10dancy) 05Open→03Resolved a:03dancy Done. [18:18:05] !log Unblock 113.128.0.0/15 [18:18:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:19:03] !log jjb: All 486 (!) jobs now updated for T423622 [18:19:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:19:05] T423622: New base images without mirrors.wikimedia.org - https://phabricator.wikimedia.org/T423622 [18:19:06] (03CR) 10Jforrester: [C:03+2] jjb: Bump ci-bullseye/-bookworm/-trixie images for mirrors.wm.org removal [integration/config] - 10https://gerrit.wikimedia.org/r/1275486 (https://phabricator.wikimedia.org/T423622) (owner: 10Jforrester) [18:22:16] (03Merged) 10jenkins-bot: jjb: Bump ci-bullseye/-bookworm/-trixie images for mirrors.wm.org removal [integration/config] - 10https://gerrit.wikimedia.org/r/1275486 (https://phabricator.wikimedia.org/T423622) (owner: 10Jforrester) [18:24:47] (03open) 10dancy: README.md: Add validation steps to cluster upgrade notes [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/573 [18:24:52] (03update) 10dancy: README.md: Add validation steps to cluster upgrade notes [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/573 [18:26:13] (03merge) 10dancy: README.md: Add validation steps to cluster upgrade notes [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/573 [18:28:14] (03approved) 10thcipriani: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) (owner: 10dancy) [18:28:23] !log Upgrading gitlab cloud runners (staging) (T423726) [18:28:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:28:25] T423726: Update gitlab-cloud-runners kubernetes - https://phabricator.wikimedia.org/T423726 [18:28:57] !log Upgrading gitlab cloud runners (staging) to 1.33.9-do.2 (T423726) [18:29:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:36:37] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [18:36:51] (03update) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [18:39:14] (03merge) 10dancy: Drop `scap security-check` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1131 (https://phabricator.wikimedia.org/T423862) [18:41:52] (03open) 10dancy: Release 4.249.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1132 [18:42:45] 06Release-Engineering-Team (Radar), 06Infrastructure-Foundations, 06SRE: New base images without mirrors.wikimedia.org - https://phabricator.wikimedia.org/T423622#11841044 (10Jdforrester-WMF) 05In progress→03Resolved OK, this should now be Resolved. Hopefully. [18:44:21] (03merge) 10dancy: Release 4.249.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1132 [18:55:22] (03open) 10dancy: watchlist.py: Adjustments for k8s 1.32 [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/39 [18:55:26] (03update) 10dancy: watchlist.py: Adjustments for k8s 1.32 [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/39 [18:56:54] (03merge) 10dancy: watchlist.py: Adjustments for k8s 1.32 [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/39 [19:00:00] 10Scap: scap security-check fails with memoryview: a bytes-like object is required, not 'str' - https://phabricator.wikimedia.org/T423862#11841103 (10dancy) 05Open→03Resolved [19:00:01] @s [19:00:10] ^ accidental keysmash. [19:02:25] 06Project-Admins: Create project tag for MW Drivel Defence - https://phabricator.wikimedia.org/T423775#11841118 (10Squeak24) I have just updated the git repository. I have now created an extension page, I need to push a new commit, I will try and do that in the coming days. I have modified the readme.md. https... [19:03:24] (03update) 10dancy: staging/digitalocean.tfvars: kubernetes_version = "1.33.9-do.2" [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/574 (https://phabricator.wikimedia.org/T423726) [19:03:27] (03open) 10dancy: staging/digitalocean.tfvars: kubernetes_version = "1.33.9-do.2" [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/574 (https://phabricator.wikimedia.org/T423726) [19:04:40] (03merge) 10dancy: staging/digitalocean.tfvars: kubernetes_version = "1.33.9-do.2" [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/574 (https://phabricator.wikimedia.org/T423726) [19:05:12] (03open) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.10 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/575 (https://phabricator.wikimedia.org/T423726) [19:05:14] (03update) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.10 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/575 (https://phabricator.wikimedia.org/T423726) [19:06:20] (03merge) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.10 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/575 (https://phabricator.wikimedia.org/T423726) [19:21:17] (03update) 10dancy: watchlist.py: More adjustments for k8s 1.32 [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/40 [19:21:19] (03open) 10dancy: watchlist.py: More adjustments for k8s 1.32 [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/40 [19:22:32] (03merge) 10dancy: watchlist.py: More adjustments for k8s 1.32 [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/40 [19:25:18] (03open) 10dancy: watchlist.py: Adjustments for k8s 1.32 (3) [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/41 [19:25:21] (03update) 10dancy: watchlist.py: Adjustments for k8s 1.32 (3) [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/41 [19:26:25] (03update) 10dancy: watchlist.py: Adjustments for k8s 1.32 (3) [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/41 [19:27:00] (03merge) 10dancy: watchlist.py: Adjustments for k8s 1.32 (3) [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/41 [19:33:37] (03open) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.11 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/576 (https://phabricator.wikimedia.org/T423726) [19:33:39] (03update) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.11 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/576 (https://phabricator.wikimedia.org/T423726) [19:34:59] (03merge) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.11 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/576 (https://phabricator.wikimedia.org/T423726) [19:45:58] (03open) 10dancy: Revised watchlist implementation (again) [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/42 [19:46:03] (03update) 10dancy: Revised watchlist implementation (again) [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/42 [19:47:07] (03merge) 10dancy: Revised watchlist implementation (again) [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/42 [20:07:02] 10Continuous-Integration-Config: Clear frontend cache when publish new coverage report under https://doc.wikimedia.org/cover - https://phabricator.wikimedia.org/T423951 (10Umherirrender) 03NEW [20:15:49] 10Continuous-Integration-Config: Purge frontend cache when publish new coverage report under https://doc.wikimedia.org/cover - https://phabricator.wikimedia.org/T423951#11841449 (10Umherirrender) [21:02:44] 10Continuous-Integration-Config, 10WikimediaApiPortal, 07ci-test-error: mediawiki-node24 failure for skin WikimediaApiPortal (due to node-sass) - https://phabricator.wikimedia.org/T423954 (10Umherirrender) 03NEW [21:08:20] 10Continuous-Integration-Config, 10WikimediaApiPortal, 07ci-test-error: mediawiki-node24 failure for skin WikimediaApiPortal (due to node-sass) - https://phabricator.wikimedia.org/T423954#11841612 (10apaskulin) Thank you for opening this and for your work on the upgrade patches, @Umherirrender! [21:11:20] (03open) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.12 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/577 [21:11:24] (03update) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.12 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/577 [21:12:47] (03merge) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Bump to v1.0.12 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/577 [21:46:18] Project beta-scap-sync-world build #254396: 04FAILURE in 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/254396/ [21:56:18] Project beta-scap-sync-world build #254397: 04STILL FAILING in 1 min 0 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/254397/ [21:56:38] 10Continuous-Integration-Config, 06Traffic: Purge frontend cache when publish new coverage report under https://doc.wikimedia.org/cover - https://phabricator.wikimedia.org/T423951#11841694 (10Reedy) [21:56:42] wmf-insecte: behave [21:56:42] Reedy you may not issue bot commands in this chat! [21:58:22] `Warning: Undefined array key "default" in /srv/mediawiki-staging/wmf-config/CommonSettings-labs.php on line 576` [21:59:40] `$wmgApprovedContentSecurityPolicyDomains['default']` is the undefined thing [22:00:09] Already fixeded [22:00:48] (03update) 10swfrench: sync-world: Offer to rollback k8s deployments [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1129 (https://phabricator.wikimedia.org/T225207 https://phabricator.wikimedia.org/T375497 https://phabricator.wikimedia.org/T390531 https://phabricator.wikimedia.org/T394858 https://phabricator.wikimedia.org/T396106) (owner: 10dancy) [22:07:28] Yippee, build fixed! [22:07:28] Project beta-scap-sync-world build #254398: 09FIXED in 2 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/254398/ [22:40:17] (03PS1) 10Dduvall: systemtests: Update jenkins to 2.555.1 [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1275545 (https://phabricator.wikimedia.org/T418109) [22:43:01] (03CR) 10Dduvall: [C:03+2] systemtests: Update jenkins to 2.555.1 [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1275545 (https://phabricator.wikimedia.org/T418109) (owner: 10Dduvall) [22:43:33] (03Merged) 10jenkins-bot: systemtests: Update jenkins to 2.555.1 [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1275545 (https://phabricator.wikimedia.org/T418109) (owner: 10Dduvall) [23:30:03] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Ensure PipelineLib works on new Jenkins host - https://phabricator.wikimedia.org/T423968 (10thcipriani) 03NEW [23:46:03] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Ensure PipelineLib works on new Jenkins host - https://phabricator.wikimedia.org/T423968#11842097 (10thcipriani) per @dduvall `make systemtest` worked under https://gerrit.wikimedi... [23:56:05] !log Unblock 76.157.0.0/16 [23:56:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL