[00:05:14] 10WikimediaDebug, 10MW-on-K8s, 10noc.wikimedia.org, 06ServiceOps new, and 2 others: Enable WikimediaDebug support for noc.wikimedia.org - https://phabricator.wikimedia.org/T395682#11656837 (10Scott_French) Thanks, @Krinkle - That helps clarify. For noc.wikimedia.org, presence of the x-wikimedia-debug heade... [01:03:38] 10Continuous-Integration-Infrastructure, 06collaboration-services, 13Patch-For-Review: setup 2 contint machines for zuul or jenkins - https://phabricator.wikimedia.org/T418521#11656920 (10Dzahn) [01:03:48] 10Continuous-Integration-Infrastructure, 06collaboration-services, 13Patch-For-Review: setup 2 contint machines for zuul or jenkins - https://phabricator.wikimedia.org/T418521#11656922 (10Dzahn) [01:09:16] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06collaboration-services, 06DC-Ops, and 2 others: codfw: request for a decom'ed R440 - Config C - https://phabricator.wikimedia.org/T418545#11656939 (10Dzahn) [01:10:03] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06collaboration-services, 06DC-Ops, and 2 others: eqiad: request for a decom'ed R440 - Config C - https://phabricator.wikimedia.org/T418544#11656940 (10Dzahn) [01:10:55] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: setup 2 contint machines for zuul or jenkins - https://phabricator.wikimedia.org/T418521#11656945 (10Dzahn) [01:11:46] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06collaboration-services, 06DC-Ops, and 2 others: codfw: request for a decom'ed R440 - Config C - https://phabricator.wikimedia.org/T418545#11656951 (10Dzahn) requesting public IPs at T418520 [01:11:50] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06collaboration-services, 06DC-Ops, and 2 others: eqiad: request for a decom'ed R440 - Config C - https://phabricator.wikimedia.org/T418544#11656954 (10Dzahn) requesting public IPs at T418520 [01:12:20] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services: Update Jenkins hosts from Java 17 to Java 21 - https://phabricator.wikimedia.org/T418109#11656957 (10Dzahn) requesting public IPs at T418520 [03:02:40] (03PS3) 10LWatson: Zuul: [mediawiki/extensions/MobileFrontend] Add ParserMigration dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1244746 (https://phabricator.wikimedia.org/T415451) (owner: 10Matthias Mullie) [03:06:14] (03CR) 10LWatson: "Good question! I updated the MobileFrontend patch to depend on this change. That way we test the new setup with ParserMigration as a CI de" [integration/config] - 10https://gerrit.wikimedia.org/r/1244746 (https://phabricator.wikimedia.org/T415451) (owner: 10Matthias Mullie) [03:23:02] (03CR) 10LWatson: [C:03+1] Zuul: [mediawiki/extensions/MobileFrontend] Add ParserMigration dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1244746 (https://phabricator.wikimedia.org/T415451) (owner: 10Matthias Mullie) [04:39:59] (03CR) 10Dmaza: [C:03+1] "I'm thinking it might make sense to consolidate 3d2png into 3D extension (they share some bits and pieces). Would it be possible to clone " [integration/config] - 10https://gerrit.wikimedia.org/r/1240998 (owner: 10Jforrester) [05:53:01] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06collaboration-services, 06DC-Ops, and 3 others: eqiad: request for a decom'ed R440 - Config C - https://phabricator.wikimedia.org/T418544#11657176 (10VRiley-WMF) a:03VRiley-WMF [08:56:18] (03CR) 10Matthias Mullie: "Either works. I had the MobileFrontend change first, so CI wouldn't block other patches until that fix was merged, but looks like Lauralyn" [integration/config] - 10https://gerrit.wikimedia.org/r/1244746 (https://phabricator.wikimedia.org/T415451) (owner: 10Matthias Mullie) [09:44:28] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Update to Phorge/Arcanist upstream (2026.xx) - https://phabricator.wikimedia.org/T410849#11657473 (10Aklapper) [09:44:29] 10Phabricator (Search), 06Release-Engineering-Team (Priority Backlog 📥): Have a more scope-restricted default global (primary) search - https://phabricator.wikimedia.org/T397558#11657472 (10Aklapper) [10:25:30] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: gerrit: move gerrit-replica behind CDN - https://phabricator.wikimedia.org/T418108#11657657 (10Novem_Linguae) [10:30:43] (03open) 10aokoth: gitlab-runner: update image version to 18.9.0 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/558 (https://phabricator.wikimedia.org/T418483) [10:32:39] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: gerrit: move gerrit-replica behind CDN - https://phabricator.wikimedia.org/T418108#11657682 (10Jelto) [10:47:56] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: gerrit: move gerrit-replica behind CDN - https://phabricator.wikimedia.org/T418108#11657772 (10Jelto) [11:20:08] 10Continuous-Integration-Config, 10Gerrit: Have an editable "CI broken" error message that can be displayed in Gerrit - https://phabricator.wikimedia.org/T418514#11657893 (10Tgr) >>! In T418514#11655655, @hashar wrote: > Although I am currently busy enough that I can't quite spend time writing such a plugin, I... [11:21:41] 10Phabricator: Port custom code in DiffusionCommitController to use upstream PhorgeRestrictableInteractionInterface - https://phabricator.wikimedia.org/T418578 (10Aklapper) 03NEW p:05Triage→03Low [11:21:57] 10Phabricator: Port custom code in DiffusionCommitController to use upstream PhorgeRestrictableInteractionInterface - https://phabricator.wikimedia.org/T418578#11657910 (10Aklapper) [11:21:58] 10Phabricator (2024-04-23): Restrict activity options (commenting, editing) related to merged commits in Diffusion - https://phabricator.wikimedia.org/T361006#11657912 (10Aklapper) [11:21:59] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Update to Phorge/Arcanist upstream (2026.xx) - https://phabricator.wikimedia.org/T410849#11657911 (10Aklapper) [12:30:36] 10Phabricator: Port custom code in DiffusionCommitController to use upstream PhorgeRestrictableInteractionInterface - https://phabricator.wikimedia.org/T418578#11658110 (10Aklapper) That upstream change removed Flags, Subscribe, Token, and the Comment field below. Our downstream change hides the Comment field be... [12:49:12] GitLab needs another short maintenance in 10 minutes [13:07:22] GitLab maintenance done [13:29:38] (03update) 10jelto: gitlab-runner: update image version to 18.9.0 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/558 (https://phabricator.wikimedia.org/T418483) (owner: 10aokoth) [14:32:17] Project mediawiki-core-doxygen build #18126: 04FAILURE in 14 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/18126/ [14:47:55] 10GitLab (Infrastructure), 06collaboration-services, 07SecTeam-Processed, 07Security, 05Vuln-VulnComponent: GitLab Security Release: 18.9.1, 18.8.5, 18.7.5 - https://phabricator.wikimedia.org/T418344#11658591 (10sbassett) [14:48:25] 10GitLab, 06collaboration-services, 07SecTeam-Processed, 07Security, 05Vuln-VulnComponent: GitLab Security Release: 18.9.1 - https://phabricator.wikimedia.org/T418483#11658595 (10sbassett) p:05Triage→03High [14:48:36] 10GitLab (Infrastructure), 06collaboration-services, 07SecTeam-Processed, 07Security, 05Vuln-VulnComponent: GitLab Security Release: 18.8.5 - https://phabricator.wikimedia.org/T418448#11658600 (10sbassett) [15:05:50] 10Continuous-Integration-Infrastructure, 07ci-test-error (WMF-deployed Build Failure): Error connecting to localhost:/workspace/db/quibble-mysql-gcm2qn1r/socket as user wikiuser: :real_connect(): (HY000/2002) - https://phabricator.wikimedia.org/T418593#11658650 (10A_smart_kitten) Copying over the full `mw-... [15:27:15] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.17 deployment blockers - https://phabricator.wikimedia.org/T413808#11658698 (10thcipriani) 05Open→03Resolved [15:31:32] Yippee, build fixed! [15:31:32] Project mediawiki-core-doxygen build #18127: 09FIXED in 13 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/18127/ [15:41:55] (03merge) 10dancy: gitlab-runner: update image version to 18.9.0 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/558 (https://phabricator.wikimedia.org/T418483) (owner: 10aokoth) [15:53:48] !log Updating gitlab-cloud-runners (staging and prod) to gitlab-runner 18.9.0. [15:53:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:39:40] Hmm, getting 'Class "JohnKary\PHPUnit\Listener\SpeedTrapListener" does not exist' when trying to run phpunit tests locally [16:39:49] High probability i managed to screw something up on my end [18:56:28] 06Release-Engineering-Team (Doing 😎): Request additional compute/storage/IP quota in gitlab-runners for Magnum - https://phabricator.wikimedia.org/T416264#11659387 (10dduvall) I'm being a little squishy with the numbers here, but according to the second highest peak of `kube_node_status_allocatable` from [[ http... [18:56:39] 06Release-Engineering-Team (Doing 😎): Request additional compute/storage/IP quota in gitlab-runners for Magnum - https://phabricator.wikimedia.org/T416264#11659388 (10dduvall) 05Open→03In progress a:03dduvall [18:57:02] 06Release-Engineering-Team (Doing 😎): Request additional compute/storage/IP quota in gitlab-runners for Magnum - https://phabricator.wikimedia.org/T416264#11659391 (10dduvall) p:05Triage→03Medium [18:57:33] 06Release-Engineering-Team (Doing 😎): Implement WMCS module in gitlab-cloud-runner - https://phabricator.wikimedia.org/T416261#11659403 (10dduvall) 05Open→03In progress p:05Triage→03Medium a:03dduvall [19:13:37] 10Deployments, 06Release-Engineering-Team, 06SRE, 07SRE-Unowned: Reduce automatic messages on #wikimedia-operations - https://phabricator.wikimedia.org/T384804#11659440 (10A_smart_kitten) (in case anyone subscribed to this ticket is interested in following it, a new/similar task was filed recently as {... [20:34:28] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06collaboration-services, 06DC-Ops, and 3 others: eqiad: request for a decom'ed R440 - Config C - https://phabricator.wikimedia.org/T418544#11659632 (10VRiley-WMF) Hey @Dzahn we have decommed Dell R440's, but none that were in the "Config... [20:55:00] 06Release-Engineering-Team (Doing 😎): Request additional compute/storage/IP quota in gitlab-runners for Magnum - https://phabricator.wikimedia.org/T416264#11659678 (10dduvall) Additional napkin math: === Instances === The `g4.cores8.ram32.disk20` instance flavor seems like the best fit based on what we're runn... [22:09:56] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [YOUR DEVELOPER ACCOUNT USERNAME HERE] - https://phabricator.wikimedia.org/T418638 (10Osuji_pius) 03NEW [22:12:22] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Osuji pius - https://phabricator.wikimedia.org/T418638#11659882 (10Osuji_pius) [22:19:45] 10Continuous-Integration-Infrastructure, 10Quibble: Quibble should not use success cache for "recheck" - https://phabricator.wikimedia.org/T418640#11659909 (10Catrope) [22:58:46] hi releng folks, is anyone aware of the CI failures with 'mkdir: cannot create directory ‘log’: Permission denied' [22:59:16] I just got a bunch of them on some trivial DonationInterface patches, e.g. https://gerrit.wikimedia.org/r/1245014 [22:59:42] Direct link to failed gate-and-submit job log: https://integration.wikimedia.org/ci/job/quibble-donationinterface-REL1_43-php82/2497/console [23:01:05] I cross-posted cscott's report of this to the releng slack channel a few minutes ago as well [23:11:34] (03PS1) 10Jeena Huneidi: catalyst jobs: Add expry time env var [integration/config] - 10https://gerrit.wikimedia.org/r/1245498 (https://phabricator.wikimedia.org/T416391) [23:12:05] (03PS2) 10Jeena Huneidi: catalyst jobs: Add expiry time env var [integration/config] - 10https://gerrit.wikimedia.org/r/1245498 (https://phabricator.wikimedia.org/T416391) [23:22:38] ejegg|afk: do you want to recheck? I'm not sure what's going on there but the following job passed [23:24:28] it looks like most of the previous ones passed too so I'm not seeing the other failures [23:38:20] hrmmm [23:40:23] that's bizarre considering the ls -laF at the end of the run that shows the log directory doesn't exist unless the parent directory had the wrong permissions [23:40:49] which seems hard to square [23:42:23] greg-g: I see you rechecked. Wanna CR+2 it to re-kick off merge jobs? (I'm wary to touch it since fundraising things) [23:45:56] 10Continuous-Integration-Infrastructure, 10Quibble: Quibble should not use success cache for "recheck" - https://phabricator.wikimedia.org/T418640#11660078 (10Umherirrender) The success cache only knows "success" and not fails. The failure in both jobs happen after the success cache was filled/checked. https:... [23:50:28] thcipriani: yeah, got distracted with the HR training, I see Umherirrender already did. [23:50:47] oh good. [23:53:52] 10Phabricator Translations Extension: Drop unused `projects/phabricator/support` group - https://phabricator.wikimedia.org/T418655 (10Pppery) 03NEW [23:53:55] 10Phabricator Translations Extension: Drop unused `projects/phabricator/support` group - https://phabricator.wikimedia.org/T418655#11660234 (10Pppery) a:03Pppery [23:54:11] 10Phabricator Translations Extension: Drop unused `projects/phabricator/support` group - https://phabricator.wikimedia.org/T418655#11660235 (10Pppery)