[03:17:06] 10Phabricator, 03Wikimedia-Hackathon-2025: Deploy user style to reduce bot comments on Phabricator - https://phabricator.wikimedia.org/T393289#10803065 (10matmarex) >>! In T393289#10793673, @Pppery wrote: > Silently hiding information by default with no way of showing it seems downright scary. I, in particular... [03:41:42] the cherrypick of gerrit 1143078 to 1.44 is not merging because of "Error: Class "Pimple\Container" not found" ( https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CheckUser/+/1143192 ). This is blocking Parsoid's patches cherrypicked to 1.44 from merging. [06:40:54] 06Release-Engineering-Team, 06collaboration-services: ProbeDown (gerrit1003) - https://phabricator.wikimedia.org/T393544#10803251 (10Jelto) 05Open→03Resolved a:03Jelto Alert resolved [06:41:11] 06Release-Engineering-Team, 06collaboration-services: ProbeDown (gerrit1003) - https://phabricator.wikimedia.org/T393563#10803256 (10Jelto) [06:43:35] 06Release-Engineering-Team, 06collaboration-services: ProbeDown (gerrit1003) - https://phabricator.wikimedia.org/T393563#10803257 (10Jelto) [06:43:48] 06Release-Engineering-Team, 06collaboration-services: ProbeDown (gerrit1003) - https://phabricator.wikimedia.org/T393563#10803260 (10Jelto) 05Open→03Resolved a:03Jelto Alert resolved [07:18:56] 10Gerrit, 06collaboration-services: Gerrit is unresponsive (2025-05-06) - https://phabricator.wikimedia.org/T393498#10803294 (10Jelto) p:05Unbreak!→03High A lot of high-traffic IP ranges were throttled and no more `ProbeDown` alerts fired. Also Gerrit is subjectively faster. I'll remove the UBN severity bu... [08:20:28] 10MediaWiki-Releasing, 06translatewiki.net, 10LPL Essential (LPL Essential 2025 Apr-Jun: CX), 05MW-1.44-release, and 2 others: Configure Translatewiki.net for REL1_44 - https://phabricator.wikimedia.org/T393514#10803401 (10Nikerabbit) p:05Triage→03Medium [08:20:55] 10MediaWiki-Releasing, 06translatewiki.net, 10LPL Essential (LPL Essential 2025 Apr-Jun: CX), 05MW-1.44-release, and 2 others: Configure Translatewiki.net for REL1_44 translation backports - https://phabricator.wikimedia.org/T393514#10803407 (10Nikerabbit) [09:06:35] Project beta-scap-sync-world build #205623: 04FAILURE in 1 min 32 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/205623/ [09:16:59] Yippee, build fixed! [09:16:59] Project beta-scap-sync-world build #205624: 09FIXED in 1 min 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/205624/ [10:40:07] hashar: why is jenkins not waking up on https://gerrit.wikimedia.org/r/c/RelPath/+/1143509 - we added to the CI list last weel? [10:40:09] week [10:43:33] RhinosF1: the email in the zuul config is lowercase, but the email in LDAP (and thus in Gerrit) has an uppercase first letter [10:44:48] (03PS2) 10RhinosF1: Zuul: fix a typo [integration/config] - 10https://gerrit.wikimedia.org/r/1143518 [10:44:54] ok ^ [10:45:13] (03CR) 10Jforrester: Zuul: fix a typo (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1143518 (owner: 10RhinosF1) [10:46:05] (03PS3) 10RhinosF1: Zuul: fix a typo [integration/config] - 10https://gerrit.wikimedia.org/r/1143518 [10:46:23] (03CR) 10RhinosF1: Zuul: fix a typo (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1143518 (owner: 10RhinosF1) [10:56:42] (03PS4) 10Jforrester: Zuul: Support capital first letter of e-mail for Aeywoo in allow list [integration/config] - 10https://gerrit.wikimedia.org/r/1143518 (owner: 10RhinosF1) [10:56:45] (03CR) 10Jforrester: [C:03+2] Zuul: Support capital first letter of e-mail for Aeywoo in allow list [integration/config] - 10https://gerrit.wikimedia.org/r/1143518 (owner: 10RhinosF1) [10:56:59] thanks James_F [10:57:07] Of course! [10:57:09] speaking of jenkins not waking up, there are two patches under https://gerrit.wikimedia.org/r/q/project:mediawiki/core+branch:REL1_44 which don't appear to have started gate-and-submit jobs after a CR+2 [10:57:22] i'm not sure whether that's expected or not but thought i should drop a note in here just in case [10:57:24] A_smart_kitten: They depend on vendor. [10:57:37] James_F: ahhhhhh, that makes sense. thanks! [10:57:50] A_smart_kitten: It's not a bug, it's how CI is configured. No CI for REL_* in vendor, they need to be manually landed by the MW release team. [10:58:01] (03Merged) 10jenkins-bot: Zuul: Support capital first letter of e-mail for Aeywoo in allow list [integration/config] - 10https://gerrit.wikimedia.org/r/1143518 (owner: 10RhinosF1) [10:58:06] (Maybe CI should be configured otherwise, but…) [10:58:24] !log Zuul: Support capital first letter of e-mail for Aeywoo in allow list [10:58:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:59:28] I'll review and land them manually. [10:59:44] Aeywoo says thank you too [11:07:38] James_F: any chance you can review https://gerrit.wikimedia.org/r/c/RelPath/+/1143509/8 [11:09:50] RhinosF1: Sure! [11:13:03] James_F: is there an assert generates no warnings [11:13:49] In PHP? Our documentation used to rule that you shouldn't use asserts in production code, because they always generate warnings and are not appropriate, but they've started to sprout anyway. :-( [11:14:02] James_F: for the test [11:14:06] it's a fix for PHP Warning: Uninitialized string offset 0 [11:14:28] Oh, right. You should be able to try/catch those? [11:14:32] If nothing else. [11:41:30] 10Phabricator: The saved-for-everyone UBN Maniphest queries don't show In Progress tasks - https://phabricator.wikimedia.org/T393690 (10Jdforrester-WMF) 03NEW [11:45:43] 10MediaWiki-Releasing, 10MediaWiki-Vendor, 05MW-1.44-release: Prune /vendor for REL1_44 - https://phabricator.wikimedia.org/T390792#10803992 (10Jdforrester-WMF) 05Open→03Resolved [12:06:38] (03PS1) 10Jforrester: Zuul: Drop StructuredDiscussions as a CI dependency on release branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143542 (https://phabricator.wikimedia.org/T393670) [12:07:48] (03CR) 10CI reject: [V:04-1] Zuul: Drop StructuredDiscussions as a CI dependency on release branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143542 (https://phabricator.wikimedia.org/T393670) (owner: 10Jforrester) [12:08:56] (03PS2) 10Jforrester: Zuul: Drop StructuredDiscussions as a CI dependency on release branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143542 (https://phabricator.wikimedia.org/T393670) [12:11:07] James_F: re T393670, unless this is a red herring or i'm missing something, it looks like there might be a composer dependency conflict on PHP >=8.2 that also relates to ext:WebAuthn(?) - I could reproduce the issue locally, and removing WebAuthn from the extensions folder resolved it [12:11:08] T393670: CheckUser REL1_44 CI blocked with 'Error: Class "Pimple\Container" not found' - https://phabricator.wikimedia.org/T393670 [12:11:18] apologies if this is already sorted out - thought i should say this just in case as i don't see it mentioned yet in the task [12:12:01] Dreamy_Jazz: was just doing a test patch on CU [12:12:01] A_smart_kitten: Meh, I'd be entirely unsurprised that WebAuthn is broken in regard to CI too, it's a bête noir for us. [12:13:07] A_smart_kitten: T363639 agrees with you; let's alter that one first. [12:13:08] T363639: web-auth/webauthn-lib must be upgraded to 4+ for PHP 8.2+ support - https://phabricator.wikimedia.org/T363639 [12:14:13] "Currently breaking CI for backports [...] Build for PHP7.4 and 8.1 succeeds". ah, deja vu :) [12:16:29] (03PS1) 10Jforrester: Zuul: Expand WebAuthn CI disable to all non-master/non-wmf branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143550 (https://phabricator.wikimedia.org/T363639) [12:16:53] * James_F now waits for CI's tox to tell him what his local tox won't. [12:17:42] (03CR) 10CI reject: [V:04-1] Zuul: Expand WebAuthn CI disable to all non-master/non-wmf branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143550 (https://phabricator.wikimedia.org/T363639) (owner: 10Jforrester) [12:17:55] (03PS2) 10Jforrester: Zuul: Expand WebAuthn CI disable to all non-master/non-wmf branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143550 (https://phabricator.wikimedia.org/T363639) [12:19:48] (03CR) 10Jforrester: [C:03+2] Zuul: Expand WebAuthn CI disable to all non-master/non-wmf branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143550 (https://phabricator.wikimedia.org/T363639) (owner: 10Jforrester) [12:21:02] (03Merged) 10jenkins-bot: Zuul: Expand WebAuthn CI disable to all non-master/non-wmf branches [integration/config] - 10https://gerrit.wikimedia.org/r/1143550 (https://phabricator.wikimedia.org/T363639) (owner: 10Jforrester) [12:21:29] Zuul: Expand WebAuthn CI disable to all non-master/non-wmf branches, for T363639, T390754, T393670 [12:21:29] T363639: web-auth/webauthn-lib must be upgraded to 4+ for PHP 8.2+ support - https://phabricator.wikimedia.org/T363639 [12:21:30] T390754: Extensions that pull in WebAuthn for their CI can support PHP 7.4 or PHP 8.2+, but not both (was: quibble-composer-mysql-php8[23] fail since 31/3) - https://phabricator.wikimedia.org/T390754 [12:21:30] T393670: CheckUser REL1_44 CI blocked with 'Error: Class "Pimple\Container" not found' - https://phabricator.wikimedia.org/T393670 [12:21:52] A_smart_kitten: Thanks! [12:22:18] James_F: no problem! [12:35:16] (side note James_F, just occurred to me, should the zuul message above have been preceded by ! log?) [12:38:18] 10Beta-Cluster-Infrastructure: Beta cluster IP block page should not point to noc@wikimedia.org - https://phabricator.wikimedia.org/T393404#10804191 (10Volans) We got another email to noc@ about this. Is there at least an address it can be forwarded to? [13:02:47] 10MediaWiki-Releasing, 06translatewiki.net, 10LPL Essential (LPL Essential 2025 Apr-Jun: CX), 05MW-1.44-release, and 2 others: Configure Translatewiki.net for REL1_44 translation backports - https://phabricator.wikimedia.org/T393514#10804273 (10abi_) 05Open→03In progress Merged and deployed the changes... [13:43:59] 06Release-Engineering-Team, 06serviceops, 07Essential-Work: train presync failed - https://phabricator.wikimedia.org/T387823#10804450 (10thcipriani) [13:44:29] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 07Essential-Work: Beta update job fails: The service mesh is unavailable, which can lead to unexpected results. - https://phabricator.wikimedia.org/T387480#10804451 (10thcipriani) [13:45:23] 10Release-Engineering-Team (Radar), 06serviceops, 06SRE, 07Essential-Work, 05Train Deployments: MW script "eval.php" failing for "testcommonswiki" during train operations - https://phabricator.wikimedia.org/T379628#10804452 (10thcipriani) [13:45:38] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 07Essential-Work: PHP on Beta cluster fails due to mismatching PCRE - https://phabricator.wikimedia.org/T387276#10804453 (10thcipriani) [13:46:06] 06Release-Engineering-Team, 10Castor, 10Wikidata, 10ci-test-error (WMF-deployed Build Failure), 07Essential-Work: Wikibase CI failing due to npm cache corruption issue - https://phabricator.wikimedia.org/T370748#10804455 (10thcipriani) [13:46:35] 10Gerrit, 06Release-Engineering-Team, 07Essential-Work: Gerrit not reachable over HTTPS - https://phabricator.wikimedia.org/T365041#10804458 (10thcipriani) [13:47:54] 10Phabricator, 06Release-Engineering-Team, 07Essential-Work: "Undefined offset: 5" error when visiting https://phabricator.wikimedia.org/T361395 - https://phabricator.wikimedia.org/T362689#10804460 (10thcipriani) [13:58:42] 06Release-Engineering-Team, 10Wikimedia-Logstash, 07Essential-Work, 07Wikimedia-production-error: MediaWiki normalized_message field has placeholders replaced since October 12th - https://phabricator.wikimedia.org/T349086#10804595 (10thcipriani) [14:00:53] 10Continuous-Integration-Infrastructure, 10Gerrit, 06Release-Engineering-Team, 07Zuul, and 2 others: CI marking all changes as -1 - https://phabricator.wikimedia.org/T340518#10804616 (10thcipriani) [14:04:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 07Zuul, 07Essential-Work: Zuul jenkins-bot user holding open SSH sessions - https://phabricator.wikimedia.org/T309376#10804649 (10thcipriani) [14:07:36] 06Release-Engineering-Team, 10MediaWiki-extensions-ReadingLists, 07Essential-Work, 10FY2024-25 KR 3.1 Content Discovery (Reading List MVP 3.1 (Intern Research)): ReadingLists items contain non-secure project URLs - https://phabricator.wikimedia.org/T389560#10804678 (10thcipriani) [14:08:11] 06Release-Engineering-Team, 07OKR-Work: [ci-charts] helm upgrading the mediawiki chart with (a) new ref(s) updates the environment - https://phabricator.wikimedia.org/T389425#10804682 (10thcipriani) [14:17:18] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Scap (SpiderPig 🕸️), 07OKR-Work: Dismiss interaction notification if someone responded to the interaction - https://phabricator.wikimedia.org/T393026#10804730 (10thcipriani) [14:17:34] 06Project-Admins, 10Release-Engineering-Team (Doing 😎), 06ShoutWiki, 07Essential-Work: Create project tag for calendar-Wikivoyage extension - https://phabricator.wikimedia.org/T393011#10804733 (10thcipriani) [14:18:21] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 07Essential-Work: Remove global default "closed_by_UNKNOWN" search query - https://phabricator.wikimedia.org/T393261#10804737 (10thcipriani) [14:18:52] 06Release-Engineering-Team, 06collaboration-services, 07Essential-Work: ProbeDown - https://phabricator.wikimedia.org/T393050#10804741 (10thcipriani) [14:23:06] 10GitLab (Infrastructure), 10Ceph, 06collaboration-services, 10Data-Persistence-Backup, and 2 others: Migrate gitlab storage to apus (also: backups from S3?) - https://phabricator.wikimedia.org/T378922#10804784 (10Jelto) Yesterday, @jcrespo, @MatthewVernon, and I discussed backups for object storage. The d... [14:37:57] 10Phabricator, 10Release-Engineering-Team (Doing 😎): The saved-for-everyone UBN Maniphest queries don't show In Progress tasks - https://phabricator.wikimedia.org/T393690#10804887 (10Aklapper) a:03Aklapper It's https://phabricator.wikimedia.org/maniphest/query/edit/ (probably for admins to show / edit "Globa... [14:39:27] 10Phabricator, 10Release-Engineering-Team (Doing 😎): The global UBN Maniphest queries don't show In Progress tasks - https://phabricator.wikimedia.org/T393690#10804891 (10Aklapper) 05Open→03Resolved p:05Triage→03Medium [14:43:13] 10Phabricator, 06Release-Engineering-Team: New Column Trigger: Assign only unassigned tasks - https://phabricator.wikimedia.org/T393464#10804898 (10Aklapper) p:05Triage→03Low a:03Aklapper [14:43:29] 10Phabricator (Upstream), 06Release-Engineering-Team, 07Upstream: New Column Trigger: Assign only unassigned tasks - https://phabricator.wikimedia.org/T393464#10804902 (10Aklapper) [14:44:14] 10Phabricator, 03Wikimedia-Hackathon-2025: Phorge (Phabricator) Code Sprint 2025 - https://phabricator.wikimedia.org/T392411#10804909 (10Aklapper) 05Open→03Resolved [14:45:08] 10Phabricator: Custom task form for #MW-Interfaces-Team - https://phabricator.wikimedia.org/T392598#10804910 (10Aklapper) @OWresch-WMF: Is a custom form still wanted? If not please set the task status to `declined`. If yes please see T392598#10794726. Thanks! [15:08:30] 10Phabricator, 10Release-Engineering-Team (Doing 😎): The global UBN Maniphest queries don't show In Progress tasks - https://phabricator.wikimedia.org/T393690#10805033 (10Jdforrester-WMF) Aha, thanks! [15:34:04] 10Beta-Cluster-Infrastructure: Beta cluster IP block page should not point to noc@wikimedia.org - https://phabricator.wikimedia.org/T393404#10805110 (10bd808) >>! In T393404#10804191, @Volans wrote: > We got another email to noc@ about this. Is there at least an address it can be forwarded to? For lack of anyth... [15:37:58] (03PS6) 10Jforrester: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:37:58] (03PS4) 10Jforrester: zuul: Remove uncontrolled recursion in extra deps in MW jobs (take 2) [integration/config] - 10https://gerrit.wikimedia.org/r/1132543 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [15:58:35] 06Release-Engineering-Team, 10Wikimedia-Logstash, 07Essential-Work, 07Wikimedia-production-error: MediaWiki normalized_message field has placeholders replaced since October 12th - https://phabricator.wikimedia.org/T349086#10805169 (10bd808) Thanks for the find and fix everyone. Connecting all the dots... [16:58:48] 10Continuous-Integration-Infrastructure, 07Developer Productivity: Provide recheck option for only failed jobs - https://phabricator.wikimedia.org/T392941#10805339 (10bd808) [17:52:16] (03open) 10dancy: Release 4.162.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/793 [17:54:25] (03merge) 10dancy: Release 4.162.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/793 [18:22:51] 10MediaWiki-Releasing, 10MediaWiki-Docker, 10MW-1.45-notes (1.45.0-wmf.1; 2025-05-13), 13Patch-For-Review: Remove docker-compose.yml from MediaWiki archive releases - https://phabricator.wikimedia.org/T393183#10805593 (10Douginamug) 05In progress→03Resolved This has been [[ https://gerrit.wikimedia... [19:05:30] 10GitLab (CI & Job Runners), 06Release-Engineering-Team: buildkit v0.21.1 released - https://phabricator.wikimedia.org/T393731 (10dancy) 03NEW [19:07:31] 10GitLab (CI & Job Runners), 06Release-Engineering-Team: buildkit v0.21.1 released - https://phabricator.wikimedia.org/T393731#10805719 (10dancy) p:05Triage→03Low a:03dancy [19:10:07] (03open) 10dancy: staging.tfvars: Bump buildkitd_tag to wmf-v0.21.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/469 (https://phabricator.wikimedia.org/T393731) [19:10:08] (03update) 10dancy: staging.tfvars: Bump buildkitd_tag to wmf-v0.21.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/469 (https://phabricator.wikimedia.org/T393731) [19:11:12] (03merge) 10dancy: staging.tfvars: Bump buildkitd_tag to wmf-v0.21.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/469 (https://phabricator.wikimedia.org/T393731) [19:20:05] (03update) 10brennen: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [19:53:21] (03open) 10dancy: .gitignore: Add more junk [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/97 [19:53:23] (03update) 10dancy: .gitignore: Add more junk [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/97 [19:54:04] (03open) 10dancy: Add support for application/vnd.oci.image.manifest.v1+json [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/98 (https://phabricator.wikimedia.org/T393731) [19:54:05] (03update) 10dancy: Add support for application/vnd.oci.image.manifest.v1+json [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/98 (https://phabricator.wikimedia.org/T393731) [19:54:57] (03merge) 10dancy: .gitignore: Add more junk [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/97 [19:55:15] (03update) 10dancy: Add support for application/vnd.oci.image.manifest.v1+json [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/98 (https://phabricator.wikimedia.org/T393731) [19:56:25] (03merge) 10dancy: Add support for application/vnd.oci.image.manifest.v1+json [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/98 (https://phabricator.wikimedia.org/T393731) [20:05:04] (03open) 10dancy: Chart.yaml 1.6.2: Bump addVersion to v1.15.0 [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/99 (https://phabricator.wikimedia.org/T393731) [20:05:05] (03update) 10dancy: Chart.yaml 1.6.2: Bump addVersion to v1.15.0 [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/99 (https://phabricator.wikimedia.org/T393731) [20:05:58] (03merge) 10dancy: Chart.yaml 1.6.2: Bump addVersion to v1.15.0 [repos/releng/reggie] - 10https://gitlab.wikimedia.org/repos/releng/reggie/-/merge_requests/99 (https://phabricator.wikimedia.org/T393731) [20:22:01] (03open) 10dancy: prod.tfvars: Bump buildkitd_tag to wmf-v0.21.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/470 [20:22:03] (03update) 10dancy: prod.tfvars: Bump buildkitd_tag to wmf-v0.21.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/470 [20:22:55] (03merge) 10dancy: prod.tfvars: Bump buildkitd_tag to wmf-v0.21.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/470 [20:28:40] !log Updating buildkitd to v0.21.1 in gitlab-cloud-runners [20:28:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:31:44] 10GitLab (CI & Job Runners), 06Release-Engineering-Team: buildkit v0.21.1 released - https://phabricator.wikimedia.org/T393731#10805931 (10dancy) [20:58:35] (03approved) 10bd808: Backport.vue: Remove unnecessary {{ item.text }} [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/788 (https://phabricator.wikimedia.org/T392508) (owner: 10dancy) [21:19:27] (03open) 10matmarex: Reduce bot comments and hide some bot actions [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/90 (https://phabricator.wikimedia.org/T393289) [21:23:53] 10Phabricator, 13Patch-For-Review, 03Wikimedia-Hackathon-2025: Deploy user style to reduce bot comments on Phabricator - https://phabricator.wikimedia.org/T393289#10805999 (10matmarex) [22:09:31] FIRING: ProbeDown: Service gerrit1003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit1003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [22:09:38] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T393743 (10phaultfinder) 03NEW [22:14:31] RESOLVED: ProbeDown: Service gerrit1003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit1003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [22:46:49] (03update) 10dancy: Backport.vue: Remove unnecessary {{ item.text }} [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/788 (https://phabricator.wikimedia.org/T392508) [22:48:53] (03merge) 10dancy: Backport.vue: Remove unnecessary {{ item.text }} [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/788 (https://phabricator.wikimedia.org/T392508) [22:53:54] 10Beta-Cluster-Infrastructure, 06Traffic: Add allowlist to make poking holes in abuse_networks:blocked_nets:networks easier - https://phabricator.wikimedia.org/T393481#10806169 (10BCornwall)