[06:09:44] (03PS1) 10Arnaudb: gerrit: add readonly plugin [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1165528 (https://phabricator.wikimedia.org/T387833) [06:12:16] (03CR) 10Arnaudb: "followed wikitech instructions to prep plugin installation via CI, let me know if anything else is required. I needed to edit the repo's ." [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1165528 (https://phabricator.wikimedia.org/T387833) (owner: 10Arnaudb) [07:01:09] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397 (10Aklapper) 03NEW [07:19:04] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10966221 (10Aklapper) [07:20:44] (03CR) 10Hashar: [C:04-1] "None of our other plugins (ex: go-import, lfs, zuul) are mentioned in `.gitignore`. If I try it I get:" [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1165528 (https://phabricator.wikimedia.org/T387833) (owner: 10Arnaudb) [07:38:43] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10966246 (10Aklapper) [07:42:11] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Performance Issue: Drop a bunch of Maniphest Custom Fields from fulltext indexing - https://phabricator.wikimedia.org/T398399 (10Aklapper) 03NEW [07:42:40] (03open) 10phuedx: Set xLab deployment window length to 30 minutes [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/187 (https://phabricator.wikimedia.org/T395025) [07:43:10] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Performance Issue: Drop a bunch of Maniphest Custom Fields from fulltext indexing - https://phabricator.wikimedia.org/T398399#10966263 (10Aklapper) [07:43:10] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10966264 (10Aklapper) [07:43:12] 10Phabricator, 10Release-Engineering-Team (Doing 😎): Exception in "./bin/search index --force": "strlen() expects parameter 1 to be string, array given" - https://phabricator.wikimedia.org/T398197#10966265 (10Aklapper) [07:45:15] (03update) 10phuedx: Add "xlab" window to deployments calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/183 (https://phabricator.wikimedia.org/T395025) [07:47:49] !log gerrit: ssh -p 29418 gerrit.wikimedia.org rename-project operations/debs/wmf-sre-laptop operations/debs/wmf-laptop # T365985 [07:47:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:47:52] T365985: Rename wmf-sre-laptop repository to wmf-laptop - https://phabricator.wikimedia.org/T365985 [07:50:55] (03approved) 10jnuche: Set xLab deployment window length to 30 minutes [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/187 (https://phabricator.wikimedia.org/T395025) (owner: 10phuedx) [07:50:59] (03merge) 10jnuche: Set xLab deployment window length to 30 minutes [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/187 (https://phabricator.wikimedia.org/T395025) (owner: 10phuedx) [07:54:01] 10Gerrit: Rename wmf-sre-laptop repository to wmf-laptop - https://phabricator.wikimedia.org/T365985#10966280 (10hashar) 05Openβ†’03Resolved a:03hashar ` $ ssh -p 29418 gerrit.wikimedia.org rename-project operations/debs/wmf-sre-laptop operations/debs/wmf-laptop Checking preconditions:... [07:55:14] (03Abandoned) 10Arnaudb: gerrit: add readonly plugin [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1165528 (https://phabricator.wikimedia.org/T387833) (owner: 10Arnaudb) [08:00:35] (03Restored) 10Arnaudb: gerrit: add readonly plugin [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1165528 (https://phabricator.wikimedia.org/T387833) (owner: 10Arnaudb) [08:04:42] (03PS2) 10Arnaudb: gerrit: add readonly plugin [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1165528 (https://phabricator.wikimedia.org/T387833) [08:10:30] 10Gerrit: Rename of a repository is not replicated - https://phabricator.wikimedia.org/T398401 (10hashar) 03NEW [08:14:30] 10Gerrit, 10Release-Engineering-Team (Seen): Support renaming repositories in Gerrit - https://phabricator.wikimedia.org/T239693#10966328 (10hashar) [08:14:32] 10Gerrit: Rename of a repository is not replicated - https://phabricator.wikimedia.org/T398401#10966329 (10hashar) [08:15:18] 10Gerrit, 10Release-Engineering-Team (Seen): Support renaming repositories in Gerrit - https://phabricator.wikimedia.org/T239693#10966331 (10hashar) When renaming operations/debs/wmf-sre-laptop to operations/debs/wmf-laptop (T365985), I have noticed the renaming is not replicated. Filed as a follow up task... [08:18:52] (03PS2) 10Arnaudb: gerrit: add readonly plugin [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1165528 (https://phabricator.wikimedia.org/T387833) [09:02:05] 10Gerrit, 06collaboration-services: Rename of a repository is not replicated - https://phabricator.wikimedia.org/T398401#10966529 (10hashar) The `rename-project` plugin needs some further configuration. An overview is: https://gerrit.wikimedia.org/r/plugins/rename-project/Documentation/index.html#replication-... [09:10:15] 10Gerrit, 06collaboration-services: Rename of a repository is not replicated - https://phabricator.wikimedia.org/T398401#10966583 (10hashar) Easiest: we can allow port 29418 when the source is the active host (gerrit1003). [09:40:30] (03PS1) 10Jforrester: parameter_functions: Drop old doc comments about dropped branches [integration/config] - 10https://gerrit.wikimedia.org/r/1165834 [09:51:21] (03CR) 10Jforrester: [C:03+2] parameter_functions: Drop old doc comments about dropped branches [integration/config] - 10https://gerrit.wikimedia.org/r/1165834 (owner: 10Jforrester) [09:52:33] (03Merged) 10jenkins-bot: parameter_functions: Drop old doc comments about dropped branches [integration/config] - 10https://gerrit.wikimedia.org/r/1165834 (owner: 10Jforrester) [09:54:01] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Rename of a repository is not replicated - https://phabricator.wikimedia.org/T398401#10966753 (10hashar) a:03hashar [10:06:01] 10Gerrit: PHP libraries as Gerrit top-level projects - https://phabricator.wikimedia.org/T125031#10966824 (10hashar) 05Stalledβ†’03Open I would like the MediaWiki libraries to be moved under `mediawiki/libs`. The new Zuul can match project names using regex which would greatly simplify the configuration. We co... [10:06:34] 10Gerrit, 06Release-Engineering-Team: Write a runbook to rename repositories in Gerrit - https://phabricator.wikimedia.org/T398320#10966830 (10hashar) [11:51:39] (03PS3) 10Hashar: jjb: publish-to-prometheus: skip on lack of .prom files [integration/config] - 10https://gerrit.wikimedia.org/r/1165568 (https://phabricator.wikimedia.org/T392894) [12:03:37] 10GitLab (Infrastructure), 10Cloud-VPS, 06collaboration-services, 10cloud-services-team (FY2024/2025-Q3-Q4): Volume is stuck to deleted instance in devtools project - https://phabricator.wikimedia.org/T396739#10967353 (10Jelto) 05Openβ†’03Resolved Yes! I delete the unused volume. So we can resolve th... [12:40:57] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10967541 (10Aklapper) [12:49:52] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T392178#10967576 (10Daimona) [12:55:55] (03PS4) 10Hashar: jjb: publish-to-prometheus: skip on lack of .prom files [integration/config] - 10https://gerrit.wikimedia.org/r/1165568 (https://phabricator.wikimedia.org/T392894) [12:57:03] (03PS1) 10Hashar: jjb: update operations-puppet-doc to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165873 (https://phabricator.wikimedia.org/T335765) [12:58:08] Why do I feel like packagist isn't updating once again? https://packagist.org/packages/mediawiki/phan-taint-check-plugin [12:58:50] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10967610 (10Aklapper) [12:59:08] (03CR) 10Hashar: [C:03+2] jjb: update operations-puppet-doc to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165873 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [12:59:16] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10967613 (10Aklapper) [12:59:19] James_F ^ [12:59:36] Tag exists on github, but packagist didn't pick it up [13:00:27] (03Merged) 10jenkins-bot: jjb: update operations-puppet-doc to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165873 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [13:00:57] (03PS4) 10Hashar: jjb: allow missing source when fetching from instances [integration/config] - 10https://gerrit.wikimedia.org/r/1165553 (https://phabricator.wikimedia.org/T392894) [13:01:48] (03CR) 10Hashar: [C:03+2] "I have tested it it with https://integration.wikimedia.org/ci/job/operations-puppet-doc/ which I fixed with Id25ebf3e38edc5bcc4e082bfb55a4" [integration/config] - 10https://gerrit.wikimedia.org/r/1165553 (https://phabricator.wikimedia.org/T392894) (owner: 10Hashar) [13:01:54] (03CR) 10Hashar: [C:03+2] jjb: publish-to-prometheus: skip on lack of .prom files [integration/config] - 10https://gerrit.wikimedia.org/r/1165568 (https://phabricator.wikimedia.org/T392894) (owner: 10Hashar) [13:02:24] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10967622 (10Aklapper) [13:02:37] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10967623 (10Aklapper) [13:03:04] (03Merged) 10jenkins-bot: jjb: allow missing source when fetching from instances [integration/config] - 10https://gerrit.wikimedia.org/r/1165553 (https://phabricator.wikimedia.org/T392894) (owner: 10Hashar) [13:03:06] (03Merged) 10jenkins-bot: jjb: publish-to-prometheus: skip on lack of .prom files [integration/config] - 10https://gerrit.wikimedia.org/r/1165568 (https://phabricator.wikimedia.org/T392894) (owner: 10Hashar) [13:04:37] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10967628 (10Aklapper) [13:12:30] 06Release-Engineering-Team, 10Data-Engineering-Roadmap, 10Dumps-Generation, 10MW-on-K8s, and 3 others: WE 5.4 KR - Hypothesis 5.4.4 - Q3 FY24/25 - Migrate current-generation dumps to run on kubernetes - https://phabricator.wikimedia.org/T352650#10967678 (10BTullis) 05Openβ†’03Resolved a:03BTullis I... [13:18:21] 10Continuous-Integration-Config, 06Test-Platform, 07ci-test-error, 13Patch-For-Review: publish-to-prometheus (...) completed. Result was FAILURE - https://phabricator.wikimedia.org/T398284#10967712 (10hashar) 05Openβ†’03Resolved Done and I validated the last changes using https://integration.wikimedi... [13:19:37] 10Continuous-Integration-Config, 06Test-Platform, 07ci-test-error, 13Patch-For-Review: publish-to-prometheus (...) completed. Result was FAILURE - https://phabricator.wikimedia.org/T398284#10967722 (10hashar) [13:20:35] Daimona: Yeah, will need to ping in security for someone to nudge it, yet again. :-( [13:21:20] A big one of the useful things that we were going to get "for free" by moving from gerrit to GitLab was proper packagist hook triggering. [13:21:30] * hashar whistles [13:21:47] I clicked the button. [13:22:08] I am pretty sure we can `curl --post` to packagist to trigger it [13:22:35] taavi: Thanks! [13:22:38] (an alternative I was wondering of is whether we could set up our own packagist system/composer host) [13:22:52] hashar: I think you need to be authed as a packagist owner, which we have I think 3 humans for. [13:23:04] well that is solveable isn't it? [13:23:20] Thanks! [13:23:24] yes [13:23:25] I mean [13:23:46] I mean that when a tag has been pushed in Gerrit, our CI could `curl --post` to packagist to trigger the refresh [13:23:52] via a post merge job [13:23:59] er publish job [13:24:10] well regardless, that should be done in the new Zuul [13:24:16] Oh, sure, but I think it only works for GitHub- or GitLab-hosted repos. [13:24:20] Which is sad. [13:43:46] Daimona: Thoughts on https://gerrit.wikimedia.org/r/c/mediawiki/tools/phan/+/1127597 (i.e., setting it so MW core allows aliases and phan by default doesn't, so people fix their code as they go)? [13:45:00] Let me see what I wrote last time, so I can disagree with my past self just for the sake of it [13:49:32] 10Continuous-Integration-Config: Create a non-voting CI job to flag deprecated code - https://phabricator.wikimedia.org/T388948#10967892 (10Daimona) >>! In T388948#10769160, @Michael wrote: >>>! In T388948#10638982, @Jdforrester-WMF wrote: >> Loathe as I am to add Yet Another CI Job at a time when CI is massivel... [13:50:37] So, I still think a separate job would be preferable. If we make that change, renaming a class in core becomes a breaking change for extensions. I suppose that might be too aggressive, at times. [13:50:51] I guess ultimately it boils down to: how hard would it be to add a new job to check for deprecations / aliases? [13:51:42] If we piggyback on the existing phan config, I think most of the effort would be in CI configuration [13:52:06] Presumably, the job itself would just read the normal phan config, apply a couple overrides, and run phan like the normal phan job. [13:53:22] The phan part is doable in 15 minutes and I can write that. I don't know about the CI side of things though. [13:54:02] Daimona: Adding more CI jobs is pretty easy, it's more a resource/load issue, for which I'd defer to hashar. [13:55:31] I see [13:55:41] (03PS1) 10Hashar: jjb: upgrade ooui ruby 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165898 [13:56:46] (03CR) 10CI reject: [V:04-1] jjb: upgrade ooui ruby 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165898 (owner: 10Hashar) [13:57:20] (03PS2) 10Hashar: Upgrade ooui ruby from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165898 (https://phabricator.wikimedia.org/T335765) [13:59:22] mediawiki/vagrant is still haunting us [14:00:23] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure (Zuul upgrade): Reduce the number of CI images - https://phabricator.wikimedia.org/T397429#10967962 (10hashar) We also still have a bunch of Buster based images: T335765 [14:02:38] James_F: may you review my commit message "Upgrade ooui ruby from 2.5 to 2.7" on https://gerrit.wikimedia.org/r/c/integration/config/+/1165898 :] [14:03:05] hashar: Ha. [14:03:05] or I guess I split it in a change that add the job and another one that does the Zuul config layout change [14:03:14] but I feel a single change is easier (to me :b ) [14:06:47] (03PS1) 10Hashar: Upgrade translatewiki ruby from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165902 [14:08:08] (03PS2) 10Hashar: Upgrade translatewiki ruby from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165902 (https://phabricator.wikimedia.org/T335765) [14:08:52] and I think I will drop mediawiki/vagrant [14:11:15] (03PS3) 10Jforrester: Zuul, jjb: Upgrade ooui-ruby* from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165898 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [14:11:28] (03CR) 10Jforrester: [C:03+2] Zuul, jjb: Upgrade ooui-ruby* from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165898 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [14:11:32] AH [14:12:17] (03PS3) 10Hashar: Zuul, jjb: Upgrade translatewiki-ruby* from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165902 (https://phabricator.wikimedia.org/T335765) [14:12:47] (03Merged) 10jenkins-bot: Zuul, jjb: Upgrade ooui-ruby* from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165898 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [14:13:44] !log Zuul: Upgrade ooui-ruby* from 2.5 to 2.7, for T335765 [14:13:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:13:49] T335765: Migrate all CI jobs from buster to bullseye or later and drop buster testing support - https://phabricator.wikimedia.org/T335765 [14:23:26] 10Continuous-Integration-Config: Create a non-voting CI job to flag deprecated code - https://phabricator.wikimedia.org/T388948#10968090 (10Daimona) The config overrides would be something like this: `lang=php,name=new-config.php ^ That's basically all that's needed for phan [14:24:54] 10Continuous-Integration-Config: Create a non-voting CI job to flag deprecated code - https://phabricator.wikimedia.org/T388948#10968091 (10Daimona) >>! In T388948#10968090, @Daimona wrote: > The config overrides would be something like this: (Context: this is being discussed in IRC, `#wikimedia-releng`. The id... [14:35:56] 10Beta-Cluster-Infrastructure: Error: 503, Backend fetch failed - https://phabricator.wikimedia.org/T398450 (10dom_walden) 03NEW [14:36:15] (03update) 10dancy: spiderpig: Don't start logstash poller if logstash_host is blank [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/874 [14:36:19] (03update) 10dancy: spiderpig: Don't start logstash poller if logstash_host is blank [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/874 [14:43:50] (03merge) 10dancy: spiderpig: Don't start logstash poller if logstash_host is blank [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/874 [14:51:00] (03CR) 10Hashar: [C:03+2] Zuul, jjb: Upgrade translatewiki-ruby* from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165902 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [14:51:01] !log Zuul: Upgrade translatewiki-ruby* from 2.5 to 2.7, for T335765 [14:51:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:51:03] T335765: Migrate all CI jobs from buster to bullseye or later and drop buster testing support - https://phabricator.wikimedia.org/T335765 [14:52:19] (03Merged) 10jenkins-bot: Zuul, jjb: Upgrade translatewiki-ruby* from 2.5 to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165902 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [14:53:06] 10Beta-Cluster-Infrastructure: Error: 503, Backend fetch failed - https://phabricator.wikimedia.org/T398450#10968201 (10Aklapper) Might be the same as T398448 ? [14:57:41] (03open) 10dancy: Release 4.185.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/875 [14:59:53] (03merge) 10dancy: Release 4.185.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/875 [15:01:28] (03PS1) 10Hashar: Zuul, jjb: Upgrade puppet-wmf-styleguide-ruby* to ruby 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165915 (https://phabricator.wikimedia.org/T335765) [15:02:22] (03PS2) 10Hashar: Zuul, jjb: Upgrade puppet-wmf-styleguide-ruby* to ruby 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165915 (https://phabricator.wikimedia.org/T335765) [15:02:34] (03CR) 10Hashar: [C:03+2] Zuul, jjb: Upgrade puppet-wmf-styleguide-ruby* to ruby 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165915 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [15:02:42] no [15:02:45] jsduck [15:03:44] (03Merged) 10jenkins-bot: Zuul, jjb: Upgrade puppet-wmf-styleguide-ruby* to ruby 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1165915 (https://phabricator.wikimedia.org/T335765) (owner: 10Hashar) [15:06:53] (03PS1) 10Hashar: Zuul: [operations/debs/ruby-jsduck] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1165917 [15:07:45] (03CR) 10Hashar: [C:03+2] Zuul: [operations/debs/ruby-jsduck] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1165917 (owner: 10Hashar) [15:11:11] (03Merged) 10jenkins-bot: Zuul: [operations/debs/ruby-jsduck] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1165917 (owner: 10Hashar) [15:11:21] (03PS1) 10Hashar: tests: remove obsolete ref to Wikibase using ruby [integration/config] - 10https://gerrit.wikimedia.org/r/1165918 [15:13:40] 10Beta-Cluster-Infrastructure: Error: 503, Backend fetch failed - https://phabricator.wikimedia.org/T398450#10968281 (10dancy) Seems to be resolved now. [15:24:03] Hey releng, just curious if Blubber can build multi-arch images on the trusted runners? based on https://doc.wikimedia.org/releng/blubber/#building-for-multiple-platforms guessing the answer is "yes"? [15:28:29] 10Phabricator, 10Server-side-upload-request: Consider adding a global Herald rule to comment on #Server-side-upload-request tickets - https://phabricator.wikimedia.org/T397024#10968336 (10Aklapper) >>! In T397024#10965034, @A_smart_kitten wrote: > (Also, not to badger about it //too// much, but I would be... [15:31:53] 10Beta-Cluster-Infrastructure: End user reported "Error: 503, Backend fetch failed" contacting an unspecified Beta Cluster hosted wiki - https://phabricator.wikimedia.org/T398450#10968355 (10bd808) [15:36:04] inflatador: It appears so but I haven't personally tested doing that so I can't vouch for it. Give it a try and let us know if you have any trouble! [15:42:32] 10Phabricator: Phab Scap template vs actual value out of sync for maniphest.custom-field-definitions - https://phabricator.wikimedia.org/T398460 (10Aklapper) 03NEW [15:42:42] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Phab Scap template vs actual value out of sync for maniphest.custom-field-definitions - https://phabricator.wikimedia.org/T398460#10968416 (10Aklapper) [15:46:59] 10Beta-Cluster-Infrastructure: End user reported "Error: 503, Backend fetch failed" contacting an unspecified Beta Cluster hosted wiki - https://phabricator.wikimedia.org/T398450#10968440 (10bd808) https://meta.wikimedia.beta.wmflabs.org/wiki/Main_Page and https://simple.wikipedia.beta.wmflabs.org/wiki/Main_Page... [15:56:29] 10Beta-Cluster-Infrastructure: End user reported "Error: 503, Backend fetch failed" contacting an unspecified Beta Cluster hosted wiki - https://phabricator.wikimedia.org/T398450#10968514 (10bd808) `lang=shell-session root@deployment-cache-text08:/var/log/trafficserver# grep '20250702' error.log | cut -c 10-12 |... [15:58:07] 10Beta-Cluster-Infrastructure: End user reported "Error: 503, Backend fetch failed" contacting an unspecified Beta Cluster hosted wiki - https://phabricator.wikimedia.org/T398450#10968530 (10bd808) [15:58:12] 10Beta-Cluster-Infrastructure, 07Epic, 13Patch-For-Review: 2025 tracking task for Beta Cluster (deployment-prep) traffic overload protection (blocking unwanted crawlers) - https://phabricator.wikimedia.org/T393487#10968531 (10bd808) [16:00:57] 10Beta-Cluster-Infrastructure: End user reported "Error: 503, Backend fetch failed" contacting an unspecified Beta Cluster hosted wiki - https://phabricator.wikimedia.org/T398450#10968549 (10bd808) 05Openβ†’03Declined I don't think there is much we can do at this point other than being aware this happened. [16:26:20] 10Beta-Cluster-Infrastructure: No Puppet resources found on instance deployment-puppetserver-1 on project deployment-prep - https://phabricator.wikimedia.org/T397879#10968668 (10bd808) 05Openβ†’03Invalid Fixed itself apparently `lang=shell-session bd808@deployment-puppetserver-1:~$ sudo -i puppet agent -tv... [16:27:26] 10Beta-Cluster-Infrastructure: No Puppet resources found on instance deployment-puppetdb05 on project deployment-prep - https://phabricator.wikimedia.org/T397012#10968683 (10bd808) 05Openβ†’03Resolved [16:35:03] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Remove remnants of Phatality ID in Phabricator - https://phabricator.wikimedia.org/T398397#10968703 (10Aklapper) [16:35:40] 10Beta-Cluster-Infrastructure, 10Testing Support, 07Epic: Run Selenium tests targeting Beta cluster - https://phabricator.wikimedia.org/T373680#10968717 (10bd808) [16:43:05] i am trying and failing to find the code within https://gitlab.wikimedia.org/repos/phabricator/phabricator that defines the `policy.create` conduit endpoint at https://phabricator.wikimedia.org/conduit/method/policy.create/ [16:43:19] if anyone has any idea of where to look, i would be very grateful :) [16:45:06] andre & brennen would know about Phab stuff I guess :) [16:45:30] (03CR) 10Hashar: [C:03+2] tests: remove obsolete ref to Wikibase using ruby [integration/config] - 10https://gerrit.wikimedia.org/r/1165918 (owner: 10Hashar) [16:46:40] A_smart_kitten: /misc/src/conduit/CreatePolicyConduitAPIMethod.php in phabricator/extensions/ is my guess [16:46:44] (03Merged) 10jenkins-bot: tests: remove obsolete ref to Wikibase using ruby [integration/config] - 10https://gerrit.wikimedia.org/r/1165918 (owner: 10Hashar) [16:48:09] andre: ahh it's wikimedia-specific? that would explain why i couldn't find it on upstream we.phorge.it [16:48:28] anyhow thank you for pointing me towards the correct repo! [16:50:31] (03PS1) 10Hashar: jjb: drop unused {name}-ruby2.5-bundle-yard-publish [integration/config] - 10https://gerrit.wikimedia.org/r/1165944 [16:55:22] (03open) 10dancy: spiderpig/api.py: Add session cookie debugging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/876 (https://phabricator.wikimedia.org/T397457) [16:55:24] (03update) 10dancy: spiderpig/api.py: Add session cookie debugging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/876 (https://phabricator.wikimedia.org/T397457) [16:55:54] (03update) 10dancy: spiderpig/api.py: Add session cookie debugging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/876 (https://phabricator.wikimedia.org/T397457) [16:56:44] (03PS1) 10Hashar: Drop generic ruby rake jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1165947 [16:56:59] (03CR) 10Hashar: [C:03+2] jjb: drop unused {name}-ruby2.5-bundle-yard-publish [integration/config] - 10https://gerrit.wikimedia.org/r/1165944 (owner: 10Hashar) [16:58:11] (03merge) 10dancy: spiderpig/api.py: Add session cookie debugging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/876 (https://phabricator.wikimedia.org/T397457) [16:58:12] (03Merged) 10jenkins-bot: jjb: drop unused {name}-ruby2.5-bundle-yard-publish [integration/config] - 10https://gerrit.wikimedia.org/r/1165944 (owner: 10Hashar) [17:00:00] 10Beta-Cluster-Infrastructure, 06Commons: Image uploads appear to be broken on the beta cluster specifically - https://phabricator.wikimedia.org/T365116#10968850 (10bd808) deployment-imagescaler03 has been replaced with deployment-imagescaler04, but thumbor things still seem to be unhappy. https://meta.wikimed... [17:00:43] 10Beta-Cluster-Infrastructure, 06Commons: Image thumbnail generation not working in Beta Cluster - https://phabricator.wikimedia.org/T365116#10968864 (10bd808) [17:15:57] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure (Zuul upgrade), 10MediaWiki-Vagrant: Remove mediawiki/vagrant from CI - https://phabricator.wikimedia.org/T398469 (10hashar) 03NEW [17:16:43] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure (Zuul upgrade), 10MediaWiki-Vagrant: Remove mediawiki/vagrant from CI - https://phabricator.wikimedia.org/T398469#10968925 (10hashar) [17:16:44] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from buster to bullseye or later and drop buster testing support - https://phabricator.wikimedia.org/T335765#10968923 (10hashar) [17:26:15] (03PS1) 10Hashar: Remove mediawiki/vagrant and ruby2.5 support [integration/config] - 10https://gerrit.wikimedia.org/r/1165954 (https://phabricator.wikimedia.org/T398469) [17:27:41] (03CR) 10CI reject: [V:04-1] Remove mediawiki/vagrant and ruby2.5 support [integration/config] - 10https://gerrit.wikimedia.org/r/1165954 (https://phabricator.wikimedia.org/T398469) (owner: 10Hashar) [17:29:03] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure (Zuul upgrade), 10MediaWiki-Vagrant, 13Patch-For-Review: Remove mediawiki/vagrant from CI - https://phabricator.wikimedia.org/T398469#10969005 (10hashar) I know @tgr still uses Vagrant. This task is to remove it from Zuul/Jenkins and... [17:29:49] (03PS2) 10Hashar: Zuul, jjb: Drop generic ruby rake jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1165947 [17:30:14] (03CR) 10Hashar: [C:03+2] Zuul, jjb: Drop generic ruby rake jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1165947 (owner: 10Hashar) [17:31:18] 10Release-Engineering-Team (Doing 😎), 10Phatality, 13Patch-For-Review: Query Maniphest Advanced Search instead of Phabricator Primary Global Search - https://phabricator.wikimedia.org/T398305#10969016 (10colewhite) Ack. I'll hold off rolling it to production to give you an opportunity to debug. [17:31:28] (03Merged) 10jenkins-bot: Zuul, jjb: Drop generic ruby rake jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1165947 (owner: 10Hashar) [17:33:54] !log Reloaded Zuul for "Drop generic ruby rake jobs" https://gerrit.wikimedia.org/r/c/integration/config/+/1165947/ [17:33:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:36:31] 10GitLab (Pipeline Services Migration🐀), 06collaboration-services, 13Patch-For-Review: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab - https://phabricator.wikimedia.org/T300171#10969071 (10Dzahn) [17:37:17] 10GitLab (Pipeline Services Migration🐀), 06collaboration-services, 13Patch-For-Review: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab - https://phabricator.wikimedia.org/T300171#10969073 (10Dzahn) legacy VMs are shut down. DNS discovery records deleted. confirmed with Arnold os-re... [17:37:20] 10GitLab (Pipeline Services Migration🐀), 06collaboration-services, 13Patch-For-Review: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab - https://phabricator.wikimedia.org/T300171#10969074 (10Dzahn) 05Openβ†’03Resolved [17:46:28] (03PS2) 10Hashar: Remove mediawiki/vagrant and ruby2.5 support [integration/config] - 10https://gerrit.wikimedia.org/r/1165954 (https://phabricator.wikimedia.org/T398469) [18:53:01] 10Continuous-Integration-Config, 07PHP 8.3 support: Move CI images to WMF PHP 8.3 packages - https://phabricator.wikimedia.org/T398491 (10Reedy) 03NEW [18:53:04] 10Continuous-Integration-Config, 07PHP 8.3 support: Move CI images to WMF PHP 8.3 packages - https://phabricator.wikimedia.org/T398491#10969414 (10Reedy) [19:06:08] 10GitLab, 06collaboration-services, 06serviceops, 07Kubernetes: Trusted gitlab runner containers need access to staging k8s cluster - https://phabricator.wikimedia.org/T325385#10969469 (10Dzahn) a:05Dzahnβ†’03None [19:12:36] 10Phabricator, 06Release-Engineering-Team: Temporarily replace the Phabricator logo for Pride Month (2025) - https://phabricator.wikimedia.org/T395806#10969495 (10brennen) 05In progressβ†’03Resolved > I for one would be happy to keep the pride logo year-round, FWIW (like e.g. /r/ProgrammingLanguages does... [19:55:40] 10Continuous-Integration-Config, 07PHP 8.3 support: Move CI images to WMF PHP 8.3 packages - https://phabricator.wikimedia.org/T398491#10969625 (10Reedy) [20:02:30] (03PS1) 10Reedy: dockerfiles: Use WMF SRE provided PHP 8.3 packages [integration/config] - 10https://gerrit.wikimedia.org/r/1165978 (https://phabricator.wikimedia.org/T398491) [20:03:07] 10Continuous-Integration-Config, 13Patch-For-Review, 07PHP 8.3 support: Move CI images to WMF PHP 8.3 packages - https://phabricator.wikimedia.org/T398491#10969643 (10Reedy) p:05Triageβ†’03Low [20:38:53] 10Beta-Cluster-Infrastructure, 06Commons, 10Thumbor: Image thumbnail generation not working in Beta Cluster - https://phabricator.wikimedia.org/T365116#10969826 (10bd808) If nothing else it looks like the deployed version of thumbor is very stale. Prod looks to have `2025-06-03-122529-production` in the helm... [20:43:02] 10Beta-Cluster-Infrastructure, 10Thumbor: Image thumbnail generation not working in Beta Cluster - https://phabricator.wikimedia.org/T365116#10969860 (10Peachey88) [21:00:31] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team: Replace deployment-imagescaler03 (stretch) with deployment-imagescaler04 (buster) - https://phabricator.wikimedia.org/T294148#10969928 (10bd808) 05Openβ†’03Resolved a:03taavi deployment-imagescaler04 exists since 2024-05-04 when @taavi built it... [21:03:14] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure (Zuul upgrade), 10MediaWiki-Vagrant, 13Patch-For-Review: Remove mediawiki/vagrant from CI - https://phabricator.wikimedia.org/T398469#10969940 (10Tgr) Vagrant actually uses ruby 2.7, which it seems integration/config is still supporting. [21:05:38] 10Beta-Cluster-Infrastructure, 06serviceops, 10Thumbor, 07Beta-Cluster-reproducible: deployment-prep needs a Thumbor instance - https://phabricator.wikimedia.org/T344605#10969961 (10bd808) 05Openβ†’03Resolved a:03taavi Thumbor is running in a docker container on deployment-imagescaler04. There appe... [21:08:03] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure (Zuul upgrade), 10MediaWiki-Vagrant, 13Patch-For-Review: Remove mediawiki/vagrant from CI - https://phabricator.wikimedia.org/T398469#10969967 (10Tgr) It's also on bullseye, not buster like the CI Vagrant image. I suppose that doesn't... [21:10:27] 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible: upload.wikimedia.beta.wmflabs.org: cannot find server - https://phabricator.wikimedia.org/T359768#10970007 (10bd808) 05Openβ†’03Invalid The reported https://upload.wikimedia.beta.wmflabs.org/wikipedia/en/1/13/Bert_Self-portrait2.png URL works, a... [21:11:06] 10Beta-Cluster-Infrastructure, 10Thumbor: Image thumbnail generation not working in Beta Cluster - https://phabricator.wikimedia.org/T365116#10970018 (10bd808) p:05Triageβ†’03High [21:16:53] 10Beta-Cluster-Infrastructure: Investigate setting up HTTPS directly on beta appservers - https://phabricator.wikimedia.org/T206158#10970046 (10bd808) 05Openβ†’03Declined Everything about serving MediaWiki in Beta Cluster needs love, but this particular task is unlikely to progress further. A more plausibl... [21:38:26] !log T289318 - Change profile::mail::mx::verp_bounce_post_url in Horizon prefix puppet, from https://meta.wikimedia.beta.wmflabs.org/w/api.php to https://meta.wikimedia.beta.wmcloud.org/w/api.php. [21:38:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:38:29] T289318: Move *.beta.wmflabs.org to *.beta.wmcloud.org - https://phabricator.wikimedia.org/T289318 [21:41:25] !log T289318 - Change service::catalog probes for mw-api-int in Horizon prefix Puppet from en.wikipedia.beta.wmflabs.org/w/api.php to en.wikipedia.beta.wmcloud.org/w/api.php [21:41:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:01:49] Krinkle: thanks for the work on that long thread :wikilove: [22:11:57] bd808: yw, it does look like there's no end in sight, but I think we're past the half-way point now :) [22:12:52] Once canonicals are switched over, I'll file public suffix request for beta.wmcloud.org. That means we'll only have a short period of time where CentralAuth isn't fully testable in beta (right now we test cross-domain semantics by having test2wiki on wmcloud and the rest on wmflabs) [22:13:24] bd808: do you know more about the dkim stuff and its references to beta.wmfabs.org? [22:14:06] I'm unsure how to test that and what order to change things in there. I'm leaving that as-is for now, swtcihing only the API call to metawiki for bounce handler, which I assumed doesn't care about the domain name it hits [22:14:21] ie. doesn't have to match the email address or anything like that. [22:15:40] * Krinkle sees "Skipping sync since all commits were beta/labs-only changes." [22:15:51] Neat :) [22:19:23] yw. :-) [22:35:27] Krinkle: I don't know about DKIM really at all. I stopped knowing how to do email properly at least a decade ago. :/ [22:36:21] t.aavi can probably either tell you or point you at someone who can. He's my SRE of choice to nag about interesting stuff these days. :) [22:43:10] (03open) 10dancy: Release 4.186.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/877 [22:43:50] 10MediaWiki-Releasing, 10MediaWiki-Documentation, 05MW-1.44-notes, 05MW-1.44-release, 13Patch-For-Review: Finish up RELEASE-NOTES for 1.44.0 - https://phabricator.wikimedia.org/T390796#10970337 (10ABreault-WMF) a:03ABreault-WMF [22:44:33] 10MediaWiki-Releasing, 05MW-1.44-release: Release MW 1.44.0-rc.1 (if needed) - https://phabricator.wikimedia.org/T390795#10970339 (10ABreault-WMF) 05Openβ†’03Declined [22:44:41] 10GitLab, 06collaboration-services, 06serviceops, 07Kubernetes: Trusted gitlab runner containers need access to staging k8s cluster - https://phabricator.wikimedia.org/T325385#10970341 (10dancy) 05Openβ†’03Declined [22:45:18] (03merge) 10dancy: Release 4.186.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/877 [22:51:28] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.44-release, 13Patch-For-Review: Mark REL1_44 in ExtensionDistributor as a stable release - https://phabricator.wikimedia.org/T390798#10970349 (10ABreault-WMF) a:03ABreault-WMF [22:52:30] 10MediaWiki-Releasing, 05MW-1.44-release: Write and send release announcement for 1.44.0 - https://phabricator.wikimedia.org/T390797#10970350 (10ABreault-WMF) 05Openβ†’03Resolved a:03MSantos https://lists.wikimedia.org/hyperkitty/list/mediawiki-announce@lists.wikimedia.org/thread/C3ZZDKSFH2PW55GRH6Y4SX... [23:01:31] 06Release-Engineering-Team, 10Scap (SpiderPig πŸ•ΈοΈ): SpiderPig session / login broken - https://phabricator.wikimedia.org/T397457#10970360 (10dancy) @Lucas_Werkmeister_WMDE Sorry about the authentication troubles that you've been experiencing, but thank you for the detailed reporting. I haven't yet been able to... [23:07:02] Project beta-scap-sync-world build #213507: 04FAILURE in 1 min 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/213507/ [23:17:23] Yippee, build fixed! [23:17:23] Project beta-scap-sync-world build #213508: 09FIXED in 2 min 7 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/213508/ [23:42:50] >00:07:02 [18 hits] Wikimedia\Services\NoSuchServiceException: No such service: CategoryTree.CategoryTreeFactory [23:42:58] Is something not updating quick enough? [23:43:03] It seems more flaky than usual [23:48:39] That service is new, but I don't know specifically why it would error out :/ -- https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CategoryTree/+/1087970 [23:52:02] here's a different mystery. Why is https://codesearch.wmcloud.org/search/?q=CategoryTreeFactory empty when https://codesearch.wmcloud.org/things/?q=CategoryTreeFactory finds things? [23:53:22] (03update) 10bd808: spiderpig: Add a favicon [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/872 (https://phabricator.wikimedia.org/T394104) (owner: 10dancy) [23:53:29] (03approved) 10bd808: spiderpig: Add a favicon [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/872 (https://phabricator.wikimedia.org/T394104) (owner: 10dancy) [23:56:12] (03merge) 10bd808: spiderpig: Add a favicon [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/872 (https://phabricator.wikimedia.org/T394104) (owner: 10dancy) [23:58:55] (03approved) 10bd808: spiderpig-otp: Add --qr flag to generate QR code [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/784 (owner: 10dancy)