[05:33:33] 10Continuous-Integration-Infrastructure, 10MediaWiki-libs-Rdbms: Raise Quibble jobs' tested version of MariaDB to 10.6 - https://phabricator.wikimedia.org/T366646#9902074 (10Marostegui) In production we run 10.6. We have packages for Bullseye, but we encourage to go to Bookworm directly if possible. [08:43:36] (03CR) 10Hashar: [C:03+2] doc: Add mwseaql Rust crate [integration/docroot] - 10https://gerrit.wikimedia.org/r/1044914 (owner: 10Legoktm) [08:44:09] (03Merged) 10jenkins-bot: doc: Add mwseaql Rust crate [integration/docroot] - 10https://gerrit.wikimedia.org/r/1044914 (owner: 10Legoktm) [08:46:04] (03CR) 10Hashar: [C:03+2] Add new dependency to BlueSpiceBookshelf [integration/config] - 10https://gerrit.wikimedia.org/r/1046622 (owner: 10Robert Vogel) [08:47:31] (03Merged) 10jenkins-bot: Add new dependency to BlueSpiceBookshelf [integration/config] - 10https://gerrit.wikimedia.org/r/1046622 (owner: 10Robert Vogel) [08:48:11] (03CR) 10Hashar: [C:03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/1046622 (owner: 10Robert Vogel) [08:52:49] 10Gerrit, 06Diffusion-Repository-Administrators, 10Projects-Cleanup, 06translatewiki.net, 10Wikimedia-GitHub: Archive the Html2Wiki extension - https://phabricator.wikimedia.org/T347236#9902405 (10hashar) [08:52:50] 10Gerrit, 06Diffusion-Repository-Administrators, 10Projects-Cleanup, 06translatewiki.net, 10Wikimedia-GitHub: Archive the Html2Wiki extension - https://phabricator.wikimedia.org/T347236#9902403 (10hashar) 05Open→03Resolved Oups sorry! It is marked {nav read only} now: https://gerrit.wikimedia.org... [08:55:09] 10Gerrit, 06Diffusion-Repository-Administrators, 10Projects-Cleanup, 10Wikimedia-GitHub: Archive the DynamicPageList (third-party) extension - https://phabricator.wikimedia.org/T353927#9902407 (10hashar) 05Open→03Resolved James Forrester marked it read-only: and I have fixed the prefix to be `[ARCH... [08:55:23] 10Gerrit, 06Diffusion-Repository-Administrators, 10Projects-Cleanup, 10Wikimedia-GitHub: Archive the DynamicPageList (third-party) extension - https://phabricator.wikimedia.org/T353927#9902412 (10hashar) [09:03:15] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 17 - https://phabricator.wikimedia.org/T365675#9902455 (10Jelto) [09:18:01] (03update) 10taavi: make-release: Stop branching OSM [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/80 (https://phabricator.wikimedia.org/T161553) [09:25:56] Project beta-update-databases-eqiad build #76818: 04FAILURE in 5 min 55 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/76818/ [09:28:49] ^ is due to cloud-VPS maintenance [09:31:12] (03open) 10jelto: add do not edit message to runner description [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/63 (https://phabricator.wikimedia.org/T366786) [09:31:54] (03PS1) 10Majavah: zuul: [operations/puppet] Make PCC voting on Puppet 7 [integration/config] - 10https://gerrit.wikimedia.org/r/1047024 (https://phabricator.wikimedia.org/T367399) [09:31:55] (03PS1) 10Majavah: jjb: Default to Puppet 7 for PCC jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1047025 (https://phabricator.wikimedia.org/T367399) [09:40:40] beta-update-databases-eqiad should pass next run [09:51:33] (03CR) 10Hashar: Copy phpunit.xml and cache result file to LOG_DIR (033 comments) [integration/quibble] - 10https://gerrit.wikimedia.org/r/1044611 (owner: 10Kosta Harlan) [09:51:39] (03PS12) 10Hashar: Copy phpunit.xml and cache result file to LOG_DIR [integration/quibble] - 10https://gerrit.wikimedia.org/r/1044611 (owner: 10Kosta Harlan) [09:57:08] 06Release-Engineering-Team, 10Scap, 06serviceops: Use conftool to build scap's kubernetes_workers host list - https://phabricator.wikimedia.org/T367862 (10Clement_Goubert) 03NEW [10:04:41] 06Release-Engineering-Team, 10Scap, 06serviceops: Use conftool to build scap's kubernetes_workers host list - https://phabricator.wikimedia.org/T367862#9902664 (10Clement_Goubert) {T366778} is looking towards removing this step completely [10:08:46] 06Release-Engineering-Team, 10Scap: scap should not run mediawiki-image-download on pooled=inactive servers - https://phabricator.wikimedia.org/T363971#9902672 (10jnuche) See https://phabricator.wikimedia.org/T367862 for an approach with puppet to fix this issue [10:12:01] Project beta-scap-sync-world build #159830: 04FAILURE in 6 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159830/ [10:19:59] Project beta-scap-sync-world build #159831: 04STILL FAILING in 1 min 15 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159831/ [10:26:09] Project beta-scap-sync-world build #159832: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159832/ [10:30:02] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog 📥), 06Traffic, 07Upstream: Consider using preconnect for https://phab.wmfusercontent.org CDN - https://phabricator.wikimedia.org/T367290#9902731 (10Aklapper) [10:30:55] Yippee, build fixed! [10:30:56] Project beta-update-databases-eqiad build #76819: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/76819/ [10:36:09] Project beta-scap-sync-world build #159833: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159833/ [10:36:28] !log contint1002: deleted `/srv/zuul/git/operations/software/homer/deploy` and `/srv/zuul/git/operations/software/homer` for topranks due to zuul-merger bug T157818 [10:36:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:36:31] T157818: zuul-merger fails when repository names overlaps - https://phabricator.wikimedia.org/T157818 [10:36:44] hashar: much obliged :) [10:37:27] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog 📥), 06Traffic, 07Upstream: Consider using preconnect for https://phab.wmfusercontent.org CDN - https://phabricator.wikimedia.org/T367290#9902756 (10Aklapper) 05Open→03Stalled [10:37:30] I dont' think I am always logging them [10:37:47] but yeah that happens each time we reimage the hosts (every 4 years or so) [10:37:59] for the few repos unfortunate to have an overlap [10:43:09] random observation: the fresh-node image doesn’t have `less` installed, so `git diff` falls back to `more` [10:43:23] I’m not sure if this is a “regression”, or if I just never tried to use `git diff` inside fresh before ^^ [10:46:08] Project beta-scap-sync-world build #159834: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159834/ [10:46:22] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog 📥), 06Traffic, 07Upstream: Consider using preconnect for https://phab.wmfusercontent.org CDN - https://phabricator.wikimedia.org/T367290#9902778 (10Vgutierrez) we already leverage preconnect on some cases but not as an HTTP Header but u... [10:56:09] Project beta-scap-sync-world build #159835: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159835/ [11:06:24] 10Release-Engineering-Team (Priority Backlog 📥), 06serviceops, 06Traffic, 13Patch-For-Review, 10Release Pipeline (Blubber): Remove blubberoid LVS/k8s service - https://phabricator.wikimedia.org/T365742#9902842 (10JMeybohm) [11:06:29] Project beta-scap-sync-world build #159836: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159836/ [11:16:08] Project beta-scap-sync-world build #159837: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159837/ [11:26:09] Project beta-scap-sync-world build #159838: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159838/ [11:36:15] Project beta-scap-sync-world build #159839: 04STILL FAILING in 53 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159839/ [11:46:13] Project beta-scap-sync-world build #159840: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159840/ [11:46:41] (03PS1) 10Hashar: Point its-phabricator to stable-3.9 [software/gerrit] (wmf/stable-3.9) - 10https://gerrit.wikimedia.org/r/1047058 [11:49:26] (03CR) 10Arthur taylor: [C:03+1] "Thanks for the fixes!" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1044611 (owner: 10Kosta Harlan) [11:50:18] Lucas_WMDE: for lack of `less` or paging in fresh, you can always file it as a task against #fresh [11:50:41] my guess is that it never was in the image which are the images used by Zuul CI for which there is no need for less [11:51:47] yeah, that’s why I didn’t open a task right away, I’m hesitant to bloat the CI image with it w^ [11:51:48] * ^^ [11:52:46] me too :D [11:53:10] my guess is in most case you are better off running git commands outside of the containers [11:53:15] eg on the host / laptop / desktop machine [11:53:21] yeah, .git is mounted read-only anyways [11:53:27] with all your customizations enabled [11:53:49] though if the lack of less causes an error message fro mgit diff we can look at adding a git config which disables paging [11:54:25] nah, more is installed as I said, just less useable ^^ [11:54:34] export GIT_PAGER='printf >&2 "you probably want to run this outside fresh\n"; more' [11:54:39] so less is more usable? [11:54:42] * hashar giggles [11:54:44] yes! [11:54:56] I'd want to use vim as a pager [11:54:57] less useable. more not useable [11:55:02] * Lucas_WMDE recoils [11:55:04] and get syntax highlighting [11:55:32] isn’t that what lesspipe is for 🤔 [11:55:41] I *think* I had that set up on a laptop years ago [11:56:09] Project beta-scap-sync-world build #159841: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159841/ [12:01:04] 06Release-Engineering-Team, 10Scap, 06serviceops, 13Patch-For-Review: Use conftool to build scap's kubernetes_workers host list - https://phabricator.wikimedia.org/T367862#9902987 (10Clement_Goubert) 05Open→03In progress p:05Triage→03Medium [12:02:56] 06Release-Engineering-Team, 10Scap, 10MW-on-K8s, 06serviceops-radar: Add a scap way to deploy mw-on-k8s without an image rebuild - https://phabricator.wikimedia.org/T351880#9903010 (10Clement_Goubert) 05In progress→03Resolved [12:06:07] Project beta-scap-sync-world build #159842: 04STILL FAILING in 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159842/ [12:10:02] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10Release Pipeline, 06serviceops-radar, 07Epic: Deploy MediaWiki to Wikimedia production in containers - https://phabricator.wikimedia.org/T238770#9903034 (10Clement_Goubert) 05Open→03Resolved a:03Clement_Goubert Unless I'm mistaken, we've been do... [12:16:06] Project beta-scap-sync-world build #159843: 04STILL FAILING in 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159843/ [12:26:09] Project beta-scap-sync-world build #159844: 04STILL FAILING in 53 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159844/ [12:36:13] Project beta-scap-sync-world build #159845: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159845/ [12:36:17] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10Release Pipeline, 06serviceops-radar, 07Epic: Deploy MediaWiki to Wikimedia production in containers - https://phabricator.wikimedia.org/T238770#9903101 (10Jdforrester-WMF) >>! In T238770#9903034, @Clement_Goubert wrote: > Unless I'm mistaken, we've b... [12:36:30] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 10Release Pipeline: Include MediaWiki testing in .pipeline/config.yaml execution graph - https://phabricator.wikimedia.org/T279340#9903104 (10Jdforrester-WMF) 05Open→03Declined We went a different way with multi-version images and custom s... [12:36:52] 10Release-Engineering-Team (Seen), 07Epic, 07Technical-Debt: Figure out which production extensions have conflicting tests and fix them - https://phabricator.wikimedia.org/T238492#9903108 (10Jdforrester-WMF) [12:36:53] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10Release Pipeline, 06serviceops-radar, 07Epic: Deploy MediaWiki to Wikimedia production in containers - https://phabricator.wikimedia.org/T238770#9903109 (10Jdforrester-WMF) [12:37:09] 10Release-Engineering-Team (Seen), 07Epic, 07Technical-Debt: Figure out which production extensions have conflicting tests and fix them - https://phabricator.wikimedia.org/T238492#9903110 (10Jdforrester-WMF) This no longer blocks mw-on-k8s work, as we went in a different direction. [12:37:11] 10Release-Engineering-Team (Seen), 10Release Pipeline, 06serviceops-radar, 07Epic: Get production MW-land images built and published - https://phabricator.wikimedia.org/T238771#9903096 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF This has been done for a while. [12:46:13] Project beta-scap-sync-world build #159846: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159846/ [12:56:07] Project beta-scap-sync-world build #159847: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159847/ [13:06:14] Project beta-scap-sync-world build #159848: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159848/ [13:13:37] pff [13:13:56] looks like the keyholder needs to be rearmerd [13:14:16] yes [13:14:41] reboot system boot 4.19.0-26-cloud- Tue Jun 18 10:13 still running [13:14:54] hashar: yes :D I was just about to SSH in and take a look, but have forgotten which host to run `keyholder arm` on [13:15:04] I thought the deployment server [13:15:17] oh, `sudo keyholder arm` [13:16:05] (I see you're doing something there, so will leave it be) [13:16:09] Project beta-scap-sync-world build #159849: 04STILL FAILING in 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159849/ [13:21:15] !log deployment-prep: rearmed keyholder on deployment-deploy03 following the unexpected restart of the instance [13:21:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:21:19] TheresNoTime: yeah :) [13:21:28] that should be good now [13:21:37] at least https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159850/console shows some progress [13:22:42] :D [13:23:24] I see T144647 exists, but it would be nice to have some sort of ping when this happens (not that it happens often I guess..) [13:23:24] T144647: Automate beta scap3/keyholder setup - https://phabricator.wikimedia.org/T144647 [13:24:55] Yippee, build fixed! [13:24:55] Project beta-scap-sync-world build #159850: 09FIXED in 5 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/159850/ [13:28:14] 10Phabricator: Custom epic form for #CommTech - https://phabricator.wikimedia.org/T367877 (10KSiebert) 03NEW [13:32:09] TheresNoTime: that is a different task [13:32:54] the ssh known_host is probably generated by Puppet nowadays [13:33:07] it might be an obsolete task :D [13:35:55] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Herald rules H295 and H301 broken as their conditions list custom "date" fields somehow not supported by Herald - https://phabricator.wikimedia.org/T367830#9903322 (10Aklapper) p:05Triage→03Low a:03Aklapper [13:35:56] (03PS1) 10Krinkle: test: make npm-publish test more strict in output validation [fresh] - 10https://gerrit.wikimedia.org/r/1047082 [13:36:07] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Herald rules H295 and H301 broken as their conditions list custom "date" fields somehow not supported by Herald - https://phabricator.wikimedia.org/T367830#9903327 (10Aklapper) 05Open→03Resolved Seems to happen for custom fields of `"type"... [13:36:46] (03CR) 10Krinkle: fresh-npm: write "command install is disabled" to stderr (031 comment) [fresh] - 10https://gerrit.wikimedia.org/r/1040855 (owner: 10Krinkle) [13:41:54] 10Phabricator: Custom epic form for #CommTech - https://phabricator.wikimedia.org/T367877#9903351 (10Aklapper) Hi, per T365578#9825097 you should be able to set this up; see https://www.mediawiki.org/wiki/Phabricator/Forms for docs [13:48:22] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Get GitLab to render `T{\d}+` in MR overviews, comments, etc. as links to Phabricator - https://phabricator.wikimedia.org/T337570#9903391 (10Sportzpikachu) I encountered the same problem with Gitlab not letting me delete... [13:56:08] (03CR) 10Jforrester: [C:03+2] test: make npm-publish test more strict in output validation [fresh] - 10https://gerrit.wikimedia.org/r/1047082 (owner: 10Krinkle) [14:06:10] (03Merged) 10jenkins-bot: test: make npm-publish test more strict in output validation [fresh] - 10https://gerrit.wikimedia.org/r/1047082 (owner: 10Krinkle) [14:07:36] (03CR) 10Kosta Harlan: [C:03+2] Copy phpunit.xml and cache result file to LOG_DIR [integration/quibble] - 10https://gerrit.wikimedia.org/r/1044611 (owner: 10Kosta Harlan) [14:12:22] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Get GitLab to render `T{\d}+` in MR overviews, comments, etc. as links to Phabricator - https://phabricator.wikimedia.org/T337570#9903479 (10brennen) > Generated a new access token for the project (/-/settings/access_tok... [14:23:56] (03Merged) 10jenkins-bot: Copy phpunit.xml and cache result file to LOG_DIR [integration/quibble] - 10https://gerrit.wikimedia.org/r/1044611 (owner: 10Kosta Harlan) [14:27:32] (03update) 10brennen: Draft: update submodules for 2024-06-18 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/45 (https://phabricator.wikimedia.org/T367775) [14:27:49] (03update) 10brennen: update submodules for 2024-06-18 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/45 (https://phabricator.wikimedia.org/T367775) [14:31:35] (03PS1) 10Kosta Harlan: PHPUnit: Specify .json suffix for result cache files [integration/quibble] - 10https://gerrit.wikimedia.org/r/1047096 [14:32:12] (03approved) 10brennen: add do not edit message to runner description [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/63 (https://phabricator.wikimedia.org/T366786) (owner: 10jelto) [14:33:31] (03CR) 10CI reject: [V:04-1] PHPUnit: Specify .json suffix for result cache files [integration/quibble] - 10https://gerrit.wikimedia.org/r/1047096 (owner: 10Kosta Harlan) [14:35:31] (03PS2) 10Kosta Harlan: PHPUnit: Specify .json suffix for result cache files [integration/quibble] - 10https://gerrit.wikimedia.org/r/1047096 [14:37:25] (03CR) 10Hashar: [C:03+2] "👍" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1047096 (owner: 10Kosta Harlan) [14:38:36] (03CR) 10Hashar: [C:03+2] zuul: [operations/puppet] Make PCC voting on Puppet 7 [integration/config] - 10https://gerrit.wikimedia.org/r/1047024 (https://phabricator.wikimedia.org/T367399) (owner: 10Majavah) [14:39:08] (03CR) 10Hashar: [C:03+2] zuul: [operations/puppet] Make PCC voting on Puppet 7 [integration/config] - 10https://gerrit.wikimedia.org/r/1047024 (https://phabricator.wikimedia.org/T367399) (owner: 10Majavah) [14:40:01] (03Merged) 10jenkins-bot: zuul: [operations/puppet] Make PCC voting on Puppet 7 [integration/config] - 10https://gerrit.wikimedia.org/r/1047024 (https://phabricator.wikimedia.org/T367399) (owner: 10Majavah) [14:40:44] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/1047024 "Make PCC voting on Puppet 7" # T367399 [14:40:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:40:46] T367399: Default to the Puppet 7 PCC CI test, make it voting and eventually remove the Puppet 5 one - https://phabricator.wikimedia.org/T367399 [14:41:11] (03CR) 10Hashar: [C:03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/1047025 (https://phabricator.wikimedia.org/T367399) (owner: 10Majavah) [14:42:37] (03Merged) 10jenkins-bot: jjb: Default to Puppet 7 for PCC jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1047025 (https://phabricator.wikimedia.org/T367399) (owner: 10Majavah) [14:42:58] (03CR) 10Hashar: [C:03+2] Point its-phabricator to stable-3.9 [software/gerrit] (wmf/stable-3.9) - 10https://gerrit.wikimedia.org/r/1047058 (owner: 10Hashar) [14:45:16] (03PS3) 10Hashar: wmf: `bazel test` our plugins [software/gerrit] (wmf/stable-3.9) - 10https://gerrit.wikimedia.org/r/1046783 [14:46:34] (03CR) 10Hashar: [C:03+2] wmf: `bazel test` our plugins [software/gerrit] (wmf/stable-3.9) - 10https://gerrit.wikimedia.org/r/1046783 (owner: 10Hashar) [14:47:23] 10Scap, 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Evaluate the performance improvements brought in by prefetching MW images on WikiKube hosts - https://phabricator.wikimedia.org/T366778#9903647 (10akosiaris) [14:50:30] (03Merged) 10jenkins-bot: Point its-phabricator to stable-3.9 [software/gerrit] (wmf/stable-3.9) - 10https://gerrit.wikimedia.org/r/1047058 (owner: 10Hashar) [14:53:38] (03Merged) 10jenkins-bot: PHPUnit: Specify .json suffix for result cache files [integration/quibble] - 10https://gerrit.wikimedia.org/r/1047096 (owner: 10Kosta Harlan) [14:55:30] (03Merged) 10jenkins-bot: wmf: `bazel test` our plugins [software/gerrit] (wmf/stable-3.9) - 10https://gerrit.wikimedia.org/r/1046783 (owner: 10Hashar) [14:57:03] (03PS1) 10Kosta Harlan: release: Quibble 1.9.1 [integration/quibble] - 10https://gerrit.wikimedia.org/r/1047103 [14:58:12] (03PS1) 10Kosta Harlan: release: Start 1.9.2 cycle [integration/quibble] - 10https://gerrit.wikimedia.org/r/1047104 [14:58:53] 10GitLab (Project Migration), 06Release-Engineering-Team: Create new GitLab project group: Moderator Tools - https://phabricator.wikimedia.org/T367885 (10jsn.sherman) 03NEW [15:03:02] (03merge) 10brennen: update submodules for 2024-06-18 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/45 (https://phabricator.wikimedia.org/T367775) [15:03:34] 10Phabricator (phabricator-next), 10Phabricator Antivandalism Extension, 10Release-Engineering-Team (Priority Backlog 📥): AVA performance improvements - https://phabricator.wikimedia.org/T366811#9903743 (10Aklapper) [15:03:49] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 13Patch-For-Review: Deploy Phabricator/Phorge 2024-06-18 - https://phabricator.wikimedia.org/T367775#9903744 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=434fb169-ea49-4b50-99c6-92fe8b7cc86e) set by jelto@cumin1002 for 0:30:... [15:04:06] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 13Patch-For-Review: Deploy Phabricator/Phorge 2024-06-18 - https://phabricator.wikimedia.org/T367775#9903745 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=f4fe9445-4599-4964-a4f6-56cd41103592) set by jelto@cumin1002 for 0:30:... [15:06:32] hi. I need to reboot all VMs in both the gitlab-runners and integration Cloud VPS projects, can that be done anytime or does that need more coordination? [15:07:18] also looking for a merge on https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/80 [15:07:46] taavi: I can coordinate w/ you to mark the WMCS runners offline. [15:09:08] (03merge) 10dancy: make-release: Stop branching OSM [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/80 (https://phabricator.wikimedia.org/T161553) (owner: 10taavi) [15:10:23] taavi: Ready to pause those runners when you are. [15:10:47] thanks. give me one moment, I just realized those have a custom flavor I need to copy to the new system [15:11:00] (this is related to https://lists.wikimedia.org/hyperkitty/list/cloud-announce@lists.wikimedia.org/thread/CKQS5FU5PSXRTWJBNJ54CRE42256ZHAQ/) [15:13:13] 👍🏾 [15:13:36] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 13Patch-For-Review: Deploy Phabricator/Phorge 2024-06-18 - https://phabricator.wikimedia.org/T367775#9903782 (10Aklapper) Today's attempt to deploy broke Phabricator, displaying https://phabricator.wikimedia.org/source/phabricator/browse/wmf%252Fs... [15:18:02] 10Phabricator (phabricator-next), 06collaboration-services, 07Technical-Debt: Investigate / remove custom downstream changes to PhabricatorClientRateLimit.php - https://phabricator.wikimedia.org/T364839#9903790 (10Aklapper) Today's attempt to deploy in {T367775} broke Phabricator, displaying the error messag... [15:20:37] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 06Data-Platform-SRE, 10Wikidata, and 2 others: move commons-query.wikimedia.org and query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#9903793 (10Gehel) [15:24:40] dancy: basically ready now, just waiting for gate-and-submit to pass on my cookbook patch. so I think you can take the first ones out of service, and let me know and I'll reboot them [15:25:00] OK. Paused. [15:25:24] all of them? or just some percentage? [15:32:28] I paused the instance-wide runner and the group assigned to /repos/. There's no way to pause individual runners in a group as far as I can tell. [15:32:57] Neither of them are heavily used so I think we're ok [15:38:04] 06Release-Engineering-Team, 10docker-pkg, 06serviceops: Attach opencontainers image metadata to docker images - https://phabricator.wikimedia.org/T345070#9903864 (10elukey) [15:40:02] (03update) 10dancy: Move logstash checker code into scap [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/358 (https://phabricator.wikimedia.org/T159991 https://phabricator.wikimedia.org/T183999 https://phabricator.wikimedia.org/T367131) [15:41:43] dancy: all done [15:42:03] Great.. Resuming the WMCS runners. [16:10:55] 10Continuous-Integration-Config, 10MediaWiki-extensions-General, 07good first task: Remove hardcoded message paths in banana-checker configuration - https://phabricator.wikimedia.org/T346178#9904031 (10Pppery) [16:11:05] 10Continuous-Integration-Config, 10MediaWiki-extensions-General, 07good first task: Remove hardcoded message paths in banana-checker configuration - https://phabricator.wikimedia.org/T346178#9904038 (10Pppery) I'm inclined to agree that we shouldn't change the world over a very obscure corner case. [16:26:20] (03PS1) 10Pppery: Zuul: [mediawiki/extensions/NamespaceHTML] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047127 (https://phabricator.wikimedia.org/T360235) [16:26:22] (03PS1) 10Pppery: Zuul: [mediawiki/extensions/NewsTicker] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047128 (https://phabricator.wikimedia.org/T353928) [16:26:23] (03PS1) 10Pppery: Zuul: [mediawiki/extensions/SideBarMenu] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047129 [16:26:23] (03PS1) 10Pppery: Zuul: [mediawiki/extensions/SocialLogin] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047130 (https://phabricator.wikimedia.org/T246273) [16:27:18] (03PS2) 10Pppery: Zuul: [mediawiki/extensions/SideBarMenu] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047129 (https://phabricator.wikimedia.org/T347216) [16:27:31] (03PS2) 10Pppery: Zuul: [mediawiki/extensions/SocialLogin] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047130 (https://phabricator.wikimedia.org/T246273) [17:26:47] brennen: I forgot earlier: Big thanks for reviewing my Phab patches! [17:35:42] sure thing [17:47:18] (03PS1) 10Dreamrimmer: zuul: Add Superzerocool to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1047152 [17:49:08] (03open) 10dancy: fake-logstash: Update for new logstash checker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/75 [17:49:11] (03update) 10dancy: fake-logstash: Update for new logstash checker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/75 [18:21:29] (03update) 10dancy: fake-logstash: Update for new logstash checker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/75 [18:38:52] (03open) 10dancy: kubernetes: Ensure progress reporter is always notified [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/359 [18:38:55] (03update) 10dancy: kubernetes: Ensure progress reporter is always notified [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/359 [18:39:08] (03open) 10dancy: kubernetes: Always use replicas:1 in traindev [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/360 [18:39:10] (03update) 10dancy: kubernetes: Always use replicas:1 in traindev [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/360 [18:39:41] (03update) 10dancy: kubernetes: Always use replicas:1 in traindev [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/360 [18:40:00] (03update) 10dancy: kubernetes: Ensure progress reporter is always notified [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/359 [18:43:25] (03open) 10dancy: enable mw-web canary / main k8s deployment [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/76 [18:43:28] (03update) 10dancy: enable mw-web canary / main k8s deployment [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/76 [18:47:28] (03open) 10dancy: Release 4.88.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/361 [18:49:09] (03merge) 10dancy: Release 4.88.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/361 [19:19:32] (03PS5) 10Hashar: Merge branch 'stable-3.10' into wmf/stable-3.10 [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1043813 (https://phabricator.wikimedia.org/T367419) [19:30:14] (03CR) 10CI reject: [V:04-1] Merge branch 'stable-3.10' into wmf/stable-3.10 [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1043813 (https://phabricator.wikimedia.org/T367419) (owner: 10Hashar) [19:48:40] (03PS6) 10Hashar: Merge branch 'stable-3.10' into wmf/stable-3.10 [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1043813 (https://phabricator.wikimedia.org/T367419) [20:29:03] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T361404#9905090 (10Jdforrester-WMF) [20:35:02] (03PS1) 10Hashar: Gerrit 3.10.x rebuild plugins and update TypeScript API [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1047175 (https://phabricator.wikimedia.org/T367419) [20:38:09] (03update) 10dancy: Move logstash checker code into scap [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/358 (https://phabricator.wikimedia.org/T159991 https://phabricator.wikimedia.org/T183999 https://phabricator.wikimedia.org/T367131) [20:38:33] (03update) 10dancy: Move logstash checker code into scap [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/358 (https://phabricator.wikimedia.org/T159991 https://phabricator.wikimedia.org/T183999 https://phabricator.wikimedia.org/T367131) [20:41:11] 10Gerrit (Gerrit 3.10), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Upgrade to Gerrit 3.10 - https://phabricator.wikimedia.org/T367419#9905126 (10thcipriani) >>! In T367419#9889400, @Jdforrester-WMF wrote: >> * Gerrit no longer send emails to everyone when additional reviewers... [20:52:58] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 07Performance Issue: Submitting actions taking ~10 seconds to load after making changes - https://phabricator.wikimedia.org/T360484#9905149 (10HNordeenWMF) hi @Aklapper, any updates on this one? Is there any other troubleshooting I can help with? [20:53:04] (03PS2) 10Hashar: Gerrit 3.10.x rebuild plugins and update TypeScript API [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1047175 (https://phabricator.wikimedia.org/T367419) [20:53:13] 10Gerrit (Gerrit 3.10), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Upgrade to Gerrit 3.10 - https://phabricator.wikimedia.org/T367419#9905153 (10hashar) a:03hashar [20:53:32] !log gitlab: squash commit templates: update all projects to use %{all_commits} (T366624) [20:53:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:53:35] T366624: GitLab's default commit templates often lose the text of existing commit messages - https://phabricator.wikimedia.org/T366624 [20:56:46] (03update) 10dancy: Move logstash checker code into scap [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/358 (https://phabricator.wikimedia.org/T159991 https://phabricator.wikimedia.org/T183999 https://phabricator.wikimedia.org/T367131) [21:00:17] (03update) 10dancy: Move logstash checker code into scap [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/358 (https://phabricator.wikimedia.org/T159991 https://phabricator.wikimedia.org/T183999 https://phabricator.wikimedia.org/T367131) [21:15:04] 14Gerrit (Gerrit 3.9): Configure Gerrit to use conflictStyle diff3 - https://phabricator.wikimedia.org/T359821#9905229 (10hashar) And here is a screenshot for posterity which shows the diff3 markers (`<<<<<<<`, `======` and `>>>>>>>`): {F55445227 size=full} Thank you @Lucas_Werkmeister_WMDE to have notice... [21:19:12] 10GitLab (Administration, Settings & Policy), 10gitlab-settings, 10Release-Engineering-Team (Yak Shaving 🐃🪒): GitLab's default commit templates often lose the text of existing commit messages - https://phabricator.wikimedia.org/T366624#9905253 (10brennen) Filed upstream: https://gitlab.com/gitlab-org/gitlab/... [21:21:10] 14Gerrit (Gerrit 3.9): Configure Gerrit to use conflictStyle diff3 - https://phabricator.wikimedia.org/T359821#9905265 (10LucasWerkmeister) Hang on, I think I got confused there. Isn’t that just a normal merge conflict display, and with `diff3` style we’d also expect a `|||||||` marker? 🤔 [21:22:05] 14Gerrit (Gerrit 3.9): Configure Gerrit to use conflictStyle diff3 - https://phabricator.wikimedia.org/T359821#9905269 (10LucasWerkmeister) At least that’s what I get when I do a quick nonsense merge conflict locally: ` <<<<<<< .merge_file_aBXbpe A b ||||||| .merge_file_acSBfm a b ======= a B >>>>>>> .merge... [21:38:47] (03open) 10brennen: configure-projects: use '%{all_commits}' for squash commit template [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/64 (https://phabricator.wikimedia.org/T366624) [21:41:10] 14Gerrit (Gerrit 3.9): Configure Gerrit to use conflictStyle diff3 - https://phabricator.wikimedia.org/T359821#9905329 (10hashar) Grmblblblb The upstream patch is https://gerrit-review.googlesource.com/c/gerrit/+/382894 which was releases in v3.9.0 (and we run a snapshot of 3.9.6). My guess is that it is on... [21:41:13] * hashar sleeps [21:46:00] hashar: maybe https://github.com/GerritCodeReview/gerrit/blob/dafe2982321d068b6cd157c9d42d06151f3d0aaf/java/com/google/gerrit/server/change/RebaseChangeOp.java#L550 needs updating to support it. But I'm not fully sure [22:27:46] 10Continuous-Integration-Config, 10MediaWiki-extensions-CentralAuth, 10MediaWiki-Installer, 06MediaWiki-Platform-Team, and 2 others: Admin account created by the installer isn't made global by CentralAuth - https://phabricator.wikimedia.org/T358985#9905425 (10matmarex) That seems to work! * Before: https:/... [22:50:54] (03open) 10aklapper: Revert "Revert custom changes to PhabricatorClientRateLimit.php" [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/59 (https://phabricator.wikimedia.org/T364839) [23:30:43] (03PS1) 10Jforrester: Zuul: Drop config for wmf branches from -composer templates [integration/config] - 10https://gerrit.wikimedia.org/r/1047195 [23:30:43] (03PS1) 10Jforrester: Zuul: Move 'in-wikimedia-production' repos to prod section [integration/config] - 10https://gerrit.wikimedia.org/r/1047196 [23:30:43] (03PS1) 10Jforrester: Zuul: [mediawiki/tools/dependency-analysis] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047197 (https://phabricator.wikimedia.org/T350258) [23:35:39] (03CR) 10Jforrester: [C:03+2] Zuul: Drop config for wmf branches from -composer templates [integration/config] - 10https://gerrit.wikimedia.org/r/1047195 (owner: 10Jforrester) [23:35:45] (03CR) 10Jforrester: [C:03+2] Zuul: Move 'in-wikimedia-production' repos to prod section [integration/config] - 10https://gerrit.wikimedia.org/r/1047196 (owner: 10Jforrester) [23:37:35] (03Merged) 10jenkins-bot: Zuul: Drop config for wmf branches from -composer templates [integration/config] - 10https://gerrit.wikimedia.org/r/1047195 (owner: 10Jforrester) [23:37:42] (03Merged) 10jenkins-bot: Zuul: Move 'in-wikimedia-production' repos to prod section [integration/config] - 10https://gerrit.wikimedia.org/r/1047196 (owner: 10Jforrester) [23:38:21] !log Zuul: Drop config for wmf branches from -composer templates [23:38:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:38:28] !log Zuul: Move 'in-wikimedia-production' repos to prod section [23:38:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:40:24] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/tools/dependency-analysis] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047197 (https://phabricator.wikimedia.org/T350258) (owner: 10Jforrester) [23:41:51] (03Merged) 10jenkins-bot: Zuul: [mediawiki/tools/dependency-analysis] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047197 (https://phabricator.wikimedia.org/T350258) (owner: 10Jforrester) [23:42:40] !log Zuul: [mediawiki/tools/dependency-analysis] Mark as archived, for T350258 [23:42:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:42:42] T350258: Archive the mediawiki/tools/dependency-analysis/ repo - https://phabricator.wikimedia.org/T350258 [23:45:02] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/NamespaceHTML] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047127 (https://phabricator.wikimedia.org/T360235) (owner: 10Pppery) [23:45:08] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/NewsTicker] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047128 (https://phabricator.wikimedia.org/T353928) (owner: 10Pppery) [23:45:13] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/SideBarMenu] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047129 (https://phabricator.wikimedia.org/T347216) (owner: 10Pppery) [23:45:43] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/SocialLogin] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047130 (https://phabricator.wikimedia.org/T246273) (owner: 10Pppery) [23:46:25] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/NamespaceHTML] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047127 (https://phabricator.wikimedia.org/T360235) (owner: 10Pppery) [23:46:36] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/NewsTicker] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047128 (https://phabricator.wikimedia.org/T353928) (owner: 10Pppery) [23:46:40] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/SideBarMenu] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047129 (https://phabricator.wikimedia.org/T347216) (owner: 10Pppery) [23:47:03] !log Zuul: [mediawiki/extensions/NamespaceHTML] Mark as archived, for T360235 [23:47:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:47:06] T360235: Archive the NamespaceHTML extension - https://phabricator.wikimedia.org/T360235 [23:47:08] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/SocialLogin] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1047130 (https://phabricator.wikimedia.org/T246273) (owner: 10Pppery) [23:47:17] !log Zuul: [mediawiki/extensions/NewsTicker] Mark as archived, for T353928 [23:47:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:47:20] T353928: Archive the NewsTicker extension - https://phabricator.wikimedia.org/T353928 [23:47:33] !log Zuul: [mediawiki/extensions/SideBarMenu] Mark as archived, for T347216 [23:47:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:47:35] T347216: Archive the SideBarMenu extension - https://phabricator.wikimedia.org/T347216 [23:47:51] !log Zuul: [mediawiki/extensions/SocialLogin] Mark as archived, for T246273 [23:47:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:47:53] T246273: Archive the SocialLogin extension - https://phabricator.wikimedia.org/T246273