[00:02:48] Reedy: ty [00:07:48] Project beta-scap-sync-world build #177342: 04STILL FAILING in 2 min 21 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177342/ [00:18:37] (03open) 10dduvall: jobs: Pass inline scap config to avoid any mwbuilder sudo'ing [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/90 [00:18:40] Yippee, build fixed! [00:18:40] Project beta-scap-sync-world build #177343: 09FIXED in 2 min 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177343/ [00:22:38] (03update) 10dduvall: jobs: Pass inline scap config to avoid any mwbuilder sudo'ing [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/90 [00:22:39] (03update) 10dduvall: jobs: Pass inline scap config to avoid any mwbuilder sudo'ing [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/90 [00:32:04] Yippee, build fixed! [00:32:04] Project beta-update-databases-eqiad build #79799: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79799/ [01:21:35] sweet [01:38:04] 10GitLab (Project Migration), 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python: Move Wmfdata-Python from Github to Gitlab - https://phabricator.wikimedia.org/T304544#10248839 (10nshahquinn-wmf) [01:41:59] 10GitLab (Project Migration), 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python: Move Wmfdata-Python from Github to Gitlab - https://phabricator.wikimedia.org/T304544#10248853 (10nshahquinn-wmf) @fkaelin is interested in co-hosting a Wmfdata work session during the Research and Data Science offsite... [08:18:26] Project beta-scap-sync-world build #177391: 04FAILURE in 2 min 24 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177391/ [08:29:09] Yippee, build fixed! [08:29:09] Project beta-scap-sync-world build #177392: 09FIXED in 2 min 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177392/ [09:12:25] * Lucas_WMDE wonders how long it’ll take for the broken WikibaseLexeme + MediaSearch changes to fall out of zuul’s gate-and-submit queue [09:21:50] now they’re gone, yay [09:27:17] does anyone see why https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseQualityConstraints/+/1082081 isn’t showing up in gate-and-submit at https://integration.wikimedia.org/zuul/ ? [09:27:25] I’m at a loss :( [09:27:36] and it’s blocking CI for several other repositories [09:39:19] feels like I might have to just force-merge that change to fix CI :S [09:41:49] Lucas_WMDE: I think that got broken because a change got force merged [09:42:03] I should remove that permission entirely. I think I will just do it [09:42:45] what. [09:42:57] there was a circular dependency [09:43:01] both changes were needed to fix CI [09:43:07] eeek :D [09:43:11] how else was I supposed to fix that without a force merge? [09:43:23] I mean [09:43:27] the force merge breaks zuul [09:43:35] it loose track of the state of things and get confused [09:43:45] what was the circular dep? [09:44:06] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseQualityConstraints/+/1082081 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexeme/+/1082086 were both needed to fix CI [09:44:11] (due to breakage introduced in MW core) [09:44:26] OH [09:44:35] one was marked Depends-On the other so I force-merged the one without Depends-On [09:44:42] and hoped that zuul would *eventually* figure things out [09:45:01] aren't changes made to mediawiki/core running tests for those two exts? [09:45:03] (the force-merge was already an hour ago, I already waited for the old gate-and-submit queue from before the force merge to clear out) [09:45:15] apparently not [09:46:19] (the breaking core change is https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1007693 btw) [09:46:49] hmm [09:47:11] https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74/35109/console : SUCCESS in 19m 30s [09:47:25] and indeed that job does not have WikibaseQualityConstraints and WikibaseLexeme in the dependencies [09:47:35] I feel like we should move Wikibase & all out of that job [09:47:39] toward a dedicated job [09:48:03] I think I had a task for that which was to move the Wikibase tests out of that job [09:48:06] aka split the large job [09:48:47] anyway, if core broke things, for sure you end up being forced to merge sometime :] [09:49:15] I have no complaint on that, it just that Zuul get confused and that explains why it takes a while for the changes to merge [09:49:36] okay, and now what [09:49:47] can I hope for Zuul to get unconfused eventually? [09:49:56] or should I “just” force-merge the WikibaseQualityConstraints change too [09:49:58] ? [09:50:42] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseQualityConstraints/+/1082081 ? [09:50:46] (shuffling the Wikibase dependencies around between builds won’t stop the same general problem from happening in other sets of extensions…) [09:50:53] yes, that’s the one that still needs to go in [09:51:06] hmm it depends on https://gerrit.wikimedia.org/r/q/I67ce14dfe0a95f67a0d86a3eee76cc5a8bddfb97 [09:51:07] (and then I’ll want to backport it too, so if you have a way to do that without force-merge I’m all ears) [09:51:10] one got merged [09:51:12] the other is abandoned [09:51:17] and I guess that is what blocking [09:51:24] yes, I thought the backport might be confusing zuul, that’s why I abandoned it [09:51:39] but normally zuul is smart enough to keep different branches apart, isn’t it? [09:52:15] nop [09:52:22] I have deleted the abandoned change [09:52:26] lets see whether that fixes it [09:52:40] yeah [09:52:42] yay [09:52:59] so I gerrited myself into a hole that I couldn’t gerrit my way out of? [09:53:05] because I don’t think I have the ability to delete changes [09:54:37] I think it queries the dependencies then check whether they are merged [09:54:42] and an abandoned change is NOT merged [09:54:55] it thus think, that dependent change is still pending [09:55:09] :D [09:55:13] :-\ [09:55:16] something something [09:55:24] * MichaelG_WMF is reading up [09:55:49] okay, so what’s the best way to backport this mess [09:56:00] because at least the WikibaseQualityConstraints change needs to get into wmf.28, I think [09:56:11] that required actual fixes in the extension code, not just test fixes [09:56:20] so it would probably break in production [09:56:33] but if I backport it on its own CI is going to fail due to WikibaseLexeme [09:59:35] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.28 deployment blockers - https://phabricator.wikimedia.org/T375659#10249389 (10Lucas_Werkmeister_WMDE) [10:01:05] for the record the backport change I have deleted was https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexeme/+/1082160 (now a 404) [10:02:30] so at least https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseQualityConstraints/+/1082081 [10:02:34] that one should merge [10:03:06] and since the core change has made it to this week wmf release branch, we need both Wikibase* patches to be proposed to the wmf branch [10:03:18] and I guess we will submit them both ignoring Zuul [10:03:21] or well push them [10:03:30] I just left a summary comment at https://phabricator.wikimedia.org/T377778 [10:04:04] if I understand correctly, force-merging one and +2ing the other change with Depends-On should work, now that the extra confusing backport is gone [10:04:09] but I’m not sure if we want to try that [10:04:14] or just force-merge both as you say [10:04:24] we can do that this afternoon if you want [10:04:48] yeah, I should be free during the backport+config window [10:05:02] but I’ll wait for the WBQC master change to merge before creating the backports [10:05:43] and we can do it before or after the backport window [10:05:45] it is fine to me [10:06:28] there’s another window right before the backport window [10:06:32] and mediawiki infrastructure right now [10:06:42] 13:00 UTC+2 looks free (in one hour), should we try it then? [10:10:53] yay, WBQC master change merged [10:12:06] I’ll make the two backports now [10:12:10] and drop Depends-On from the WBQC one [10:14:32] uploaded https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexeme/+/1082180 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseQualityConstraints/+/1082181 [10:16:11] I’ll go make lunch and hopefully be back in 45 minutes [10:30:20] Yippee, build fixed! [10:30:20] Project mediawiki-core-doxygen build #5154: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/5154/ [10:30:30] (03open) 10mvolz: Wikilink to deployment directions for Zotero / Citoid in the calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/124 [10:30:54] (03update) 10mvolz: Wikilink to deployment directions for Zotero / Citoid in the calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/124 [10:32:15] Yippee, build fixed! [10:32:16] Project beta-update-databases-eqiad build #79809: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79809/ [10:47:29] I am off for lunch [10:47:39] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexeme/+/1082180 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseQualityConstraints/+/1082181 can be pushed [10:47:43] s/can/should/ [10:47:52] I am not sure whether the permissions allow that though :D [10:47:53] Project beta-scap-sync-world build #177406: 04FAILURE in 2 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177406/ [10:47:58] I think wmf-deployment members can [10:58:17] Yippee, build fixed! [10:58:17] Project beta-scap-sync-world build #177407: 09FIXED in 2 min 35 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177407/ [11:05:45] (03update) 10jnuche: remove plugins abstraction from Scap [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/485 [11:12:25] (03merge) 10jnuche: remove plugins abstraction from Scap [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/485 [11:28:01] (03open) 10jnuche: housekeeping [repos/releng/scap3-dev] - 10https://gitlab.wikimedia.org/repos/releng/scap3-dev/-/merge_requests/42 [11:28:54] (03merge) 10jnuche: housekeeping [repos/releng/scap3-dev] - 10https://gitlab.wikimedia.org/repos/releng/scap3-dev/-/merge_requests/42 [11:59:29] (03merge) 10aklapper: Disable publishing notifications for new repositories by default [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/75 (https://phabricator.wikimedia.org/T143162) [12:05:05] hasharLunch: FYI, those backports are deployed onw [12:05:07] *now [12:05:16] awesome [12:12:04] 10Diffusion, 10Phabricator (phabricator-next), 10Release-Engineering-Team (Yak Shaving 🐃🪒), 07Developer Productivity: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162#10249943 (10Aklapper) >>! In T143162#10061966, @Aklapper wrote:... [12:26:02] 10Diffusion, 10Phabricator (phabricator-next), 10Release-Engineering-Team (Yak Shaving 🐃🪒), 07Developer Productivity: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162#10249992 (10Aklapper) I disabled publishing in those 140 repos w... [12:26:50] Lucas_WMDE: thank you for taking care of the patches [12:27:21] np, thanks for figuring out how to get them through [13:38:56] [2024-10-22T13:28:05.654Z] [INFO] > FORCE_COLOR=1 eslint eslint.config.js src/main/resources -f eslint-formatter-multiple [13:38:56] [2024-10-22T13:28:05.761Z] [INFO] 'FORCE_COLOR' is not recognized as an internal or external command, [13:38:56] [2024-10-22T13:28:05.761Z] [INFO] operable program or batch file. [13:39:02] way to Windows! [13:41:46] (03open) 10aklapper: Update mimemailparser from May 2011 version to 8.0.4 [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/76 [13:43:24] PowerShell probably wants you to write something like [13:43:24] With-Environment-Variable-Set-To-String-Run-Command FORCE_COLOR 1 eslint eslint.config.js … [13:44:43] I went with `eslint --color` :D [13:45:02] I think composer is able to transform them on the flyfor window [13:45:03] how facile :P [13:45:23] converting VAR=value to whatever Windows magic requires ( SET /? var=value ???) [13:45:44] huh, ok [13:45:59] that sounds cool in theory and fragile in practice ^^ [13:46:08] depends on how complete their `sh` syntax parser is I guess [14:16:24] 10Phabricator maintenance bot, 06Content-Transform-Team-WIP, 10Wiki-Setup, 07OKR-Work, 13Patch-For-Review: Add a new checklist item to the Wiki creation process for Parsoid Read Views - https://phabricator.wikimedia.org/T376827#10250393 (10MSantos) p:05Triage→03High a:03MSantos [14:16:36] (03update) 10brennen: Add temporary debug output for T374441 [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/74 (https://phabricator.wikimedia.org/T374441) (owner: 10aklapper) [14:17:39] (03approved) 10brennen: Update mimemailparser from May 2011 version to 8.0.4 [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/76 (owner: 10aklapper) [14:17:55] (03approved) 10brennen: Add temporary debug output for T374441 [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/74 (https://phabricator.wikimedia.org/T374441) (owner: 10aklapper) [14:18:13] (03merge) 10aklapper: Update mimemailparser from May 2011 version to 8.0.4 [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/76 [14:20:41] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2024-10-22 - https://phabricator.wikimedia.org/T377850 (10brennen) 03NEW [14:28:27] (03open) 10brennen: update submodules for 2024-10-22 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/57 (https://phabricator.wikimedia.org/T377850) [14:42:12] (03PS21) 10Ebomani: Updating Patch Demo plugin to return legacy/new URL as needed and modifying tests to reflect current process. [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1079624 (https://phabricator.wikimedia.org/T374954) [14:42:12] (03CR) 10Ebomani: "Hi Antoine, thank you so much for the detailed comments, great catch on the legacy bit too! I agree with your thoughts on combining the re" [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1079624 (https://phabricator.wikimedia.org/T374954) (owner: 10Ebomani) [14:42:56] (03approved) 10dancy: jobs: Pass inline scap config to avoid any mwbuilder sudo'ing [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/90 (owner: 10dduvall) [14:43:57] (03update) 10dancy: Wikilink to deployment directions for Zotero / Citoid in the calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/124 (owner: 10mvolz) [14:48:08] (03CR) 10Hashar: [C:03+2] Updating Patch Demo plugin to return legacy/new URL as needed and modifying tests to reflect current process. [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1079624 (https://phabricator.wikimedia.org/T374954) (owner: 10Ebomani) [14:48:48] (03Merged) 10jenkins-bot: Updating Patch Demo plugin to return legacy/new URL as needed and modifying tests to reflect current process. [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1079624 (https://phabricator.wikimedia.org/T374954) (owner: 10Ebomani) [14:50:16] (03update) 10dancy: SpiderPig API server [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [14:52:05] (03merge) 10brennen: update submodules for 2024-10-22 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/57 (https://phabricator.wikimedia.org/T377850) [14:54:40] 10Phabricator (phabricator-next): Restrict editing task priority to Trusted-Contributors/staff/etc - https://phabricator.wikimedia.org/T363337#10250679 (10Aklapper) [15:06:53] 10Phabricator (2024-10-22), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Deploy Phabricator/Phorge 2024-10-22 - https://phabricator.wikimedia.org/T377850#10250740 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=844ed657-55d7-4a30-9688-3f9fd4438a2c) set by eo... [15:10:15] the magic of the day: figuring out how I can rebase a pull request sent by a third party [15:11:20] `@dependabot rebase` [15:11:22] :D [15:14:32] (03update) 10dancy: SpiderPig API server [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [15:28:18] Project beta-scap-sync-world build #177434: 04FAILURE in 2 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177434/ [15:29:06] what now [15:29:15] 16:28:17 [12 hits] PHP Notice: Undefined variable: namespace [15:29:47] not completely useful error message... [15:32:24] 10Scap: scap log doesn't always provide enough information on an error - https://phabricator.wikimedia.org/T377866 (10Reedy) 03NEW [15:32:31] Yeah, I had the same issue last night [15:32:38] I was able to work it out when I saw the db update log [15:33:07] 10Scap: scap log doesn't always provide enough information on an error - https://phabricator.wikimedia.org/T377866#10250986 (10Reedy) [15:36:38] 10Beta-Cluster-Infrastructure, 10Scap: scap log doesn't always provide enough information on an error - https://phabricator.wikimedia.org/T377866#10250990 (10Reedy) [15:37:11] 10Beta-Cluster-Infrastructure, 10Scap: scap log doesn't always provide enough information on an error - https://phabricator.wikimedia.org/T377866#10251004 (10Reedy) [15:39:07] Yippee, build fixed! [15:39:07] Project beta-scap-sync-world build #177435: 09FIXED in 2 min 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/177435/ [15:39:16] That's... less than helpful [15:42:23] magically fixed? [15:43:49] https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/518594/console shows some new translations coming in, but seems unlikely it actually fixed whatever was the issue [15:45:25] 10Gerrit, 10Catalyst (Camp Muir): Update gerrit plugin for PatchDemo - https://phabricator.wikimedia.org/T374954#10251061 (10EBomani) 05Open→03Resolved Now works in all cases, including when there both Legacy Patchdemo and new Patchdemo links as shown in the attached image. {F57633912} [16:19:00] (03PS1) 10Dduvall: Fix stacktrace blame links [releng/phatality] - 10https://gerrit.wikimedia.org/r/1082242 [16:41:42] 10GitLab (Auth & Access), 10Phabricator, 06Trust-and-Safety: Account recovery help needed for Developer account jgiannelos - https://phabricator.wikimedia.org/T377117#10251282 (10taavi) [16:53:10] (03open) 10dancy: build-images.py: Remove hard-coded reference to host /srv/mediawiki-staging [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/125 [16:53:12] (03update) 10dancy: build-images.py: Remove hard-coded reference to host /srv/mediawiki-staging [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/125 [16:56:16] (03merge) 10dancy: build-images.py: Remove hard-coded reference to host /srv/mediawiki-staging [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/125 [16:59:47] (03CR) 10Ahmon Dancy: [C:03+1] Fix stacktrace blame links [releng/phatality] - 10https://gerrit.wikimedia.org/r/1082242 (owner: 10Dduvall) [17:00:15] 10Continuous-Integration-Infrastructure, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Grant bd808 membership in the contint-roots and contint-docker groups - https://phabricator.wikimedia.org/T377792#10251358 (10Dzahn) This all looks ready to go except I think it needs approval from @Bmueller as the m... [17:13:47] 10Gerrit, 06collaboration-services: Add support for abuse blocking to Gerrit - https://phabricator.wikimedia.org/T351817#10251467 (10Dzahn) Nowadays this is mostly a duplicate of T348734 (which has recent activity!) and T365259. [17:14:50] 10Gerrit, 06collaboration-services: Add support for abuse blocking to Gerrit - https://phabricator.wikimedia.org/T351817#10251481 (10Dzahn) [17:17:44] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [17:22:44] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [17:36:46] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [17:38:41] 10MediaWiki-Releasing, 05MW-1.43-release: Branch REL1_43 for MediaWiki and all extensions and skins - https://phabricator.wikimedia.org/T372318#10251625 (10ABreault-WMF) 05Open→03Resolved [17:39:19] 10Continuous-Integration-Infrastructure, 06collaboration-services: allow mwmaint/cumin hosts to connect to http on contint - https://phabricator.wikimedia.org/T340788#10251627 (10Dzahn) >>! In T340788#8991308, @hashar wrote: > I'd prefer to stick to only allowing http traffic from ATS/Varnish caches to avoid u... [17:44:26] (03open) 10dancy: Add missing DEBIAN_FRONTEND=noninteractive to apt-get calls [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/126 [17:44:28] (03update) 10dancy: Add missing DEBIAN_FRONTEND=noninteractive to apt-get calls [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/126 [17:46:46] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [17:47:11] (03merge) 10dduvall: jobs: Pass inline scap config to avoid any mwbuilder sudo'ing [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/90 [17:56:48] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap, 06SRE Observability: Scap prometheus migration: Reduce the cardinality of scap timers/statsd metrics - https://phabricator.wikimedia.org/T377883 (10thcipriani) 03NEW [17:57:52] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.43-release: Add REL1_43 to ExtensionDistributor as the development snapshot - https://phabricator.wikimedia.org/T372322#10251687 (10MacFan4000) a:03MacFan4000 [18:00:43] 10Phabricator (2024-10-22), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2024-10-22 - https://phabricator.wikimedia.org/T377850#10251696 (10Aklapper) 05Open→03Resolved a:03brennen [18:02:39] 10Diffusion, 10Phabricator (2024-10-22), 10Release-Engineering-Team (Yak Shaving 🐃🪒), 07Developer Productivity: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162#10251702 (10Aklapper) 05Open→03Resolved This (not emitting not... [18:06:48] (03open) 10dancy: make-container-image/build_image_incr.py: Check for empty rsync to default base image [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/127 [18:06:50] (03update) 10dancy: make-container-image/build_image_incr.py: Check for empty rsync to default base image [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/127 [18:08:13] (03update) 10dancy: make-container-image/build_image_incr.py: Check for empty rsync to default base image [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/127 [18:10:11] (03approved) 10dduvall: make-container-image/build_image_incr.py: Check for empty rsync to default base image [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/127 (owner: 10dancy) [18:10:29] (03merge) 10dancy: make-container-image/build_image_incr.py: Check for empty rsync to default base image [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/127 [18:12:00] (03open) 10dduvall: prep: Support `--reference` argument for user-supplied git reference dir [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/502 [18:12:27] (03update) 10dduvall: prep: Support `--reference` argument for user-supplied git reference dir [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/502 [18:12:28] (03update) 10dduvall: prep: Support `--reference` argument for user-supplied git reference dir [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/502 [18:15:04] (03approved) 10dancy: prep: Support `--reference` argument for user-supplied git reference dir [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/502 (owner: 10dduvall) [18:18:39] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#10251748 (10Dzahn) >>! In T338470#8949865, @LSobanski wrote: > This will most likely wait until (if ever) we migrate to Bookworm. We now have... [18:31:40] 10Release-Engineering-Team (Radar), 06collaboration-services, 06SRE, 06SRE-OnFire, 07Sustainability: Remove old scap repositories from deploy1002 - https://phabricator.wikimedia.org/T309162#10251781 (10Dzahn) >>! In T309162#8877794, @hashar wrote: > This was long forgotten. The problem is when a `Scap::T... [18:37:51] (03open) 10dduvall: build-images.py: Support a `--latest-tag` argument [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/128 [18:38:42] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [18:38:45] (03update) 10dduvall: build-images.py: Support a `--latest-tag` argument [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/128 [18:38:47] (03update) 10dduvall: build-images.py: Support a `--latest-tag` argument [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/128 [18:40:02] (03merge) 10dduvall: prep: Support `--reference` argument for user-supplied git reference dir [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/502 [18:44:32] (03approved) 10dancy: build-images.py: Support a `--latest-tag` argument [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/128 (owner: 10dduvall) [18:45:03] (03open) 10dduvall: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 [18:45:31] (03merge) 10dduvall: build-images.py: Support a `--latest-tag` argument [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/128 [18:45:45] (03update) 10dduvall: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 [18:45:47] (03update) 10dduvall: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 [18:45:50] (03update) 10dduvall: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 [18:45:52] (03update) 10dduvall: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 [18:45:55] (03update) 10dduvall: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 [18:47:50] (03approved) 10dancy: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 (owner: 10dduvall) [18:48:00] (03merge) 10dduvall: build-images: Support `--latest-tag` argument [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/503 [18:48:57] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [18:51:10] (03open) 10dduvall: Release 4.116.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/504 [18:51:45] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#10251815 (10Dzahn) There is more to this since the whole gerrit site dir is based on the system user, as in: ` profile:... [18:52:21] (03merge) 10dduvall: Release 4.116.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/504 [18:53:04] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#10251819 (10Dzahn) This task would be simplified quite a bit if we would limit it to reserving the global UID/GID but dro... [18:54:16] (03PS1) 10Michael Große: Automoderator: Make CommunityConfiguration a dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1082263 (https://phabricator.wikimedia.org/T374428) [18:54:17] (03CR) 10Michael Große: "In addition to this change I notice: Automoderator is not part of the gated extensions in CI: Should it be?" [integration/config] - 10https://gerrit.wikimedia.org/r/1082263 (https://phabricator.wikimedia.org/T374428) (owner: 10Michael Große) [18:55:19] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#10251821 (10Dzahn) also Bacula backup sets: ` modules/profile/manifests/backup/filesets.pp: includes => [ '/srv/... [18:57:15] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#10251825 (10Dzahn) a:05hashar→03Dzahn [18:58:34] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#10251826 (10Dzahn) Either way, this is definitely still a valid TODO: ` # TODO convert to systemd::sysuser user... [19:03:58] (03CR) 10Scardenasmolinar: [C:03+1] Automoderator: Make CommunityConfiguration a dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1082263 (https://phabricator.wikimedia.org/T374428) (owner: 10Michael Große) [19:08:21] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#10251863 (10Dzahn) While doing that I found we already reserved UID/GID 925 for "gerrit" in https://gerrit.wikimedia.org/... [19:08:45] (03open) 10dduvall: jobs: Use a git reference directory for mediawiki/core [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/91 [19:09:15] (03update) 10dduvall: jobs: Use a git reference directory for mediawiki/core [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/91 [19:09:17] (03update) 10dduvall: jobs: Use a git reference directory for mediawiki/core [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/91 [19:12:18] (03open) 10dduvall: jobs: Tag the latest single-version image with the version string [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/92 [19:12:24] (03update) 10dduvall: jobs: Tag the latest single-version image with the version string [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/92 [19:12:26] (03update) 10dduvall: jobs: Tag the latest single-version image with the version string [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/92 [19:19:15] (03open) 10dduvall: jobs: Disable IRC logging [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/93 [19:19:25] (03update) 10dduvall: jobs: Disable IRC logging [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/93 [19:19:26] (03update) 10dduvall: jobs: Disable IRC logging [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/93 [19:24:33] (03approved) 10dancy: jobs: Tag the latest single-version image with the version string [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/92 (owner: 10dduvall) [19:24:55] (03approved) 10dancy: jobs: Disable IRC logging [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/93 (owner: 10dduvall) [19:25:53] (03approved) 10dancy: jobs: Use a git reference directory for mediawiki/core [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/91 (owner: 10dduvall) [19:41:44] 10MediaWiki-Releasing, 05MW-1.43-release: Add REL1_43 to on-wiki documentation as the development snapshot - https://phabricator.wikimedia.org/T372334#10251971 (10ABreault-WMF) 05Open→03Resolved a:03ABreault-WMF https://www.mediawiki.org/w/index.php?title=Module%3AVersion&diff=6807127&oldid=6779082 [19:44:42] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [19:49:42] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [20:01:42] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [20:06:42] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [20:09:43] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [20:12:38] (03open) 10dancy: Move a message outside of TRAIN_DEV_NO_HINTS [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/103 [20:12:41] (03update) 10dancy: Move a message outside of TRAIN_DEV_NO_HINTS [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/103 [20:14:04] (03merge) 10dancy: Move a message outside of TRAIN_DEV_NO_HINTS [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/103 [20:14:16] (03open) 10dancy: Dockerfile.deploy: Don't VOLUME /var/lib/docker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/104 [20:14:19] (03update) 10dancy: Dockerfile.deploy: Don't VOLUME /var/lib/docker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/104 [20:14:34] (03update) 10dancy: Dockerfile.deploy: Don't VOLUME /var/lib/docker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/104 [20:19:43] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [20:20:17] (03open) 10dancy: Remove unused "deploy" subcommand [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/105 [20:20:19] (03update) 10dancy: Remove unused "deploy" subcommand [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/105 [20:20:52] (03merge) 10dancy: Remove unused "deploy" subcommand [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/105 [20:24:43] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [20:34:43] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [20:57:01] (03merge) 10dduvall: jobs: Use a git reference directory for mediawiki/core [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/91 [20:57:24] (03update) 10dduvall: jobs: Tag the latest single-version image with the version string [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/92 [20:57:46] (03merge) 10dduvall: jobs: Tag the latest single-version image with the version string [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/92 [20:57:58] (03update) 10dduvall: jobs: Disable IRC logging [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/93 [20:58:07] (03merge) 10dduvall: jobs: Disable IRC logging [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/93 [21:07:00] (03approved) 10dduvall: Dockerfile.deploy: Don't VOLUME /var/lib/docker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/104 (owner: 10dancy) [21:09:03] (03update) 10dancy: Dockerfile.deploy: Don't VOLUME /var/lib/docker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/104 [21:09:41] (03merge) 10dancy: Dockerfile.deploy: Don't VOLUME /var/lib/docker [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/104 [21:09:45] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [21:14:45] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [21:32:06] (03open) 10dancy: scap train: Fix bug in final visualization [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/505 [21:32:07] (03update) 10dancy: scap train: Fix bug in final visualization [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/505 [21:33:10] (03merge) 10dancy: scap train: Fix bug in final visualization [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/505 [21:36:19] (03open) 10dancy: Release 4.117.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/506 [21:37:20] (03merge) 10dancy: Release 4.117.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/506 [21:42:58] (03open) 10dancy: Revert "Dockerfile.deploy: Don't VOLUME /var/lib/docker" [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/106 [21:43:00] (03update) 10dancy: Revert "Dockerfile.deploy: Don't VOLUME /var/lib/docker" [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/106 [21:44:00] (03merge) 10dancy: Revert "Dockerfile.deploy: Don't VOLUME /var/lib/docker" [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/106 [21:51:45] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [21:57:00] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [22:00:30] 06Release-Engineering-Team, 06Data Products, 06Data-Platform-SRE, 10Dumps-Generation, and 2 others: Migrate current-generation dumps to run from our containerized images - https://phabricator.wikimedia.org/T352650#10252263 (10BTullis) >>! In T352650#10058764, @xcollazo wrote: > I think there were 2 ideas:... [22:13:47] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [22:19:02] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [23:36:57] (03open) 10dduvall: jobs: Clone local mediawiki/core reference repo with git commands [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/94 [23:37:06] (03update) 10dduvall: jobs: Clone local mediawiki/core reference repo with git commands [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/94 [23:37:07] (03update) 10dduvall: jobs: Clone local mediawiki/core reference repo with git commands [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/94