[00:13:27] 10Release-Engineering-Team (Done by Feb 23🔥), 10Release, 10Train Deployments: 1.38.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T300199 (10thcipriani) p:05Triage→03Medium a:03dduvall [00:14:23] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.38.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T300200 (10thcipriani) p:05Triage→03Medium a:03brennen [00:14:54] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10thcipriani) p:05Triage→03Medium a:03dancy [00:15:27] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.38.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T300202 (10thcipriani) p:05Triage→03Medium a:03jeena [00:16:28] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.39.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T300203 (10thcipriani) p:05Triage→03Medium a:03hashar [05:59:37] (03PS1) 10Catrope: Codex: Pass the custom docroot to the npm command as an env variable [integration/config] - 10https://gerrit.wikimedia.org/r/763644 (https://phabricator.wikimedia.org/T293705) [08:48:33] James_F: why does deployment-docker-wikifunctions01 need a floating IP address? [09:05:45] 10Release-Engineering-Team (Done by Feb 23🔥), 10Scap, 10Documentation: scap help needs updating - https://phabricator.wikimedia.org/T301343 (10jnuche) 05Open→03Resolved a:03jnuche After completion of {T301716}, help output no longer shows `sync` as an available command: ` usage: scap [-h] .... [09:17:59] (03PS1) 10Legoktm: Add CheckUser as a OATHAuth dependency [integration/config] - 10https://gerrit.wikimedia.org/r/763695 [09:19:34] (03CR) 10Hashar: [C: 03+2] Add CheckUser as a OATHAuth dependency [integration/config] - 10https://gerrit.wikimedia.org/r/763695 (owner: 10Legoktm) [09:19:46] thank you hashar :D [09:19:51] :] [09:21:21] (03Merged) 10jenkins-bot: Add CheckUser as a OATHAuth dependency [integration/config] - 10https://gerrit.wikimedia.org/r/763695 (owner: 10Legoktm) [09:21:50] !log Reloading Zuul for I1494abb5e9e28da951ffb72154a074a16a0f8381 [09:21:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:32:10] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests: PHP notices should fail CI tests - https://phabricator.wikimedia.org/T302033 (10kostajh) I think this is more an issue with our PHPUnit setup and not necessarily #continuous-integration-config. Unit tests that generate PHP Notices do seem to get escalat... [11:50:19] 10Release-Engineering-Team (Done by Feb 23🔥), 10Release, 10Train Deployments: 1.38.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T300199 (10Zabe) [12:54:27] (03PS1) 10Btullis: Improve the datahub pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/763724 (https://phabricator.wikimedia.org/T301453) [12:56:11] (03PS2) 10Btullis: Improve the datahub pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/763724 (https://phabricator.wikimedia.org/T301453) [13:20:12] btullis: your CI patch has reached an extra level of awesomeness :D [13:21:39] (03CR) 10Hashar: [C: 03+2] "Excellent! I am always cheerful when people start using the branch filter. That is awesome :]" [integration/config] - 10https://gerrit.wikimedia.org/r/763724 (https://phabricator.wikimedia.org/T301453) (owner: 10Btullis) [13:23:50] (03CR) 10jerkins-bot: [V: 04-1] Improve the datahub pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/763724 (https://phabricator.wikimedia.org/T301453) (owner: 10Btullis) [13:27:45] 10Phabricator: Change Phabricator username - https://phabricator.wikimedia.org/T302022 (10rook) Merci! [13:35:41] (03CR) 10Lucas Werkmeister (WMDE): Fix time of utc_afternoon_training (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/763268 (owner: 10Lucas Werkmeister (WMDE)) [13:41:15] (03CR) 10Hashar: [C: 03+2] "The test suite failed since I forgot to create the Jenkins job." [integration/config] - 10https://gerrit.wikimedia.org/r/763724 (https://phabricator.wikimedia.org/T301453) (owner: 10Btullis) [13:43:06] (03Merged) 10jenkins-bot: Improve the datahub pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/763724 (https://phabricator.wikimedia.org/T301453) (owner: 10Btullis) [13:43:52] (03CR) 10Hashar: [C: 03+2] Improve the datahub pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/763724 (https://phabricator.wikimedia.org/T301453) (owner: 10Btullis) [13:44:18] !log Reloading Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/763724 T301453 [13:44:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:44:21] T301453: Create DataHub containers with deployment pipeline - https://phabricator.wikimedia.org/T301453 [13:44:43] hashar: Many thanks. Still trying to get my head around it all, but do you think it's going to work as intended? [13:44:58] btullis: I think that will do it yes ;) [13:45:22] one of the CI job running on integration/config does show a difference of the Zuul workflow before and after a proposed change [13:45:39] https://integration.wikimedia.org/ci/job/integration-config-zuul-layout-diff-docker/7779/console [13:45:55] that shows the job being added to postmerge with the "wmf" filter [13:45:58] which is the branch filter [13:46:14] the output should really says somtehing such as: BRANCH: wmf [13:59:54] Got it. Thanks agin. [14:23:02] https://usercontent.irccloud-cdn.com/file/IwNugyMB/image.png [14:24:01] We can delete these old jenkins jobs as well now, but that has to be a Jenkins admin, right? [14:49:06] btullis: yes correct! [14:49:21] I confess old jobs keep piling up, we manually delete them from time to time [15:11:32] 10Project-Admins: Mark the #Contributors-Team group as inactive - https://phabricator.wikimedia.org/T300558 (10Aklapper) 05Open→03Resolved a:03Aklapper Archived https://phabricator.wikimedia.org/project/manage/70/ - thanks everyone! [16:14:58] (03CR) 10Thcipriani: Fix time of utc_afternoon_training (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/763268 (owner: 10Lucas Werkmeister (WMDE)) [16:15:22] (03CR) 10Lucas Werkmeister (WMDE): Fix time of utc_afternoon_training (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/763268 (owner: 10Lucas Werkmeister (WMDE)) [16:26:06] (03PS2) 10Lucas Werkmeister (WMDE): Fix time of Thursday afternoon backport window [tools/release] - 10https://gerrit.wikimedia.org/r/763268 [16:26:08] (03PS2) 10Lucas Werkmeister (WMDE): Fix wikitext name of training windows [tools/release] - 10https://gerrit.wikimedia.org/r/763269 [16:26:10] (03PS1) 10Lucas Werkmeister (WMDE): Fix internal name of second training window [tools/release] - 10https://gerrit.wikimedia.org/r/763775 [16:26:16] ^ hope those changes make sense [16:26:47] thcipriani: moare deployment calendar tweaks above ^ :-] [16:27:57] I’m also tempted to remove these <%= window_name %> vars for the backport windows, as far as I can tell they’re always the same? [16:28:08] but maybe there’s already enough changes in the pipeline for now ^^ [16:30:29] (03CR) 10Thcipriani: [C: 03+2] "thanks 🎉" [tools/release] - 10https://gerrit.wikimedia.org/r/763268 (owner: 10Lucas Werkmeister (WMDE)) [16:31:58] (03CR) 10Thcipriani: [C: 03+2] Fix internal name of second training window [tools/release] - 10https://gerrit.wikimedia.org/r/763775 (owner: 10Lucas Werkmeister (WMDE)) [16:32:16] (03CR) 10Thcipriani: [C: 03+2] Fix wikitext name of training windows [tools/release] - 10https://gerrit.wikimedia.org/r/763269 (owner: 10Lucas Werkmeister (WMDE)) [16:32:34] Lucas_WMDE: thanks for cleaning up all my mistakes :) [16:32:40] no problem :D [16:32:51] I’ll update next week’s calendar on wiki then [16:33:06] the right way around this time, rather than my wrong guess from yesterday ^^ [16:33:17] some days I wonder if squinting at json is any better than squinting at wikitext for this sort of thing [16:33:18] (03Merged) 10jenkins-bot: Fix time of Thursday afternoon backport window [tools/release] - 10https://gerrit.wikimedia.org/r/763268 (owner: 10Lucas Werkmeister (WMDE)) [16:33:44] (03Merged) 10jenkins-bot: Fix internal name of second training window [tools/release] - 10https://gerrit.wikimedia.org/r/763775 (owner: 10Lucas Werkmeister (WMDE)) [16:33:46] (03Merged) 10jenkins-bot: Fix wikitext name of training windows [tools/release] - 10https://gerrit.wikimedia.org/r/763269 (owner: 10Lucas Werkmeister (WMDE)) [16:33:58] deployment calendar might be one layer of abstraction too far [16:34:17] a yak for me to shave another day :) [16:35:34] ah, you already fixed the on-wiki calendar, nevermind :D [16:39:57] yep, after you pointed out what to fix in the patch :P [16:40:07] (03PS1) 10Lucas Werkmeister (WMDE): Remove window_name vars from backport windows [tools/release] - 10https://gerrit.wikimedia.org/r/763780 [16:40:32] (03CR) 10Lucas Werkmeister (WMDE): "Just a suggestion, feel free to abandon if you disagree." [tools/release] - 10https://gerrit.wikimedia.org/r/763780 (owner: 10Lucas Werkmeister (WMDE)) [16:42:26] 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10dancy) [16:43:19] 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10dancy) @JMeybohm or @Jelto, can you use cumin to run "python3 -V" on production hosts that have scap installed? [16:43:48] 10Release-Engineering-Team, 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10dancy) [16:49:04] (03PS1) 10Ahmon Dancy: make-container-image: Mods for scap integration [tools/release] - 10https://gerrit.wikimedia.org/r/763783 [16:53:35] 10Release-Engineering-Team, 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10MoritzMuehlenhoff) There's still quite a few Stretch systems with scap in production: ` ===== NODE GROUP ===== (46) aqs[1004-1009].eqiad.wmnet,doc1001.eqiad.wmnet,ores[2001-2009]... [16:55:10] 10Release-Engineering-Team, 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10dancy) Thanks @MoritzMuehlenhoff. Can you point me to an existing "upgrade Stretch hosts" task that I can hang this one off of? [16:57:39] (03CR) 10Ahmon Dancy: [C: 03+2] make-container-image: Mods for scap integration [tools/release] - 10https://gerrit.wikimedia.org/r/763783 (owner: 10Ahmon Dancy) [17:01:53] 10Release-Engineering-Team, 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10MoritzMuehlenhoff) >>! In T302086#7721573, @dancy wrote: > Thanks @MoritzMuehlenhoff. Can you point me to an existing "upgrade Stretch hosts" task that I can hang this one off of?... [17:02:50] 10Release-Engineering-Team, 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10dancy) [17:04:32] (03PS2) 10Ahmon Dancy: make-container-image: Mods for scap integration [tools/release] - 10https://gerrit.wikimedia.org/r/763783 [17:30:32] (03CR) 10Ahmon Dancy: make-container-image: Mods for scap integration [tools/release] - 10https://gerrit.wikimedia.org/r/763783 (owner: 10Ahmon Dancy) [17:30:35] (03CR) 10Ahmon Dancy: [C: 03+2] make-container-image: Mods for scap integration [tools/release] - 10https://gerrit.wikimedia.org/r/763783 (owner: 10Ahmon Dancy) [17:33:24] (03Merged) 10jenkins-bot: make-container-image: Mods for scap integration [tools/release] - 10https://gerrit.wikimedia.org/r/763783 (owner: 10Ahmon Dancy) [17:34:39] (Queue (Jenkins jobs + Zuul functions) alert) firing: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [17:48:06] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:58:17] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:59:40] (Queue (Jenkins jobs + Zuul functions) alert) firing: (2) Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [18:19:39] (Queue (Jenkins jobs + Zuul functions) alert) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [19:00:17] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests: PHP notices should fail CI tests - https://phabricator.wikimedia.org/T302033 (10Umherirrender) `convertNoticesToExceptions` works for the test runs, but the init code from unit test or our own in `bootstrap.php` is not covered and the mention error is al... [19:03:02] (03PS1) 10Ahmon Dancy: Mods to webserver image build for scap [tools/release] - 10https://gerrit.wikimedia.org/r/763808 (https://phabricator.wikimedia.org/T297673) [19:04:27] 10Release-Engineering-Team, 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10Majavah) [19:04:40] 10Release-Engineering-Team, 10Scap, 10serviceops: Set scap minimum version to 3.7 - https://phabricator.wikimedia.org/T302086 (10Majavah) [19:05:31] (03CR) 10Ahmon Dancy: [C: 03+2] Mods to webserver image build for scap [tools/release] - 10https://gerrit.wikimedia.org/r/763808 (https://phabricator.wikimedia.org/T297673) (owner: 10Ahmon Dancy) [19:06:47] (03Merged) 10jenkins-bot: Mods to webserver image build for scap [tools/release] - 10https://gerrit.wikimedia.org/r/763808 (https://phabricator.wikimedia.org/T297673) (owner: 10Ahmon Dancy) [19:29:16] CI seems to be falling over from full disks? https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/136852/console [19:49:50] 10Phabricator: Rename Phabricator account Samtar to TheresNoTime - https://phabricator.wikimedia.org/T302111 (10Samtar) [20:07:27] 10Phabricator: Rename Phabricator account Samtar to TheresNoTime - https://phabricator.wikimedia.org/T302111 (10Aklapper) 05Open→03Resolved a:03Aklapper Done [20:09:59] Argh! [20:10:15] Kemayo: Looking into it [20:20:49] Looks like it was transient. [20:54:39] (Queue (Jenkins jobs + Zuul functions) alert) firing: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [21:04:27] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [21:14:07] (03PS1) 10Catrope: Codex: Add branchdeploy job as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) [21:14:20] the Zuul alarm is bunch of CR+2 done by James :) [21:14:43] thanks for letting us know it's known [21:19:56] (03PS2) 10Ahmon Dancy: WIP: build/push/deploy container images [tools/scap] - 10https://gerrit.wikimedia.org/r/763609 [21:29:41] dancy: Thanks for looking at it [21:33:10] hashar: why both jinxer-wm and icinga-wm [21:35:34] (03PS1) 10Ahmon Dancy: Kill off build-mw-image-loop.py [tools/train-dev] - 10https://gerrit.wikimedia.org/r/763846 (https://phabricator.wikimedia.org/T297673) [21:36:47] (03PS2) 10Ahmon Dancy: Kill off build-mw-image-loop.py [tools/train-dev] - 10https://gerrit.wikimedia.org/r/763846 (https://phabricator.wikimedia.org/T297673) [21:42:49] (03PS2) 10Jforrester: jjb: [codex-docker-doc-publish] Pass the custom docroot to the npm command [integration/config] - 10https://gerrit.wikimedia.org/r/763644 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [21:43:39] (03PS3) 10Jforrester: jjb: [codex-docker-doc-publish] Pass the custom docroot to the npm command [integration/config] - 10https://gerrit.wikimedia.org/r/763644 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [21:45:18] * James_F hashar: No, it was from the push of a stack of 12 patches to puppet. :-( [21:45:31] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/763644 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [21:50:29] (03PS4) 10Catrope: jjb: [codex-docker-doc-publish] Pass the custom docroot to the npm command [integration/config] - 10https://gerrit.wikimedia.org/r/763644 (https://phabricator.wikimedia.org/T293705) [21:50:59] (03CR) 10Jforrester: [C: 03+2] "…" [integration/config] - 10https://gerrit.wikimedia.org/r/763644 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [21:53:52] 10Release-Engineering-Team (Done by Feb 23🔥), 10Release, 10Train Deployments: 1.38.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T300199 (10Jdlrobson) [21:54:12] (03Merged) 10jenkins-bot: jjb: [codex-docker-doc-publish] Pass the custom docroot to the npm command [integration/config] - 10https://gerrit.wikimedia.org/r/763644 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:01:26] Oh, oops, forgot to log. [22:01:58] * James_F !log Jenkins: Defined BRANCHDEPLOY_AUTH_TOKEN_codex and BRANCHDEPLOY_SITE_ID_codex secrets for T293705 [22:04:01] James_F: you didn't log again [22:09:52] * James_F RhinosF1: I'm waiting for IRC to do something. [22:10:29] The waiting can be long then James_F :) [22:11:03] (03PS3) 10Ahmon Dancy: Add release_repo_build_and_push_images_cmd config parameter [tools/scap] - 10https://gerrit.wikimedia.org/r/763609 (https://phabricator.wikimedia.org/T297673) [22:11:59] * James_F Ah, so you got that message but not my !log one? [22:13:27] mainly all your messages except one are /me 'ed [22:13:41] ^ [22:13:43] ^ [22:14:04] Curious. [22:14:05] me log, you log, we all log [22:14:06] The reason is Friday evening [22:14:17] The reason is IRCCloud isn't being very nice? [22:14:21] James_F: that curious message was fine [22:14:26] !log Jenkins: Defined BRANCHDEPLOY_AUTH_TOKEN_codex and BRANCHDEPLOY_SITE_ID_codex secrets for T293705 [22:14:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:14:29] T293705: Investigate deploying an instance of the docs site per-patch - https://phabricator.wikimedia.org/T293705 [22:14:31] Huh. [22:14:33] You've gone normal again [22:14:33] Odd. [22:14:36] Strange [22:14:40] I did nothing. [22:14:45] * James_F shrugs. [22:14:50] (That was intentional.) [22:15:06] Maybe the stars were aligned in a weird way [22:16:28] (03PS2) 10Jforrester: jjb, Zuul: [branchdeploy-codex-node14-npm-docker] Create as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:16:32] (03PS3) 10Jforrester: jjb, Zuul: [branchdeploy-codex-node14-npm-docker] Create as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:20:37] * RhinosF1 is currently getting setup for when james makes all the branches [22:20:54] * James_F Ha, please no. [22:20:59] * James_F Did that come as a /me ? [22:21:02] Oh dear. [22:21:50] James_F: oops sorry mate :D [22:21:59] hashar: *hugs* [22:22:02] I confess I only looked at ONE change [22:22:11] and just assumed you went on a code review spree [22:22:17] I mean, I /am/. [22:22:26] But it was in a death-stall before I did so. [22:22:29] I should probably drop that alarm entirely, it is usefulness is doubtful [22:22:30] (Oops.) [22:22:34] :D [22:22:40] Well, it does tell us that CI is broken. [22:22:48] It just doesn't have anything people can meaningfully do about it. [22:23:01] I guess I could shell into the CI server and drop the jobs for the stack, but… [22:23:02] I have a plan to unbreak it though [22:23:18] Is the plan 'stop people from pushing lots of patches'? [22:23:32] (03CR) 10Jforrester: [C: 03+2] "jjb part deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:24:16] well at least make it for people to push bunch of patches without raising bunch of false alarms [22:24:23] * James_F nods. [22:24:39] (Queue (Jenkins jobs + Zuul functions) alert) firing: (2) Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [22:25:01] pls git be smart [22:25:17] RhinosF1: You must be new to git. ;-( [22:25:32] tss tss [22:26:00] James_F: I'm trying to juggle repos that are simultaneously 100s of commits ahead and behind at the same time [22:26:02] I am still discovering git subcommands and options after all this years [22:26:34] RhinosF1: `git nuke && git submodule update --recursive` on `extensions/` takes my box about 3 hours to run. [22:26:38] and bring them into some sensible enough mess that when you cut 1.38 on my birthday week, i don't have to spend my birthday dying [22:26:54] that's erm fun [22:27:44] (`git nuke` is a local alias for `stash && reset --hard && checkout -B master -t origin/master && pull`) [22:28:26] (03CR) 10Jforrester: [C: 03+2] "…" [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:28:45] Hmm, why isn't that triggering? [22:28:53] 3 HOURS ?! [22:29:01] hashar: We have 1800 repos. [22:29:33] James_F: I have about 300 extensions to argue with [22:29:40] oh for every single repo, yeah that might be a while [22:29:49] My local checkout of extensions.git is ~120 GiB. [22:29:57] RhinosF1: Yeah, that's not fun. [22:30:34] James_F: I'm trying to automate most of the stuff [22:30:36] at least for the git submodule update they can happen in parallel [22:30:46] Right. [22:30:47] but automate requires git to not cry [22:30:51] which never happens [22:30:56] [submodule] [22:30:56] fetchJobs = 8 [22:31:04] that is what I get in my .gitconfig [22:31:07] hashar: Can I deploy 763592 for you? [22:31:23] !gerrit 763592 [22:31:23] https://gerrit.wikimedia.org/r/#/q/763592 [22:31:41] Oh, sorry. The quibble-full-run log compresser. [22:31:44] James_F: sure if it plesaes you :) [22:31:59] there is some pending change to bump quibble from 1.3.0 to 1.4.0 but we are not sure about the aftermath [22:32:13] so I prefer to have the quibble developers around [22:32:20] Yeah, let's not. [22:32:28] Or we could do that on Monday during the holiday. [22:32:57] if it is an holiday in the US yeah that is very convenient [22:33:41] I love how the US seems to be constantly in holiday, that opens up a lot of maintenance windows [22:33:52] https://usercontent.irccloud-cdn.com/file/xegvEAx7/Screenshot%202022-02-18%20at%2022.33.44.png [22:33:56] * RhinosF1 cries [22:34:07] hmm [22:34:23] looks like something is completely off RhinosF1 ;) [22:34:54] (03PS2) 10Jforrester: jjb: compress quibble full run debug log [integration/config] - 10https://gerrit.wikimedia.org/r/763592 (owner: 10Hashar) [22:35:40] I like the view offered by `git log --oneline --decorate --graph --simplify-by-decoration` [22:35:49] (03CR) 10Jforrester: [C: 03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/763592 (owner: 10Hashar) [22:36:02] with the list of branches or alternatively --all to see everything [22:36:42] * James_F My day-to-day is `git log --topo-order --no-merges --oneline`. And yes, the inconsistency of `no-merges but `oneline` irritates me a lot. :-) [22:37:34] OH [22:37:58] reminds me I should one day write a propose to change the default submit strategy to `Rebase if necessary` [22:38:03] which will get rid of all the crazy merges [22:38:13] s/crazy/mostly useless/ [22:38:15] For integration/config? [22:38:23] for mediawiki/* [22:38:23] Or for all repos? [22:38:27] Ah, hmm. [22:38:41] (03Merged) 10jenkins-bot: jjb: compress quibble full run debug log [integration/config] - 10https://gerrit.wikimedia.org/r/763592 (owner: 10Hashar) [22:38:52] (03PS4) 10Jforrester: jjb, Zuul: [branchdeploy-codex-node14-npm-docker] Create as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:38:54] that is what we use on integration/config or puppet.git [22:39:32] and given our CI always tests against the tip of the branch and always submit for us, there is no need for those merge commits really [22:39:36] (03CR) 10Jforrester: [C: 03+2] "…" [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:39:45] Yeah. [22:39:54] +1 [22:40:00] done [22:40:04] hmm no wait [22:40:08] not done :-D [22:40:30] but that will be part of a proposal I am hoping to be able to write next week [22:40:46] Ack. [22:49:08] sleep & [22:49:32] (03CR) 10Jforrester: [C: 03+2] "…" [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:52:14] (03Merged) 10jenkins-bot: jjb, Zuul: [branchdeploy-codex-node14-npm-docker] Create as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/763841 (https://phabricator.wikimedia.org/T293705) (owner: 10Catrope) [22:54:05] !log Zuul: [branchdeploy-codex-node14-npm-docker] Create as experimental for T293705 [22:54:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:54:08] T293705: Investigate deploying an instance of the docs site per-patch - https://phabricator.wikimedia.org/T293705 [23:39:39] (Queue (Jenkins jobs + Zuul functions) alert) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [23:56:09] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1