[00:40:28] <wikibugs>	 (03PS1) 10Krinkle: build: Enable gzip for dev/router.php [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072
[00:40:55] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] build: Enable gzip for dev/router.php [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072 (owner: 10Krinkle)
[00:41:56] <wikibugs>	 (03PS2) 10Krinkle: build: Enable gzip for dev/router.php [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072
[00:42:23] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] build: Enable gzip for dev/router.php [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072 (owner: 10Krinkle)
[00:43:04] <wikibugs>	 (03PS3) 10Krinkle: build: Enable gzip for dev/router.php [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072
[00:43:36] <wikibugs>	 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Create mw-web helmfile deployment - https://phabricator.wikimedia.org/T321900 (10Krinkle)
[00:43:44] <wikibugs>	 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Create mw-jobrunner helmfile deployment - https://phabricator.wikimedia.org/T321897 (10Krinkle)
[00:43:54] <wikibugs>	 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Create mw-videoscaler helmfile deployment - https://phabricator.wikimedia.org/T321899 (10Krinkle)
[00:44:05] <wikibugs>	 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Create mw-api-int helmfile deployment - https://phabricator.wikimedia.org/T321895 (10Krinkle)
[00:44:27] <wikibugs>	 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Create mw-api-ext helmfile deployment - https://phabricator.wikimedia.org/T321896 (10Krinkle)
[00:45:55] <wikibugs>	 (03CR) 10Krinkle: [C: 03+2] "I used this for the metrics at T322168, but had the router change uncommitted in a git-stash I popped on and off. Might as well commit it " [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072 (owner: 10Krinkle)
[00:46:06] <wikibugs>	 (03CR) 10Krinkle: [C: 03+2] build: Enable gzip for dev/router.php [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072 (owner: 10Krinkle)
[00:46:45] <wikibugs>	 (03Merged) 10jenkins-bot: build: Enable gzip for dev/router.php [integration/docroot] - 10https://gerrit.wikimedia.org/r/853072 (owner: 10Krinkle)
[02:53:06] <wikibugs>	 (03PS1) 10Deni: Zuul: Add Denis Smajlović (@deni), and company, to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/853078
[09:33:04] <Lucas_WMDE>	 Krinkle: the faster Zuul is definitely noticeable, very nice!
[10:20:30] <wikibugs>	 (03PS1) 10QChris: Allow “Gerrit Managers” to import history [libs/Bcp47Code] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/853242
[10:20:32] <wikibugs>	 (03CR) 10QChris: [V: 03+2 C: 03+2] Allow “Gerrit Managers” to import history [libs/Bcp47Code] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/853242 (owner: 10QChris)
[10:20:36] <wikibugs>	 (03PS1) 10QChris: Import done. Revoke import grants [libs/Bcp47Code] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/853243
[10:20:38] <wikibugs>	 (03CR) 10QChris: [V: 03+2 C: 03+2] Import done. Revoke import grants [libs/Bcp47Code] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/853243 (owner: 10QChris)
[10:25:16] <kostajh>	 hashar: I'll be around for a while if you want to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/843421
[10:31:24] <hashar>	 kostajh: I am finishing a call then will cook for the kids. Looks like this one is straightforward and I can do it this afternoon ;)
[10:59:40] <jbond>	 is there some issue with CI, changes from the following chain dont seem to be triggering tox-buster checks 
[10:59:43] <jbond>	 https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/835212/32
[11:33:11] <wikibugs>	 10Release-Engineering-Team (Radar), 10serviceops, 10User-Joe: Create jenkins job for creating deployment artifacts for `docker-pkg-deploy` - https://phabricator.wikimedia.org/T179562 (10jbond)
[11:55:20] <hashar>	 jbond: it might be broken indeed
[11:57:50] <hashar>	 jbond:  Exception: Dependency cycle detected: 835579 in ['835212', 835168, 835579]
[11:58:02] <hashar>	 some changes depend on something else which ends up in a loop
[11:58:11] <hashar>	 and Zuul silently discard that
[11:58:53] <hashar>	 maybe because https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/835579/  Depends-On itself ;)
[11:59:36] <jbond>	 hashar: thanks i have removed that dependency (the actual dependency is merged now)
[12:00:06] <jbond>	 hashar: btw where did yuo get that exception from?
[12:00:41] <hashar>	 jbond: from contint.wikimedia.org by grepping the change 835212 in /var/log/zuul/debug.log
[12:00:48] <jbond>	 great thanks
[12:01:03] <hashar>	 bonus if one day someone find a way to relay all those logs to our ELK stack :-]
[12:01:15] <hashar>	 kostajh: I have updated the two jobs 
[12:01:32] <jbond>	 :) cwh.ite would be the best person to help with that i think 
[12:01:36] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "I have updated the jobs:" [integration/config] - 10https://gerrit.wikimedia.org/r/843421 (https://phabricator.wikimedia.org/T320935) (owner: 10Kosta Harlan)
[12:03:33] <wikibugs>	 (03Merged) 10jenkins-bot: jjb: Deployed version of Quibble has QUIBBLE_APACHE flag [integration/config] - 10https://gerrit.wikimedia.org/r/843421 (https://phabricator.wikimedia.org/T320935) (owner: 10Kosta Harlan)
[12:11:10] <kostajh>	 hashar: thanks!
[12:52:57] <wikibugs>	 (03PS2) 10Hashar: Move custom CSS style to a standalone plugin [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/853057 (https://phabricator.wikimedia.org/T319378)
[12:53:01] <wikibugs>	 (03PS2) 10Hashar: Move custom links to a standalone plugin [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/853058 (https://phabricator.wikimedia.org/T319378)
[13:19:36] <wikibugs>	 10Project-Admins: Create project tag for MediaWiki-libs-Bcp47Code - https://phabricator.wikimedia.org/T322381 (10cscott) https://www.mediawiki.org/w/index.php?title=Gerrit/New_repositories/Requests/Entries&type=revision&diff=5573016&oldid=5572563&diffmode=source
[13:20:08] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: find a way to systematically update the deployment server name across all repos - https://phabricator.wikimedia.org/T197470 (10jbond)
[13:26:42] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team, 10SRE, 10serviceops: operations/docker-images/production-images has no CI - https://phabricator.wikimedia.org/T283855 (10jbond)
[13:32:22] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Scap, 10serviceops: Re-imaged mw app servers can end up with missing l10n cache for old versions of MW needed for rollback - https://phabricator.wikimedia.org/T273334 (10jbond)
[13:32:51] <wikibugs>	 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10serviceops-collab: Migrate GitLab Shared Runners from profile::gitlab::runner to role::gitlab_runner - https://phabricator.wikimedia.org/T322409 (10Jelto)
[13:35:14] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Radar), 10serviceops-collab, 10serviceops-radar, 10Sustainability (Incident Followup): Phabricator: Unable to view tasks in DB read-only mode - https://phabricator.wikimedia.org/T313879 (10jbond)
[13:40:06] <James_F>	 Krinkle: Aha, it's all *your* fault that the Zuul status page now looks so bad. ;-)
[13:43:25] <wikibugs>	 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10serviceops-collab: Self-build and publish buildkit helper images - https://phabricator.wikimedia.org/T321316 (10Jelto) In the last IC sync meeting we discussed that it makes more sense to add the self-hosted `dockerfile/copy` image to [production-ima...
[13:57:41] <Krinkle>	 James_F: hm.. css cache bug?
[14:02:19] <James_F>	 Krinkle: No, I meant that we'd moved from Bootstrap to Wikimedia UI so the "failure"/"success" tablets are bigger, bolder, and more eye-catching.
[14:03:00] <James_F>	 (Which is "right" for the WikimediaUI use case, but that's because we don't have a use case with dozens of passing/failing status tablets.)
[14:04:47] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog 📥), 10MediaWiki-Core-Tests, 10Code-Health, and 7 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10kostajh) Three years later: maybe we should target a more realistic nu...
[14:05:35] <wikibugs>	 10Scap: Completely port l10nupdate to scap - https://phabricator.wikimedia.org/T133913 (10jbond)
[14:09:37] <Krinkle>	 James_F: ah bad for showing bad test results, got it :D
[14:11:28] <Krinkle>	 there's some wiggleroom in how we interpret the style guide, e.g. a ligher pastel is also in the palette. I deemphased some other things as well so that adds to making the rest appear more prominent. e.g. the pipeline icons to the side were quite large before (the dots to the left of each box, smaller now), and the count badges and pieline descriptions are now dimmed as well.
[14:11:42] <Krinkle>	 de-emphasized 
[14:12:14] <James_F>	 Yeah, I wouldn't worry about it for now.
[14:12:29] <James_F>	 Longer-term we'll probably review the style guide for the same need on Wikifunctions.
[14:34:08] <wikibugs>	 (03PS1) 10Jforrester: [WIP] dockerfiles: Provide tox-bullseye, dropping Python 2.x etc. [integration/config] - 10https://gerrit.wikimedia.org/r/853301
[14:58:47] <wikibugs>	 (03PS1) 10Hashar: Add npm test to Gerrit deployment branch [integration/config] - 10https://gerrit.wikimedia.org/r/853305 (https://phabricator.wikimedia.org/T319378)
[14:58:53] <wikibugs>	 (03PS1) 10Hashar: build: add eslint for JavaScript plugins [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/853306 (https://phabricator.wikimedia.org/T319378)
[15:07:00] <wikibugs>	 (03Abandoned) 10Hashar: Zuul: Add Denis Smajlović (@deni), and company, to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/853078 (owner: 10Deni)
[15:12:09] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog 📥), 10MediaWiki-Core-Tests, 10Code-Health, and 7 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10hashar) {T287582} is a potential low hanging fruit.  The Wikibase Sele...
[15:17:43] <wikibugs>	 10Beta-Cluster-Infrastructure, 10SRE: "Obama" page on Beta Cluster often responds with 500 or 503 - https://phabricator.wikimedia.org/T188913 (10jbond) 05Open→03Resolved a:03jbond Boldly closing this task as we have not had any recent reports.  please re-open if this is still an issue
[15:35:57] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10serviceops-collab: contint hardware refresh - https://phabricator.wikimedia.org/T294276 (10LSobanski) p:05Triage→03High
[15:36:03] <wikibugs>	 10Continuous-Integration-Infrastructure, 10SRE, 10serviceops, 10serviceops-collab: contint1002 service implementation tracking - https://phabricator.wikimedia.org/T313832 (10LSobanski) p:05Triage→03High
[15:36:56] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Bonus Level 🕹️), 10serviceops-collab, 10Patch-For-Review: decom phab2001 - https://phabricator.wikimedia.org/T322250 (10LSobanski) p:05Triage→03Medium
[15:40:01] <wikibugs>	 10Scap: scap deploy --init on deployment server fails on first puppet run - https://phabricator.wikimedia.org/T257317 (10jbond)
[15:40:09] <wikibugs>	 10Scap, 10SRE: scap sudo violation on first puppet run - https://phabricator.wikimedia.org/T185189 (10jbond)
[15:41:24] <wikibugs>	 10GitLab (Administration, Settings & Policy), 10serviceops-collab: Configure a default cleanup policy for GitLab package registry - https://phabricator.wikimedia.org/T315877 (10LSobanski) p:05Triage→03Low
[15:42:25] <wikibugs>	 10GitLab (Administration, Settings & Policy), 10serviceops-collab: Configure a default cleanup policy for GitLab package registry - https://phabricator.wikimedia.org/T315877 (10LSobanski) p:05Low→03Medium
[15:43:40] <wikibugs>	 10Continuous-Integration-Infrastructure, 10SRE, 10serviceops, 10serviceops-collab: contint1002 service implementation tracking - https://phabricator.wikimedia.org/T313832 (10hashar) @Jnuche we will have to setup a spare Jenkins and a Zuul merger on this new host contint1002 :-)
[15:49:42] <wikibugs>	 10Continuous-Integration-Infrastructure, 10SRE, 10serviceops, 10serviceops-collab: contint1002 service implementation tracking - https://phabricator.wikimedia.org/T313832 (10jnuche) @hashar sounds like a good opportunity to pair!
[15:52:19] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "Thanks, that should be included in the next Deployment calendar generation ;)" [tools/release] - 10https://gerrit.wikimedia.org/r/849112 (owner: 10Hashar)
[15:54:16] <wikibugs>	 (03Merged) 10jenkins-bot: Add RESTBase to Content Transform Team window [tools/release] - 10https://gerrit.wikimedia.org/r/849112 (owner: 10Hashar)
[15:54:39] <wikibugs>	 10GitLab, 10Release-Engineering-Team, 10Data-Engineering, 10serviceops-collab: Experiencing pipeline failure due to disk-space issues - https://phabricator.wikimedia.org/T310593 (10Jelto) 05Resolved→03Open CI builds fail again with `No space left on device`. See: https://gitlab.wikimedia.org/repos/rele...
[16:17:10] <wikibugs>	 10GitLab (Integrations), 10Release-Engineering-Team, 10Wikimedia-GitHub: Mirror repositories hosted on our GitLab to GitHub - https://phabricator.wikimedia.org/T321597 (10hashar) > How to stop Gerrit from doing it: does archiving a repo in Gerrit mean it's still mirrored from Gerrit to GitHub if the GitHub r...
[16:20:39] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Puppet failure on phabricator-stage-1001.devtools - https://phabricator.wikimedia.org/T299997 (10hashar) Ah I see the intent. Then apparently it is not being used and since it is fully puppetize I am guessing it is easy to bring one back up if nee...
[16:20:50] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Puppet failure on phabricator-stage-1001.devtools - https://phabricator.wikimedia.org/T299997 (10hashar) 05Open→03Resolved
[16:31:59] <wikibugs>	 (03PS1) 10Ahmon Dancy: PrivateSettings.php: Add $wmgSwiftConfig [tools/train-dev] - 10https://gerrit.wikimedia.org/r/853371
[16:32:16] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] PrivateSettings.php: Add $wmgSwiftConfig [tools/train-dev] - 10https://gerrit.wikimedia.org/r/853371 (owner: 10Ahmon Dancy)
[16:32:45] <wikibugs>	 (03Merged) 10jenkins-bot: PrivateSettings.php: Add $wmgSwiftConfig [tools/train-dev] - 10https://gerrit.wikimedia.org/r/853371 (owner: 10Ahmon Dancy)
[16:53:52] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog 📥), 10MediaWiki-Core-Tests, 10Code-Health, and 7 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Krinkle) @hashar Afaik for PHPUnit we only install (and thus test) dep...
[17:03:49] <Krinkle>	 dancy: In CI of wmf-config afaik we use the readme.php dummy file to satisfy such private settings. Perhaps that's usable for train dev as it seems to re-implement the same kind of summy file
[17:04:14] <Krinkle>	 wmf-config.git:/private/readme.php that is.
[18:04:17] <dancy>	 Krinkle:   I assume you mean copy readme.php to PrivateSettings.php.  Does the wmf-config CI perform any modifications after making the copy?   I suppose I could make a PrivateSettings.php that includes readme.php and then makes customizations.
[18:09:39] <wikibugs>	 (03PS1) 10Ahmon Dancy: php7.2 to php7.4 [tools/train-dev] - 10https://gerrit.wikimedia.org/r/853390
[18:11:48] <Krinkle>	 dancy: ack yeah, I'd include it directly from the local untracked PS file
[18:12:06] <dancy>	 Makes sense.  Thanks for the advice.
[18:12:27] <Krinkle>	 and no in CI we use it as-is, no modications required. I don't know if for train we enable some private settings 'genuinely', if so you can append those after the include indeed :)
[18:12:30] <Krinkle>	 yw :)
[18:35:00] <wikibugs>	 (03PS1) 10Ahmon Dancy: train-dev: Clone scap from gitlab [tools/train-dev] - 10https://gerrit.wikimedia.org/r/853394
[18:41:59] <wikibugs>	 (03PS2) 10Ahmon Dancy: train-dev: Clone scap from gitlab [tools/train-dev] - 10https://gerrit.wikimedia.org/r/853394 (https://phabricator.wikimedia.org/T321847)
[19:38:41] <wikibugs>	 10GitLab, 10Release-Engineering-Team, 10Data-Engineering, 10serviceops-collab: Experiencing pipeline failure due to disk-space issues - https://phabricator.wikimedia.org/T310593 (10Dzahn) on runner-1021:   ` dzahn@runner-1021:~$ systemctl status clear-docker-cache.timer ● clear-docker-cache.timer - Periodi...
[21:10:48] <Daimona>	 Is anyone around who knows how to manually trigger the phan job for every deployed extension and would be willing to do that shortly?
[21:11:59] <Reedy>	 :popcorn:
[21:12:23] <Reedy>	 Daimona: Doesn't hashar usually just create a changeset in gerrit for this sort of thing?
[21:13:01] <Daimona>	 I've no idea :)
[21:13:12] <Reedy>	 He used to do mass "test ci" type stuff
[21:13:41] <Daimona>	 BTW, context is: I have a patch ready for https://phabricator.wikimedia.org/T320268, and I want to make sure we catch any new issues soon enough.
[21:14:50] <wikibugs>	 10Beta-Cluster-Infrastructure: Account creation broken on beta - https://phabricator.wikimedia.org/T322442 (10Daimona)
[21:18:48] <Daimona>	 Actually, there doesn't seem to be many extensions using ITextFormatter, so I'm just gonna do that manually.
[21:38:01] <hashar>	 Daimona: Reedy: I did mass trigger a while ago
[21:38:16] <hashar>	 nowadays that is more or less covered by the l10n-bot and LibUp randomly triggering pretty much everything
[21:38:28] <Daimona>	 Ah yeah, that makes sense
[21:38:50] <Daimona>	 I mean, in this specific I should have probably checked codesearch before even asking, but... :D
[21:39:15] <hashar>	 else I think Kunal had a script which would trigger the job using Gearman and injectng the parameters from zuul/parameters_functions.py
[21:39:31] <hashar>	 is that cause you envision a breaking change to the Phan job?
[21:39:45] <legoktm>	 zuul-test-repo :)
[21:39:54] <hashar>	 <3
[21:41:12] <hashar>	 oh it is in Puppet
[21:43:39] <hashar>	 Daimona: maybe we could have a huge Phan jobs which runs against core + vendor + extensions/skins deployed  and would trigger for mediawiki/core
[21:43:57] <hashar>	 anyway, we can find a way to trigger just the phan job, but not tonight it is too late for me
[21:47:40] <Daimona>	 It's just that I thought the new annotation could unmask some issues, and since they'd be security issues, I wanted to make sure we saw them asap. But since it's only 3 repos using ITextFormatter, I tested them locally
[21:47:58] <Daimona>	 So no need to do anything in CI
[21:51:03] <hashar>	 great :-]
[22:38:46] <wikibugs>	 10Gerrit, 10Gerrit-Privilege-Requests, 10cloud-services-team (Kanban): Clean up analytics/quarry/web ACL in Gerrit - https://phabricator.wikimedia.org/T289695 (10thcipriani) 05Open→03Resolved a:03thcipriani > My understanding is that cloud-services-team is working on / owning Quarry now, so presumably...
[23:03:23] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10serviceops, 10Release Pipeline (Blubber): WMF container registry does not accept a manifest list (aka OCI manifest index, or "fat" manifest) - https://phabricator.wikimedia.org/T322453 (10dduvall)
[23:06:19] <wikibugs>	 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10serviceops-collab: Self-build and publish buildkit helper images - https://phabricator.wikimedia.org/T321316 (10dduvall) This may be blocked by {T321316} which is preventing the publishing of multi-platform images. The `dockerfile-copy` image needs t...