[00:09:31] (03PS2) 10Jforrester: MediaWiki.Usage.DeprecatedConstantUsage: Replace DB_MASTER from 1.36.0+ [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/679917 (https://phabricator.wikimedia.org/T278026) [00:09:35] (03PS3) 10Jforrester: MediaWiki.Usage.DeprecatedConstantUsage: Replace DB_MASTER from 1.36.0+ [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/679917 (https://phabricator.wikimedia.org/T278026) [00:11:30] (03PS2) 10Jforrester: code: Rename and remove variables in FunctionCommentSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/679895 (owner: 10Umherirrender) [00:11:34] (03CR) 10Jforrester: [C: 03+2] code: Rename and remove variables in FunctionCommentSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/679895 (owner: 10Umherirrender) [00:13:31] (03Merged) 10jenkins-bot: code: Rename and remove variables in FunctionCommentSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/679895 (owner: 10Umherirrender) [01:00:03] (03CR) 10DannyS712: [C: 03+1] "Looks good to me, but since I haven't dealt with this sniff before, nor the code for checking if an extension supports a specific version " [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/679917 (https://phabricator.wikimedia.org/T278026) (owner: 10Jforrester) [02:05:54] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [02:10:42] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [03:40:46] 10LibUp: libup commit and upload upgrade twice for the same package on the same repo - https://phabricator.wikimedia.org/T280438 (10Legoktm) In theory https://gerrit.wikimedia.org/r/plugins/gitiles/labs/libraryupgrader/+/refs/heads/master/libup/tasks.py#132 is supposed to stop this from happening, but it is affe... [04:15:57] 10LibUp: libup commit and upload upgrade twice for the same package on the same repo - https://phabricator.wikimedia.org/T280438 (10Legoktm) For some reason I don't remember, checking whether an open patch exists is done when the patch is being created, rather than when we're trying to push it (the call to `gerr... [04:25:09] 10LibUp: New upstream release for PHP Parallel Lint - https://phabricator.wikimedia.org/T280474 (10LibUp-bot) [04:53:04] 10LibUp, 10Phabricator, 10Wikimedia-General-or-Unknown, 10Patch-For-Review: Create a release monitoring bot for Phabricator - https://phabricator.wikimedia.org/T205365 (10Legoktm) 05Open→03Resolved This is now enabled and will run every day. Once I finish the documentation I'll send an email to wikitec... [05:56:21] 10LibUp: New upstream release for covers-validator - https://phabricator.wikimedia.org/T280475 (10LibUp-bot) [05:59:47] 10LibUp: New upstream release for covers-validator - https://phabricator.wikimedia.org/T280475 (10Legoktm) 05Open→03Invalid [06:03:38] 10LibUp, 10Phabricator, 10Wikimedia-General-or-Unknown: Create a release monitoring bot for Phabricator - https://phabricator.wikimedia.org/T205365 (10Legoktm) Announced: https://lists.wikimedia.org/pipermail/wikitech-l/2021-April/094445.html Documented: https://www.mediawiki.org/w/index.php?title=Libraryupg... [07:05:16] so it looks like ruby tests are now broken on master for wikibase https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35870/console [07:20:07] It broke on April 16, before and after https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35863/console and https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35864/console [07:20:41] the gem dependencies are the same nothing has changed [07:20:55] it's very likely due to the buster upgrade in the ci [07:45:46] (03CR) 10Kosta Harlan: [C: 03+1] jjb: Use newer SonarScanner for codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/680384 (https://phabricator.wikimedia.org/T277527) (owner: 10Daimona Eaytoy) [09:23:47] 10Continuous-Integration-Infrastructure, 10Browser-Tests, 10User-Addshore: Centrally look for flakey browser tests - https://phabricator.wikimedia.org/T277205 (10Addshore) The last period saw an increase from 228 to 707. The top offenders would be: - ✖ "before all" hook - ✖ "before each" hook But there i... [10:42:45] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)), 10Scap, 10SRE, 10Python3-Porting: Porting scap to Python 3 - https://phabricator.wikimedia.org/T279628 (10LarsWirzenius) a:03LarsWirzenius [10:52:02] Can anyone from CI look https://gerrit.wikimedia.org/r/c/integration/config/+/680697? [10:58:01] 10Continuous-Integration-Infrastructure, 10Browser-Tests, 10User-Addshore, 10User-zeljkofilipin: Centrally look for flakey browser tests - https://phabricator.wikimedia.org/T277205 (10zeljkofilipin) [12:22:59] 10Release-Engineering-Team-TODO, 10Scap, 10serviceops: Deploy Scap version 3.17.0-1 - https://phabricator.wikimedia.org/T279695 (10LarsWirzenius) 05Open→03Invalid We're going to tag a new version with fixes, and open a new task for that. [12:27:14] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)), 10Scap, 10SRE, 10Python3-Porting: Porting scap to Python 3 - https://phabricator.wikimedia.org/T279628 (10LarsWirzenius) I was hoping we could retire Scap before WMF moves to bullseye, but @... [12:29:55] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)), 10Scap, 10SRE, 10Python3-Porting: Porting scap to Python 3 - https://phabricator.wikimedia.org/T279628 (10MoritzMuehlenhoff) >>! In T279628#7013821, @LarsWirzenius wrote: > I was hoping we c... [14:06:43] oh golly... scap is still python2? really? *sigh* [14:21:09] apergos: It’s tech debt all the way down. [14:21:31] 10Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)), 10Scap: Tag Scap release 1.37.1 and ask serviceops to deploy it to production - https://phabricator.wikimedia.org/T272145 (10LarsWirzenius) [14:54:08] 🐢🐢🐢🐢🐢 [14:55:39] isn't that turtles part of the way across? :P [14:56:27] hello, I know that "ruby selenium" tests might be a swear word, I wanted to point out the issue WMDE spotted today. Looks that with the updated of quibble containers to buster ones, Wikibase's old ruby tests started to break. We've managed to sort out most of the issues on ruby end. There still seems to be a problem with test runner failing due to lack of libav it intends to use for video recording. It seems ffmpeg is in [14:56:27] to be used (present in the container) but somehow in buster env it is not picked out. Anyone in the know how this could be fixed (I see no hashar today sadly) [14:57:13] see eg. https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35874/console, failure on PS2 of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996 [14:59:27] leszek_wmde: Might be worth asking James_F very nicely and sending bribes [14:59:59] are brexit jokes still accepted bribe currency? [15:10:19] James_F: pretty asking does the above aka https://phabricator.wikimedia.org/T280491 sound like something you could help with? [15:13:17] I think he switched most of them over [15:14:49] https://github.com/wikimedia/integration-config/blob/8de078298179aa43f3b03268cc9ddbe792365a24/dockerfiles/quibble-buster-php72-bundle/Dockerfile.template#L5 [15:14:54] >RUN {{ "ffmpeg build-essential rubygems-integration ruby ruby-dev bundler" | apt_install }} [15:19:45] yeah, we suspect something around this option not being set might be going on there https://github.com/leonid-shevtsov/headless/blob/master/lib/headless/video/video_recorder.rb#L12 [15:20:04] while it is not clear why it seeming did work in the old version [15:21:30] I wonder if the "dependancies" got messed up [15:22:19] leszek_wmde: Hey, was talking about this with Amir (and through him, Lucas). Sorry about this; I thought I'd not migrated Wikibase because I found the issue on Saturday. [15:24:01] !log Re-pushing mwselenium-quibble-docker back to master for T280491 [15:24:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:24:04] T280491: Ruby Browser Tests broken after OS update - https://phabricator.wikimedia.org/T280491 [15:25:21] James_F: it is okay, as long as we get this unstuck, we're absolutely happy [15:26:22] leszek_wmde: Should work now; seems I left CI prod in the test state and didn't revert successfully, sorry! [15:28:56] (03PS1) 10Jforrester: Zuul: Label each use of mwselenium-quibble-docker with a TODO [integration/config] - 10https://gerrit.wikimedia.org/r/681111 (https://phabricator.wikimedia.org/T220035) [15:32:24] (03PS5) 10Jforrester: jjb: Switch quibble 'bundle' jobs to buster [integration/config] - 10https://gerrit.wikimedia.org/r/674193 (https://phabricator.wikimedia.org/T278203) [15:32:41] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Migrate all CI jobs from stretch to buster or later and drop stretch testing support - https://phabricator.wikimedia.org/T278203 (10Jdforrester-WMF) [15:32:46] (03CR) 10Jforrester: [C: 03+2] Zuul: Label each use of mwselenium-quibble-docker with a TODO [integration/config] - 10https://gerrit.wikimedia.org/r/681111 (https://phabricator.wikimedia.org/T220035) (owner: 10Jforrester) [15:34:09] (03Merged) 10jenkins-bot: Zuul: Label each use of mwselenium-quibble-docker with a TODO [integration/config] - 10https://gerrit.wikimedia.org/r/681111 (https://phabricator.wikimedia.org/T220035) (owner: 10Jforrester) [15:38:46] thanks James_F: We'll continue on the task (we're almost ready to buster migration but not quite there yet) [15:41:02] leszek_wmde: Brilliant. Sorry again for the disruption. [16:12:26] (03PS1) 10Lars Wirzenius: Release 3.17.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/681122 [16:16:06] (03CR) 10Ahmon Dancy: [C: 03+1] Release 3.17.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/681122 (owner: 10Lars Wirzenius) [16:21:59] (03CR) 10Lars Wirzenius: [C: 03+2] Release 3.17.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/681122 (owner: 10Lars Wirzenius) [16:22:44] (03Merged) 10jenkins-bot: Release 3.17.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/681122 (owner: 10Lars Wirzenius) [16:33:53] (03CR) 10Ahmon Dancy: [C: 03+2] Fix interpolation for variables with dashes in name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/680433 (owner: 10Dduvall) [16:35:05] (03Merged) 10jenkins-bot: Fix interpolation for variables with dashes in name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/680433 (owner: 10Dduvall) [16:36:22] 10Release-Engineering-Team-TODO, 10Scap, 10serviceops: Deploy Scap version 3.17.0-1 - https://phabricator.wikimedia.org/T279695 (10LarsWirzenius) 05Invalid→03Open Re-opening and editing, because I'm lazy. [16:36:50] 10Release-Engineering-Team-TODO, 10Scap, 10serviceops: Deploy Scap version 3.17.1-1 - https://phabricator.wikimedia.org/T279695 (10LarsWirzenius) [16:37:16] 10Release-Engineering-Team-TODO, 10Scap, 10serviceops: Deploy Scap version 3.17.1-1 - https://phabricator.wikimedia.org/T279695 (10LarsWirzenius) 05Open→03Stalled [16:38:04] 10Release-Engineering-Team-TODO, 10Scap, 10serviceops: Deploy Scap version 3.17.1-1 - https://phabricator.wikimedia.org/T279695 (10LarsWirzenius) [16:38:26] 10Release-Engineering-Team-TODO, 10Scap, 10serviceops: Deploy Scap version 3.17.1-1 - https://phabricator.wikimedia.org/T279695 (10LarsWirzenius) [16:54:54] 10Release-Engineering-Team-TODO, 10Scap, 10serviceops: Deploy Scap version 3.17.1-1 - https://phabricator.wikimedia.org/T279695 (10LarsWirzenius) 05Stalled→03Open We've tagged the bug fix release 3.17.1 and tested it on the beta cluster. We'd like it be built as a .deb and deployed to production, when th... [16:58:32] (03PS1) 10Ahmon Dancy: RELEASE.md: Update the command used to install scap on beta cluster [tools/scap] - 10https://gerrit.wikimedia.org/r/681134 [17:01:54] (03CR) 10Lars Wirzenius: [C: 03+2] RELEASE.md: Update the command used to install scap on beta cluster [tools/scap] - 10https://gerrit.wikimedia.org/r/681134 (owner: 10Ahmon Dancy) [17:02:22] (03Merged) 10jenkins-bot: RELEASE.md: Update the command used to install scap on beta cluster [tools/scap] - 10https://gerrit.wikimedia.org/r/681134 (owner: 10Ahmon Dancy) [17:23:22] 10Beta-Cluster-Infrastructure, 10User-Urbanecm: Beta cron jobs seem broken - https://phabricator.wikimedia.org/T277206 (10Urbanecm) [17:35:24] 10Beta-Cluster-Infrastructure, 10User-Urbanecm: Beta cron jobs seem broken - https://phabricator.wikimedia.org/T277206 (10Urbanecm) I added this to the file (locally on mwmaint01): ` urbanecm@deployment-mwmaint01:~$ cat /etc/conftool-state/mediawiki.yaml primary_dc: labs urbanecm@deployment-mwmaint01:~$ ` th... [17:42:36] (03CR) 10Dduvall: Allow inline blubber configuration at the pipeline scope (031 comment) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674931 (owner: 10Dduvall) [17:44:48] (03PS4) 10Dduvall: Support default values in variable expressions [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674689 [17:46:18] 10Project-Admins: Create User-Carn project - https://phabricator.wikimedia.org/T280575 (10Carn) [17:46:34] 10Project-Admins: Create User-Carn project - https://phabricator.wikimedia.org/T280575 (10Carn) [17:46:36] 10Phabricator (Upstream), 10Upstream: Per-user projects for personal work in progress tracking - https://phabricator.wikimedia.org/T555 (10Carn) [17:47:58] (03PS2) 10Dduvall: Allow inline blubber configuration at the pipeline scope [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674931 [17:48:35] (03CR) 10Ahmon Dancy: [C: 03+1] "rebase looks good" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674689 (owner: 10Dduvall) [17:53:09] 10Beta-Cluster-Infrastructure, 10User-Urbanecm: Beta cron jobs seem broken - https://phabricator.wikimedia.org/T277206 (10Majavah) Puppet deletes that file, since Conftool is not supposed to be there. {T278007} would be the only proper fix, but since that isn't realistically happening now and people need these... [17:57:10] (03CR) 10Jeena Huneidi: [C: 03+2] Support default values in variable expressions [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674689 (owner: 10Dduvall) [17:58:13] (03Merged) 10jenkins-bot: Support default values in variable expressions [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674689 (owner: 10Dduvall) [17:58:19] !log apply hack (https://phabricator.wikimedia.org/T277206#7015609) to deployment-puppetmaster04 to unbreak maintenance scripts until we have conftool [17:58:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:00:39] (03CR) 10Jeena Huneidi: [C: 03+2] Allow inline blubber configuration at the pipeline scope [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674931 (owner: 10Dduvall) [18:01:45] (03Merged) 10jenkins-bot: Allow inline blubber configuration at the pipeline scope [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/674931 (owner: 10Dduvall) [18:32:16] (03PS1) 10Ahmon Dancy: RELEASE.md: Fix markup [tools/scap] - 10https://gerrit.wikimedia.org/r/681166 [18:33:28] (03CR) 10Ahmon Dancy: [C: 03+2] RELEASE.md: Fix markup [tools/scap] - 10https://gerrit.wikimedia.org/r/681166 (owner: 10Ahmon Dancy) [18:34:10] (03Merged) 10jenkins-bot: RELEASE.md: Fix markup [tools/scap] - 10https://gerrit.wikimedia.org/r/681166 (owner: 10Ahmon Dancy) [18:52:12] (03PS11) 10Umherirrender: Check for superfluous @return statements and missing void types [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/658070 [19:01:19] 10LibUp: libup commit and upload upgrade twice for the same package on the same repo - https://phabricator.wikimedia.org/T280438 (10Umherirrender) I am not sure if that race condition is tracked by the moved code. timeline * 2021-04-15 22:42:48 +0000 - The first patch set is created, the patch and the log is st... [19:17:12] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Patch-For-Review: Drop MediaWiki testing in stretch and instead test only in buster - https://phabricator.wikimedia.org/T252432 (10Dzahn) Thanks! fyi: mwdebug1003 (stretch debug... [20:47:28] 10LibUp: Add a `composer phpcs` command to all composer.json files - https://phabricator.wikimedia.org/T280592 (10Reedy) [20:52:02] 10LibUp: Add a `composer phpcs` command to all repos that require mediawiki/mediawiki-codesniffer - https://phabricator.wikimedia.org/T280592 (10Reedy) [20:57:36] hey releng! is there any (easy) way to change the link to the job that is run for a given repo? For example, i have a repo that is using the "trigger-{my-sevice}-pipeline-test" job, and in gerrit the UI links to that trigger job, but ideally id like it to link to the actual job itself (instead of the trigger job). is there a way to change that? [21:07:38] nikkinikk: on some jobs (dunno how this is setup anymore) the "pipelinebot" leaves a comment with a link to the *actual* job -- would that work? [21:10:16] yea! cause right now the PipelineBot is leaving a comment with a link to the trigger job. so if that could point to the "actual" job that would be cool [21:11:00] ah, neat, I'm in a meeting at the moment, but I bet either marxarelli or longma know the right magic for this [21:13:36] nikkinikk, thcipriani: we have a task for that https://phabricator.wikimedia.org/T271107 [21:15:00] i'm not sure how relevant the discussion there is though. it's a lot of low-level jenkins stuff around why we need trigger jobs to begin with [21:15:47] i think the more immediately actionable part would be getting pipelinelib to construct a url to the non-trigger job console [21:16:45] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [21:16:51] oh wait. that's not pipelinelib constructing the link is it... [21:17:06] it's zuul. and *sigh* that's why it links to the trigger job [21:17:10] :( [21:18:35] 10Project-Admins: Create User-Carn project - https://phabricator.wikimedia.org/T280575 (10Peachey88) 05Open→03Resolved a:03Peachey88 Hi @Carn, Your new user project #User-Carn has been created. [21:18:37] 10Phabricator (Upstream), 10Upstream: Per-user projects for personal work in progress tracking - https://phabricator.wikimedia.org/T555 (10Peachey88) [21:21:00] nikkinikk: looking a bit more closely, i believe the comment left by PipelineBot does link to the _actual_ job. it's the jenkins-bot that links to the trigger job. are you seeing something different? [21:24:36] Im looking a little closer and it might just be that I setup the job incorrectly in CI. So for this repo for example, it only leaves jenkins-bot comments, no pipelineBot comments https://gerrit.wikimedia.org/r/c/wvui/+/679964 [21:24:59] but i see others that leave the pipeline-bot comments (I misspoke earlier, confused pipeline bot and jenkins-bot) [21:27:51] ok, yeah the issue with jenkins-bot linking to the trigger job is unfortunately not something we can (easily) fix at the moment. the whole trigger-* job thing is a workaround for getting Zuul (our gating system) to work with jenkins pipeline jobs [21:28:14] let me see why pipelinebot didn't comment [21:29:45] oh right. pipelinebot only comments if there were images published. not otherwise [21:29:55] that _is_ something we could change [21:30:34] marxarelli: ah! good to know, is that a small change or a larger one? [21:30:46] medium :) [21:32:43] it would definitely make gerrit twice as chatty too, since both jenkins-bot and pipelinebot would be commenting for every set of tests run [21:33:01] i'll bring it up in our team meeting tomorrow and see if folks have thoughts [21:35:39] ok cool sounds good! thanks for the follow up! [21:35:50] np [21:44:55] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [21:50:47] (03PS1) 10Brennen Bearnes: Review access change [gitlab-ansible] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/681149 [21:56:46] (03CR) 10Thcipriani: [V: 03+2 C: 03+2] "This group is itself owned by Release Engineering, so this looks fine to me." [gitlab-ansible] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/681149 (owner: 10Brennen Bearnes) [22:22:42] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/PageTriage] Drop mwgate-rake [integration/config] - 10https://gerrit.wikimedia.org/r/681205 [22:22:44] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/VisualEditor] Drop mwgate-rake [integration/config] - 10https://gerrit.wikimedia.org/r/681206 [22:22:46] (03PS1) 10Jforrester: jjb: Provide legacy-quibble-rubyselenium-docker [integration/config] - 10https://gerrit.wikimedia.org/r/681207 (https://phabricator.wikimedia.org/T280491) [22:22:48] (03PS1) 10Jforrester: Zuul: Add legacy-quibble-rubyselenium-docker as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/681208 (https://phabricator.wikimedia.org/T280491) [22:24:05] (03CR) 10jerkins-bot: [V: 04-1] jjb: Provide legacy-quibble-rubyselenium-docker [integration/config] - 10https://gerrit.wikimedia.org/r/681207 (https://phabricator.wikimedia.org/T280491) (owner: 10Jforrester) [22:24:17] (03CR) 10jerkins-bot: [V: 04-1] Zuul: Add legacy-quibble-rubyselenium-docker as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/681208 (https://phabricator.wikimedia.org/T280491) (owner: 10Jforrester) [22:28:21] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/PageTriage] Drop mwgate-rake [integration/config] - 10https://gerrit.wikimedia.org/r/681205 (owner: 10Jforrester) [22:28:24] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/VisualEditor] Drop mwgate-rake [integration/config] - 10https://gerrit.wikimedia.org/r/681206 (owner: 10Jforrester) [22:30:18] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/PageTriage] Drop mwgate-rake [integration/config] - 10https://gerrit.wikimedia.org/r/681205 (owner: 10Jforrester) [22:30:27] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/VisualEditor] Drop mwgate-rake [integration/config] - 10https://gerrit.wikimedia.org/r/681206 (owner: 10Jforrester) [22:34:35] (03PS2) 10Jforrester: jjb: Provide legacy-quibble-rubyselenium-docker [integration/config] - 10https://gerrit.wikimedia.org/r/681207 (https://phabricator.wikimedia.org/T280491) [22:36:16] (03CR) 10Jforrester: [C: 03+2] jjb: Provide legacy-quibble-rubyselenium-docker [integration/config] - 10https://gerrit.wikimedia.org/r/681207 (https://phabricator.wikimedia.org/T280491) (owner: 10Jforrester) [22:37:35] (03Merged) 10jenkins-bot: jjb: Provide legacy-quibble-rubyselenium-docker [integration/config] - 10https://gerrit.wikimedia.org/r/681207 (https://phabricator.wikimedia.org/T280491) (owner: 10Jforrester) [22:38:46] (03PS2) 10Jforrester: Zuul: Add legacy-quibble-rubyselenium-docker as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/681208 (https://phabricator.wikimedia.org/T280491) [23:02:23] (03CR) 10Jforrester: [C: 03+2] Zuul: Add legacy-quibble-rubyselenium-docker as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/681208 (https://phabricator.wikimedia.org/T280491) (owner: 10Jforrester) [23:03:28] (03Merged) 10jenkins-bot: Zuul: Add legacy-quibble-rubyselenium-docker as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/681208 (https://phabricator.wikimedia.org/T280491) (owner: 10Jforrester) [23:04:38] !log Zuul: Add legacy-quibble-rubyselenium-docker as experimental T280491 [23:04:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:04:41] T280491: Ruby Browser Tests broken after OS update - https://phabricator.wikimedia.org/T280491 [23:14:19] 10Phabricator, 10serviceops: move phabricator to new hardware generation - https://phabricator.wikimedia.org/T280597 (10Dzahn) [23:32:34] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [23:37:52] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [23:57:00] 10LibUp: Use packaged composer - https://phabricator.wikimedia.org/T249525 (10Reedy) For more fun and games... Buster (testing) now has composer 2.0 - https://packages.debian.org/bullseye/composer