[00:00:38] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:05:38] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [03:05:40] 06Gerrit-Privilege-Requests: Request REMOVAL of membership in mediawiki/extensions/Lingo group for Foxtrott - https://phabricator.wikimedia.org/T372133#10083588 (10matmarex) 05Stalled→03Invalid Hopefully! I also removed you as a reviewer from the already proposed patches for the Lingo extension (https:/... [03:27:02] 10Phabricator maintenance bot, 10SectionTranslation, 10LPL Essential (LPL Essential 2024 Jul-Sep): Complete enablement Section Translation in new wikis and make the process less manual for the future - https://phabricator.wikimedia.org/T371420#10083618 (10KartikMistry) 05Open→03In progress [06:29:40] (03PS1) 10Your1: Added YOUR1 to the trusted users list [integration/config] - 10https://gerrit.wikimedia.org/r/1064536 [07:38:56] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation), 13Patch-For-Review: Remove or replace deployment-restbase04.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370460#10083783 (10Jgiannelos) Could it be something wr... [07:50:29] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation), 13Patch-For-Review: Remove or replace deployment-restbase04.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370460#10083799 (10Jgiannelos) I think I found the issu... [08:09:29] 10Release-Engineering-Team (Radar), 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization, and 2 others: Validate CI integration so that Ci can release Maven artifacts on user's demand - https://phabricator.wikimedia.org/T367403#10083866 (10Gehel) [08:50:16] (03PS2) 10Zfilipin: jjb: Update teams responsible for selenium-daily jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1064038 (https://phabricator.wikimedia.org/T372868) [08:50:53] (03CR) 10Zfilipin: jjb: Update teams responsible for selenium-daily jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1064038 (https://phabricator.wikimedia.org/T372868) (owner: 10Zfilipin) [09:50:55] (03open) 10jelto: gitlab-runner: bump image version to alpine-v17.1.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/406 (https://phabricator.wikimedia.org/T373074) [14:02:26] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06Growth-Team, 10MediaWiki-extensions-CommunityConfiguration: Enable CommunityConfiguration Example in CI - https://phabricator.wikimedia.org/T373114 (10Michael) 03NEW [14:06:12] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06Growth-Team, 10MediaWiki-extensions-CommunityConfiguration: Enable CommunityConfiguration Example in CI - https://phabricator.wikimedia.org/T373114#10085057 (10Michael) [14:06:53] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06Growth-Team, 10MediaWiki-extensions-CommunityConfiguration: Enable CommunityConfiguration Example in CI - https://phabricator.wikimedia.org/T373114#10085062 (10Michael) [14:43:26] !log Upgrading gitlab-cloud-runners to gitlab-runner v17.1.1 [14:43:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:43:36] (03merge) 10dancy: gitlab-runner: bump image version to alpine-v17.1.1 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/406 (https://phabricator.wikimedia.org/T373074) (owner: 10jelto) [15:15:56] 10GitLab (Support), 07Documentation: Simplify fork-and-merge-request workflow descriptions in local documentation - https://phabricator.wikimedia.org/T373122 (10bd808) 03NEW [15:28:13] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation), 13Patch-For-Review: Remove or replace deployment-restbase04.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370460#10085369 (10Eevans) I found this in the logs: `... [15:36:40] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T366964#10085397 (10Aklapper) 05Open→03Resolved [15:40:44] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 (https://phabricator.wikimedia.org/T311080) [15:55:20] (03PS1) 10Zfilipin: zuul: Add John Bolorinos to trusted users [integration/config] - 10https://gerrit.wikimedia.org/r/1064793 (https://phabricator.wikimedia.org/T280652) [15:58:15] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Ask for confirmation instead of aborting if Train task status is no good - https://phabricator.wikimedia.org/T369831#10085527 (10thcipriani) @Aklapper is this one complete following your patch? Or are there other things to do here? [16:00:09] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation), 13Patch-For-Review: Remove or replace deployment-restbase04.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370460#10085514 (10Eevans) > @Jgiannelos is there no wa... [16:08:17] 10Phabricator (Search), 06Release-Engineering-Team: Revisit Elasticsearch in Phorge - https://phabricator.wikimedia.org/T373130 (10brennen) 03NEW [16:37:11] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests, 13Patch-For-Review: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10085667 (10Zabe) >>! In T372767#10076004, @Dzahn wrote: > - removed both use... [16:43:27] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Ask for confirmation instead of aborting if Train task status is no good - https://phabricator.wikimedia.org/T369831#10085683 (10Aklapper) 05Open→03Resolved Oh true this is fixed, thanks! [17:01:58] 10Phabricator (Search), 06Release-Engineering-Team: Revisit Elasticsearch in Phorge - https://phabricator.wikimedia.org/T373130#10085737 (10brennen) [17:55:08] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T366965#10085881 (10matmarex) [17:55:22] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T366964#10085885 (10matmarex) [18:35:19] (03merge) 10pwangai: Replace deprecated "sonar.login" by "sonar.token" [repos/ci-tools/sonarqube] - 10https://gitlab.wikimedia.org/repos/ci-tools/sonarqube/-/merge_requests/9 (owner: 10don-vip) [19:03:46] thcipriani: do you know if mw-deployment-vars.sh vars is still used? [19:04:03] https://codesearch.wmcloud.org/search/?q=mw-deployment-vars.sh&files=&excludeFiles=&repos= [19:04:18] I'm not sure who consumes that file, context is puppet 8 upgrade prep, https://phabricator.wikimedia.org/T372664 [19:05:04] I know that scap does not directly use that file. [19:06:04] nod, thanks, a bit hard to tell if anyone still consumes it [19:06:27] happy to try and fix the puppet code, but I would rather not, if it can just be removed [19:08:08] Since the file claims to be related to scap and I know scap doesn't use it, go ahead and remove it. [19:08:35] (claims, by way of being in the scap module) [19:08:35] the whole "This file should be BASH / Python polyglot" comment makes me thing it was used during the initial building of scap. Given codesearch comes up empty + we know scap and release don't use it: +1 remove [19:08:51] s/thing/think [19:09:44] an ERB template that must render as both Python and Bash. maybe YAML engineering is actually better [19:16:16] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/WikiLambda] Also depend on VE for tests [integration/config] - 10https://gerrit.wikimedia.org/r/1064835 [19:16:16] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/WikifunctionsClient] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/1064836 [19:25:37] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 (https://phabricator.wikimedia.org/T311080) [19:25:42] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 (https://phabricator.wikimedia.org/T311080) [19:29:02] (03update) 10dduvall: image: Remove inline credentials from remote context URLs [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/104 [19:29:21] great, thanks thcipriani [19:46:51] hi. i am seeing a lot of test failures with weird messages. one of them says "mkdir(): No space left on device": https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php82/2372/console which may be also the cause of other, less obvious failures [19:47:35] i think the last 3 or 4 patches i tried to merge today all failed with completely nonsensical errors [19:47:42] (all different) [19:48:27] (03approved) 10dancy: image: Remove inline credentials from remote context URLs [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/104 (owner: 10dduvall) [19:49:07] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests, 13Patch-For-Review: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10086219 (10Dzahn) >>! In T372767#10085667, @Zabe wrote: >>>! In T372767#1007... [19:49:08] (03merge) 10dduvall: image: Remove inline credentials from remote context URLs [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/104 [20:23:53] !log `sudo docker system prune` on `integration-agent-docker-1056` to release some disk space. [20:23:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:24:17] MatmaRex: I freed up 17GB of space in /var/lib/docker. That should improve the situation. [20:24:35] nice, thank you [22:05:44] 10Beta-Cluster-Infrastructure, 06Editing-team, 10VisualEditor: VisualEditor is not opening on Beta cluster, Error: Cannot read properties of undefined (reading 'getDefaultMode') - https://phabricator.wikimedia.org/T373153 (10Ryasmeen) 03NEW [22:11:46] 10Beta-Cluster-Infrastructure, 06Editing-team, 10VisualEditor, 13Patch-For-Review: VisualEditor is not opening on Beta cluster, Error: Cannot read properties of undefined (reading 'getDefaultMode') - https://phabricator.wikimedia.org/T373153#10086560 (10matmarex) a:03matmarex @Krinkle and I broke it. [22:19:06] MatmaRex: I was just about to write the same patch for CodeMirror :D [22:19:15] figured I'd check if someone else made one already [22:21:47] 10Beta-Cluster-Infrastructure, 06Community-Tech, 06Editing-team, 10MediaWiki-extensions-CodeMirror, and 3 others: VisualEditor is not opening on Beta cluster, Error: Cannot read properties of undefined (reading 'getDefaultMode') - https://phabricator.wikimedia.org/T373153#10086564 (10Krinkle) 05Open→... [23:26:03] !log Manually abort quibble-phpunit jobs for GrowthExperiments patches in gate-and-submit pipeline given their selenium build has already failed, so that other patches +2'ed can start their retry and get merged. Ref T373157 [23:26:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:26:05] T373157: Flaky selenium test "Failed to wait for ext.growthExperiments.PostEdit to be ready after 5000 ms." - https://phabricator.wikimedia.org/T373157 [23:29:58] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 (https://phabricator.wikimedia.org/T311080) [23:33:54] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 (https://phabricator.wikimedia.org/T311080) [23:48:11] (03PS1) 10Krinkle: zuul: Fix literal "/" being inserted when using it as keyboard shortcut [integration/docroot] - 10https://gerrit.wikimedia.org/r/1065039 [23:49:12] (03CR) 10Krinkle: [C:03+2] zuul: Fix literal "/" being inserted when using it as keyboard shortcut [integration/docroot] - 10https://gerrit.wikimedia.org/r/1065039 (owner: 10Krinkle) [23:49:44] (03Merged) 10jenkins-bot: zuul: Fix literal "/" being inserted when using it as keyboard shortcut [integration/docroot] - 10https://gerrit.wikimedia.org/r/1065039 (owner: 10Krinkle)