[00:00:58] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:05:53] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [03:01:22] (03PS1) 10Subramanya Sastry: MFE: Suppress notifications [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073921 [03:01:58] (03PS2) 10Subramanya Sastry: MFE: Suppress notifications [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073921 [03:02:07] (03PS3) 10Subramanya Sastry: MFE: Suppress notifications [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073921 [03:05:59] (03CR) 10Hashar: operations-puppet: Upgrade to bullseye (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 (owner: 10JHathaway) [03:06:05] (03CR) 10Hashar: [C:04-1] operations-puppet: Upgrade to bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 (owner: 10JHathaway) [04:45:36] 10Continuous-Integration-Infrastructure, 07Jenkins, 10Release-Engineering-Team (Priority Backlog šŸ“„), 07ci-test-error, 07Upstream: Various CI jobs failing after "mkdir: cannot create directory ā€˜logā€™: Permission denied" - https://phabricator.wikimedia.org/T282893#10159421 (10Pppery) [07:05:31] 10Continuous-Integration-Infrastructure, 10LDAP-Access-Requests, 06SRE, 10SRE-Access-Requests: Requesting access to `contint-admins`, `contint-docker`, LDAP `ciadmin` for 'Arthur taylor' - https://phabricator.wikimedia.org/T373969#10159512 (10ArthurTaylor) Are there any additional steps I need to take here... [07:09:21] 10Phabricator (Upstream), 10Release-Engineering-Team (Radar), 07Developer Productivity, 07Upstream: Add Open Graph support to Phabricator Maniphest Tasks to have link preview on Telegram, Slack, and other messaging apps - https://phabricator.wikimedia.org/T288117#10159522 (10hashar) From #upstream changelo... [07:12:44] 06Project-Admins: umbrella project for Wikimedia Ukraine - https://phabricator.wikimedia.org/T374503#10159524 (10Antanana) thank you, then let's have this new umbrella, with a tech subproject there to move the tasks from the old one, and then archive it. thank you! anything else you need from us to be able to do... [07:21:14] 10Continuous-Integration-Infrastructure, 10LDAP-Access-Requests, 06SRE, 10SRE-Access-Requests: Requesting access to `contint-admins`, `contint-docker`, LDAP `ciadmin` for 'Arthur taylor' - https://phabricator.wikimedia.org/T373969#10159527 (10hashar) @Ladsgroup Arthur already has shell access to production... [07:27:09] 10Continuous-Integration-Infrastructure, 07Jenkins, 10Release-Engineering-Team (Seen), 06collaboration-services, 06SRE: Upgrade CI Jenkins ssh key to ecdsa - https://phabricator.wikimedia.org/T177826#10159535 (10hashar) >>! In T177826#10049124, @Dzahn wrote: > Turns out there is another jenkins SSH key h... [08:19:04] (03CR) 10Isabelle Hurbain-Palatin: [C:03+2] ":giggle:" [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073474 (owner: 10Subramanya Sastry) [08:19:38] (03Merged) 10jenkins-bot: Stop hiding cite errors from legacy output [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073474 (owner: 10Subramanya Sastry) [08:28:08] 10Beta-Cluster-Infrastructure: access to zh.wikipedia.beta.wmflabs.org fails with ServerError 500 - https://phabricator.wikimedia.org/T373761#10159668 (10Xqt) ā†’14Duplicate dup:03T373668 [08:46:58] 10Continuous-Integration-Infrastructure, 07Jenkins: CI Jenkins wanrs about deprecated script hashes - https://phabricator.wikimedia.org/T375160 (10hashar) 03NEW [08:47:09] 10Continuous-Integration-Infrastructure, 07Jenkins: CI Jenkins warns about deprecated script hashes - https://phabricator.wikimedia.org/T375160#10159722 (10hashar) [08:50:49] 10Continuous-Integration-Infrastructure, 07Jenkins: Jenkins warning lacks details: WARNING: [hudson.security.LDAPSecurityRealm updateUserDetails] Failed to associate the e-mail address - https://phabricator.wikimedia.org/T375161 (10hashar) 03NEW [08:54:54] 10Continuous-Integration-Infrastructure, 07Jenkins: Jenkins warning lacks details: WARNING: [hudson.security.LDAPSecurityRealm updateUserDetails] Failed to associate the e-mail address - https://phabricator.wikimedia.org/T375161#10159784 (10hashar) [08:55:08] Yippee, build fixed! [08:55:08] Project beta-code-update-eqiad build #513823: 09FIXED in 2 min 8 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/513823/ [08:57:57] 10Continuous-Integration-Infrastructure, 07Jenkins: Jenkins warning lacks details: WARNING: [hudson.security.LDAPSecurityRealm updateUserDetails] Failed to associate the e-mail address - https://phabricator.wikimedia.org/T375161#10159786 (10hashar) 05Openā†’03Declined [09:01:49] 10Continuous-Integration-Infrastructure, 07Jenkins: CI Jenkins warns about deprecated script hashes - https://phabricator.wikimedia.org/T375160#10159793 (10hashar) https://integration.wikimedia.org/ci/manage/scriptApproval/ says: > Script approvals are stored in Jenkins as the hashed value of the script. Old... [09:01:58] !log CI Jenkins: approved 3 scripts we wrote which were pending approval at https://integration.wikimedia.org/ci/manage/scriptApproval/ [09:02:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:02:05] !log CI Jenkins: approved 3 scripts we wrote which were pending approval at https://integration.wikimedia.org/ci/manage/scriptApproval/ # T375160 [09:02:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:02:07] T375160: CI Jenkins warns about deprecated script hashes - https://phabricator.wikimedia.org/T375160 [09:03:34] !log Cleared deprecated approvals from CI Jenkins # T375160 [09:03:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:06:48] 10Continuous-Integration-Infrastructure, 07Jenkins: CI Jenkins warns about deprecated script hashes - https://phabricator.wikimedia.org/T375160#10159811 (10hashar) 05Openā†’03Resolved a:03hashar [09:09:14] fun [09:09:23] gitlab does not let one search for a given commit https://gitlab.wikimedia.org/search?search=661e53131ca80556b6e91803997b62b6454637e6&nav_source=navbar [09:09:24] :D [09:28:04] (03merge) 10jelto: update replica names [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/67 (https://phabricator.wikimedia.org/T366882) [09:28:46] jnuche: for the Jenkins log formatting, do you know where the field numbers are documented? [09:28:47] java.util.logging.SimpleFormatter.format = %4$s: [%2$s] %5$s %6$s %n [09:29:02] %6 is the exception but I am wondering where those numbers are documented [09:32:04] ah https://docs.oracle.com/javase/8/docs/api/java/util/logging/SimpleFormatter.html [09:49:30] hashar: yep, or for J17 https://docs.oracle.com/en/java/javase/17/docs/api/java.logging/java/util/logging/SimpleFormatter.html [10:20:30] 10GitLab (Pipeline Services MigrationšŸ¤), 06Editing-team: Migrate Citoid/Zotero Pipeline Repos to GitLab - https://phabricator.wikimedia.org/T344736#10160013 (10Mvolz) [10:20:35] 10GitLab (Pipeline Services MigrationšŸ¤), 06Editing-team: Migrate Citoid/Zotero Pipeline Repos to GitLab - https://phabricator.wikimedia.org/T344736#10160014 (10Mvolz) [10:23:39] 10GitLab (Pipeline Services MigrationšŸ¤), 06Editing-team: Migrate Citoid/Zotero Pipeline Repos to GitLab - https://phabricator.wikimedia.org/T344736#10160022 (10Mvolz) @dchan @zoe before we actually start using the zotero repo in gitlab we need to set up ci to build the docker image. @dchan would this be someth... [12:32:38] (03CR) 10Jgiannelos: [C:03+1] MFE config: Hide all kartographer maps to work around loading issues [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073883 (owner: 10Subramanya Sastry) [12:36:16] (03CR) 10Jgiannelos: "How are we reading those custom CSS styles? If we don't yet maybe its better to avoid having them on yaml per wiki and instead keep them i" [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073477 (owner: 10Subramanya Sastry) [14:21:22] (03CR) 10JHathaway: operations-puppet: Upgrade to bullseye (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 (owner: 10JHathaway) [14:24:29] Yippee, build fixed! [14:24:29] Project beta-code-update-eqiad build #513854: 09FIXED in 1 min 28 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/513854/ [14:34:53] (03CR) 10Hashar: [C:04-1] operations-puppet: Upgrade to bullseye (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 (owner: 10JHathaway) [14:35:23] jhathaway: it might be possible that none of the Python scripts in operations/puppet need python 2.7 anymore [14:35:26] but I haven't checked [14:35:59] thanks, I was considering checking that angle as well [14:36:15] most have been moved to python3 [14:36:38] how did you find all the python2 files? [14:36:42] and for the few with `/usr/bin/python` in the shebang, maybe `python` is already python3 [14:36:57] but afaik that requires the install of python-is-python3 package or something like taht [14:37:06] git grep -P '^#.*bin/python(2|$)' [14:37:19] (-P to use perl regular expression, which really should be the default) [14:37:30] or I need to make an alias such as `git pcre` [14:37:35] hashar: It kind-of makes sense to require a narrower scope on the canonical gitlab, but can we enable all-repo search on the Wikimedia instance? [14:37:35] or `git search` [14:38:29] thanks hashar [14:38:39] awight: no, it is behind the proprietary license. See my rant^Wcomment at https://phabricator.wikimedia.org/T268196#6637264 [14:38:40] hashar: Interesting config flag: global_search_commits_tab [14:39:30] haaargh, okay I see. Sorry to hear that [14:41:43] jhathaway: there is something else I haven't mentioned is that the version of ruby will change which might well cause bunch of issue with Puppet 5. [14:41:48] which is the default in the Gemfile [14:42:07] we would see :] [14:45:17] (03PS3) 10JHathaway: operations-puppet: Upgrade to bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 [14:46:33] (03CR) 10CI reject: [V:04-1] operations-puppet: Upgrade to bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 (owner: 10JHathaway) [14:46:51] right, I did some intial tests on the image, and made a patch, https://gerrit.wikimedia.org/r/c/operations/puppet/+/1073893 [14:46:59] tests that I ran pass [14:47:50] (03PS4) 10JHathaway: operations-puppet: Upgrade to bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 [14:51:49] jhathaway: which test have you done? [14:52:13] bundle exec rake test [14:52:14] I am asking cause the default rake target does optimizations to avoid running ab unch of tests [14:52:15] bundle exec tox [14:52:17] bundle exec rake global:spec [14:52:19] depending on whatever HEAD is [14:52:20] ah yeah [14:52:22] cool :-] [14:53:11] ah yeah [14:53:13] install tox first [14:53:16] then override virtualenv [14:53:17] clever [14:53:22] (03CR) 10Hashar: [C:03+2] operations-puppet: Upgrade to bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 (owner: 10JHathaway) [14:53:30] I'll build it [14:53:38] and we can switch the jenkins job tomorrow morning [14:53:49] (I am about to enter a four+ hours long meeting tunel) [14:53:53] sounds good thanks [14:54:02] just poke me tomorrow :) [14:54:02] enjoy the tunnel :) [14:54:15] and congratulations on dealing with Dockerfile.template! [14:54:42] (03Merged) 10jenkins-bot: operations-puppet: Upgrade to bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/1073902 (owner: 10JHathaway) [14:55:13] ah T335765 [14:55:13] T335765: Migrate all CI jobs from buster to bullseye or later and drop buster testing support - https://phabricator.wikimedia.org/T335765 [14:55:35] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Migrate all CI jobs from buster to bullseye or later and drop buster testing support - https://phabricator.wikimedia.org/T335765#10161353 (10hashar) [14:55:53] I am sure something will break eventually, we shall see [15:06:49] jhathaway: I have built `docker-registry.wikimedia.org/releng/operations-puppet:0.9.3` [15:06:59] great [15:07:14] please poke me about it tomorrow and I'll do the switch [15:09:30] will do [15:28:08] 10Deployments, 06Release-Engineering-Team, 06serviceops: sync-testservers-k8s takes 4 minutes when deploying a mediawiki-config change - https://phabricator.wikimedia.org/T374907#10161530 (10akosiaris) >>! In T374907#10153807, @akosiaris wrote: >>>! In T374907#10153052, @hashar wrote: > >> For the OCI image... [15:35:02] (03CR) 10Subramanya Sastry: "Check this. :-) https://github.com/wikimedia/integration-visualdiff/commit/352f845f9ca82d42a6ce83b09d8283093acee620" [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073477 (owner: 10Subramanya Sastry) [15:42:54] (03update) 10hashar: Remove Docker image pulling functionality [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/447 (https://phabricator.wikimedia.org/T366778) (owner: 10akosiaris) [15:43:58] 10Deployments, 06Release-Engineering-Team, 06serviceops: sync-testservers-k8s takes 4 minutes when deploying a mediawiki-config change - https://phabricator.wikimedia.org/T374907#10161586 (10hashar) Awesome thank you @akosiaris , given the step is removed, I guess there is no need to investigate why it took... [15:47:37] (03CR) 10Jgiannelos: "I think its OK if we already use it, just wanted to avoid it if its a new method." [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073477 (owner: 10Subramanya Sastry) [15:47:54] (03CR) 10Jgiannelos: [C:03+1] Add jawikivoyage specific CSS hacks [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073477 (owner: 10Subramanya Sastry) [15:59:45] 06Project-Admins: umbrella project for Wikimedia Ukraine - https://phabricator.wikimedia.org/T374503#10161643 (10Aklapper) Nothing else needed, I just need to find time in the next ~24h to create the projects and silently mass-re-tag existing tasks not to spam you all with notifications [16:22:54] 06Project-Admins: umbrella project for Wikimedia Ukraine - https://phabricator.wikimedia.org/T374503#10161813 (10Aklapper) 05Openā†’03Resolved Requested public project #WMUA has been created: https://phabricator.wikimedia.org/project/view/7435/ I've also create three subprojects: https://phabricator.wikim... [16:28:44] 10Continuous-Integration-Infrastructure, 07Jenkins, 10Release-Engineering-Team (Seen), 06collaboration-services, 06SRE: Upgrade CI Jenkins ssh key to ecdsa - https://phabricator.wikimedia.org/T177826#10161860 (10Dzahn) Thanks! Can we just click "revoke" then? Just so nobody wonders again in the future. [17:08:12] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services, 13Patch-For-Review: Switch Jenkins instances from Java 11 to Java 17 - https://phabricator.wikimedia.org/T359795#10162068 (10Dzahn) >>! In T359795#10148316, @hashar wrote: > The first entry in `profile::java::java_packages` craft... [17:36:23] (03merge) 10dancy: Remove Docker image pulling functionality [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/447 (https://phabricator.wikimedia.org/T366778) (owner: 10akosiaris) [17:37:10] (03update) 10dancy: Remove Docker image pulling functionality [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/447 (https://phabricator.wikimedia.org/T366778) (owner: 10akosiaris) [17:38:10] (03update) 10dancy: Release 4.104.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/449 [17:38:13] (03open) 10dancy: Release 4.104.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/449 [17:40:04] (03merge) 10dancy: Release 4.104.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/449 [17:48:23] 10Phabricator: Restrict maximum file size for Phabricator file uploads - https://phabricator.wikimedia.org/T151544#10162232 (10Iniquity) >>! In T151544#9068863, @Pginer-WMF wrote: > Recording short screencasts to illustrate bugs in a workflow results in hitting this limit quite often. I completely agree :)... [17:49:17] (03update) 10dancy: Establish a status file for the spiderpig jobrunner [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/448 [17:50:29] 10Scap, 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Evaluate the performance improvements brought in by prefetching MW images on WikiKube hosts - https://phabricator.wikimedia.org/T366778#10162246 (10dancy) Deployed with scap 4.104.0 [17:50:59] 10Phabricator: Restrict maximum file size for Phabricator file uploads - https://phabricator.wikimedia.org/T151544#10162261 (10Dzahn) > screencasts to illustrate bugs If there are videos or animations that are larger and relevant to our tech I would say it's also totally valid to upload them on https://comm... [18:43:48] (03open) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [18:43:49] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [18:43:58] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [18:44:01] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [19:23:05] Project beta-update-databases-eqiad build #79026: 04FAILURE in 3 min 4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79026/ [19:53:31] 06Release-Engineering-Team, 10MediaWiki-Core-Tests, 07Developer Productivity, 10MediaWiki-Platform-Team (Radar): Create a daily job running core + extension PHPUnit tests serially - https://phabricator.wikimedia.org/T372618#10162551 (10kostajh) It looks like there is already some infrastructure set up for... [19:59:53] (03open) 10dancy: WIP: HTTP API [repos/releng/scap] (master-I82ac284ef47c65d4449b570582d0e2691772df8e) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [19:59:53] (03update) 10dancy: Establish a status file for the spiderpig jobrunner [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/448 [19:59:54] (03update) 10dancy: WIP: HTTP API [repos/releng/scap] (master-I82ac284ef47c65d4449b570582d0e2691772df8e) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [20:00:01] (03update) 10dancy: WIP: HTTP API [repos/releng/scap] (master-I82ac284ef47c65d4449b570582d0e2691772df8e) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [20:00:01] (03update) 10dancy: Establish a status file for the spiderpig jobrunner [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/448 [20:00:09] (03update) 10dancy: Draft: WIP: HTTP API [repos/releng/scap] (master-I82ac284ef47c65d4449b570582d0e2691772df8e) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [20:26:43] (03update) 10dancy: WIP: HTTP API [repos/releng/scap] (master-I82ac284ef47c65d4449b570582d0e2691772df8e) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [20:26:47] (03update) 10dancy: WIP: HTTP API [repos/releng/scap] (master-I82ac284ef47c65d4449b570582d0e2691772df8e) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [20:32:12] Yippee, build fixed! [20:32:12] Project beta-update-databases-eqiad build #79027: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79027/ [21:12:20] (03CR) 10C. Scott Ananian: [C:03+2] Add jawikivoyage specific CSS hacks [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073477 (owner: 10Subramanya Sastry) [21:12:43] (03CR) 10C. Scott Ananian: [C:03+2] Diffserver: Add both desktop & mobile web urls [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073877 (owner: 10Subramanya Sastry) [21:12:56] (03Merged) 10jenkins-bot: Add jawikivoyage specific CSS hacks [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073477 (owner: 10Subramanya Sastry) [21:13:09] (03CR) 10C. Scott Ananian: [C:03+2] MFE config: Hide all kartographer maps to work around loading issues [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073883 (owner: 10Subramanya Sastry) [21:13:25] (03Merged) 10jenkins-bot: Diffserver: Add both desktop & mobile web urls [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073877 (owner: 10Subramanya Sastry) [21:13:42] (03Merged) 10jenkins-bot: MFE config: Hide all kartographer maps to work around loading issues [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073883 (owner: 10Subramanya Sastry) [21:14:07] (03CR) 10C. Scott Ananian: [C:03+2] MFE: Suppress notifications [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073921 (owner: 10Subramanya Sastry) [21:14:35] (03Merged) 10jenkins-bot: MFE: Suppress notifications [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1073921 (owner: 10Subramanya Sastry) [21:53:57] (03open) 10dduvall: deploy: Support docker based mwscript execution [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/89 (https://phabricator.wikimedia.org/T369115) [21:53:57] (03update) 10dduvall: deploy: Support docker based mwscript execution [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/89 (https://phabricator.wikimedia.org/T369115) [21:54:07] (03update) 10dduvall: deploy: Support docker based mwscript execution [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/89 (https://phabricator.wikimedia.org/T369115) [21:54:08] (03update) 10dduvall: deploy: Support docker based mwscript execution [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/89 (https://phabricator.wikimedia.org/T369115) [22:11:47] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [22:13:13] 10Phabricator: Restrict maximum file size for Phabricator file uploads - https://phabricator.wikimedia.org/T151544#10162948 (10matmarex) Try recording the screencasts as WEBM files encoded with VP9. We've allowed WEBM files on Phabricator in T215360 (in 2020), and this 4 MB limit has become a lot less annoyi... [22:13:38] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [22:22:58] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [22:23:06] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [22:42:50] (03close) 10dduvall: deploy: Support docker based mwscript execution [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/89 (https://phabricator.wikimedia.org/T369115) [22:43:18] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [23:15:08] (03update) 10dduvall: mwscript: Containerize MediaWiki script execution [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/450 (https://phabricator.wikimedia.org/T369115) [23:25:17] (03PS1) 10Jdlrobson: Visual diff: Remove noise from known low priority issues [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1074284 [23:33:29] (03PS2) 10Jdlrobson: Visual diff: Remove noise from known low priority issues [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1074284 [23:44:30] (03PS3) 10Stoyofuku-wmf: Visual diff: Remove noise from known low priority issues [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1074284 (https://phabricator.wikimedia.org/T375248) (owner: 10Jdlrobson) [23:46:53] FIRING: Queue (Jenkins jobs + Zuul functions) alert: - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [23:54:23] PROBLEM - Work requests waiting in Zuul Gearman server on contint1002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [23:56:23] RECOVERY - Work requests waiting in Zuul Gearman server on contint1002 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [23:56:54] RESOLVED: Queue (Jenkins jobs + Zuul functions) alert: - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [23:57:08] FIRING: Queue (Jenkins jobs + Zuul functions) alert: - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert