[00:00:39] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:04:16] (03open) 10addbot: Bump docker-registry.wikimedia.org/repos/search-platform/cirrussearch-elasticsearch-image from v7.10.2-5 to v7.10.2-12 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/575 [00:05:39] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:05:53] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:10:53] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [01:59:59] 10GitLab, 06collaboration-services: gitlab/devtools: send logs to a new disk - https://phabricator.wikimedia.org/T371066#10034301 (10Dzahn) I made T371573 and if that gets resolved we could also puppetize the mounting of the cinder volume and avoid running the script / mount commands manually. [03:31:05] Project mwcore-phpunit-coverage-master build #3751: 04STILL FAILING in 31 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/3751/ [07:39:34] 10Continuous-Integration-Infrastructure, 07Jenkins, 10Release-Engineering-Team (Seen), 06collaboration-services, 06SRE: Upgrade ci ssh key to ecdsa - https://phabricator.wikimedia.org/T177826#10034509 (10hashar) >>! In T177826#10034118, @Dzahn wrote: > ACK! I see the key in `modules/profile/manifests/ci/... [07:50:47] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [07:56:29] 10Release-Engineering-Team (Radar), 06collaboration-services, 06serviceops, 06SRE: replace production buster deployment servers - https://phabricator.wikimedia.org/T364656#10034567 (10akosiaris) 05Open→03Resolved a:03akosiaris deploy1003 has been tracked in T364417, deploy2002 reimaging as bullse... [08:00:47] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [08:14:32] 10Continuous-Integration-Infrastructure, 07Jenkins, 10Release-Engineering-Team (Seen), 06collaboration-services, 06SRE: Upgrade CI Jenkins ssh key to ecdsa - https://phabricator.wikimedia.org/T177826#10034583 (10hashar) [09:37:30] https://bundler.io/blog/2022/01/23/bundler-v2-3.html [09:38:03] bundler now enforces the bundler version in Gemfile.lock :/ [09:41:16] and our rake-ruby3.1 claims to be ruby3.1 but it is based on Bullseye which ships 2.7... [09:42:41] (03CR) 10Hashar: "The image is based on Bullseye which comes with Ruby 2.7 so this change is misleading. The image should have been based on Bookworm instea" [integration/config] - 10https://gerrit.wikimedia.org/r/1028587 (owner: 10Reedy) [10:35:58] 10Gerrit, 06SRE: How do handle old/unneeded Gerrit groups - https://phabricator.wikimedia.org/T371581#10034794 (10Aklapper) +#Gerrit (not sure why #SRE was added?) [11:38:03] 10Gerrit: How do handle old/unneeded Gerrit groups - https://phabricator.wikimedia.org/T371581#10034943 (10AndrewTavis_WMDE) Thank you, @Aklapper! Maybe #sre was wrong. Thinking was adding on @hashar's team. Apologies if that wasn't correct. This is for documentation anyway, so let's remove it :) [11:38:08] 10Gerrit: How do handle old/unneeded Gerrit groups - https://phabricator.wikimedia.org/T371581#10034946 (10AndrewTavis_WMDE) [11:49:49] (03update) 10jnuche: install_world: remove deploy master from list to install only if present [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/395 [12:14:33] 10Gerrit: How do handle old/unneeded Gerrit groups - https://phabricator.wikimedia.org/T371581#10035038 (10hashar) That was asked on Slack and I made a response yesterday but it was something like 11pm so I wrote the following in a rush: > Gerrit does not support deleting groups since they usually were referred... [12:19:05] 10Gerrit: How do handle old/unneeded Gerrit groups - https://phabricator.wikimedia.org/T371581#10035058 (10hashar) Looks like the groups on that other task were left as is and I guess we leave them as is given the repositories have been archived. Else, you can optionally: * Update the description with a `[ARCH... [12:50:11] (03PS1) 10Hashar: dockerfiles: rename rake-ruby3.1 to rake-ruby2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1059066 [12:51:35] (03CR) 10Hashar: [C:03+2] "I have manually pushed the tag `docker-registry.discovery.wmnet/releng/rake-ruby2.7:0.1.0` (and `latest`) to the same as the 3.1 image." [integration/config] - 10https://gerrit.wikimedia.org/r/1059066 (owner: 10Hashar) [12:52:47] (03Merged) 10jenkins-bot: dockerfiles: rename rake-ruby3.1 to rake-ruby2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1059066 (owner: 10Hashar) [12:55:32] (03PS1) 10Hashar: Rename ruby 3.1 jobs to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1059070 [12:58:29] (03PS2) 10Hashar: Rename ruby 3.1 jobs to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1059070 [12:59:45] (03CR) 10Hashar: [C:03+2] Rename ruby 3.1 jobs to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1059070 (owner: 10Hashar) [13:00:40] 10Scap, 13Patch-For-Review: Proposal for refactoring of backport dependency handling - https://phabricator.wikimedia.org/T362987#10035300 (10jnuche) [13:00:48] but why does oojs/ui requires ruby? :/ [13:00:54] (03Merged) 10jenkins-bot: Rename ruby 3.1 jobs to 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1059070 (owner: 10Hashar) [13:37:07] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Backport: implement new criteria for relevant dependencies - https://phabricator.wikimedia.org/T365146#10035447 (10jnuche) [13:38:50] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Backport: implement new criteria for relevant dependencies - https://phabricator.wikimedia.org/T365146#10035451 (10jnuche) [14:02:26] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Introduce new workflow for relevant dependencies calculation - https://phabricator.wikimedia.org/T371611 (10jnuche) 03NEW [14:03:18] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Introduce new workflow for relevant dependencies calculation - https://phabricator.wikimedia.org/T371611#10035545 (10jnuche) I'll help here writing the new tests [14:06:05] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Introduce new workflow for relevant dependencies calculation - https://phabricator.wikimedia.org/T371611#10035557 (10jnuche) [14:23:03] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Buster Deprecation): Migrate deployment-prep away from Debian Buster to Bullseye/Bookworm - https://phabricator.wikimedia.org/T327742#10035620 (10Andrew) [14:27:24] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Remove or replace poolcounter06.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370458#10035634 (10Andrew) [14:27:55] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Remove or replace poolcounter06.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370458#10035635 (10Andrew) Oh, I see that the prod poolcounter hosts are also still r... [14:28:09] (03CR) 10Jforrester: [C:03+2] build: Updating eslint-config-wikimedia to 0.28.2 [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1048762 (owner: 10Libraryupgrader) [14:30:41] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Remove or replace deployment-restbase04.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370460#10035657 (10Andrew) [14:57:14] (03approved) 10dancy: ci: run python tests with both Bullseye and Bookworm [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/98 (https://phabricator.wikimedia.org/T371511) (owner: 10hashar) [15:18:10] 10Continuous-Integration-Infrastructure: Add Python 3.10, 3.11 and 3.12 to Wikimedia CI - https://phabricator.wikimedia.org/T342019#10035778 (10Xqt) [15:27:22] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Rebuild or delete deployment-docker-cpjobqueue01 - https://phabricator.wikimedia.org/T369914#10035808 (10Urbanecm_WMF) >>! In T369914#9997770, @Southparkfan wrote: > Instance is offline, seems to be superseded by... [15:28:11] Project mwcore-phpunit-coverage-master build #3752: 04STILL FAILING in 28 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/3752/ [15:30:46] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Buster Deprecation): Migrate deployment-prep away from Debian Buster to Bullseye/Bookworm - https://phabricator.wikimedia.org/T327742#10035833 (10Andrew) [15:31:14] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Rebuild or delete deployment-docker-cpjobqueue01 - https://phabricator.wikimedia.org/T369914#10035828 (10Andrew) 05Open→03Resolved a:03Andrew deleted! [15:31:15] 10GitLab (CI & Job Runners), 06collaboration-services: (some) Gitlab builds hanging - https://phabricator.wikimedia.org/T371620 (10MatthewVernon) 03NEW [15:31:23] 10GitLab (CI & Job Runners), 06collaboration-services: (some) Gitlab builds hanging - https://phabricator.wikimedia.org/T371620#10035845 (10MatthewVernon) p:05Triage→03High [15:56:11] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Remove or replace poolcounter06.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370458#10035969 (10Dzahn) That would be T332015 and a question for the serviceops team. [15:56:36] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Remove or replace poolcounter06.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370458#10035971 (10Dzahn) [16:00:35] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Remove or replace poolcounter06.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370458#10035995 (10Dzahn) @Andrew Good news, see T332015#10035989 [16:02:41] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation): Remove or replace poolcounter06.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370458#10036010 (10Andrew) Oh, great. @Southparkfan does that unblock you here? [16:53:52] (03merge) 10dancy: update wikidata-query-gui path [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/94 (https://phabricator.wikimedia.org/T350793) (owner: 10jelto) [16:54:49] (03approved) 10dancy: install_world: remove deploy master from list to install only if present [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/395 (owner: 10jnuche) [18:27:37] (03PS1) 10Catrope: Add JsonConfig as a dependency for Chart [integration/config] - 10https://gerrit.wikimedia.org/r/1059146 [18:28:07] (03CR) 10Jforrester: [C:03+2] Add JsonConfig as a dependency for Chart [integration/config] - 10https://gerrit.wikimedia.org/r/1059146 (owner: 10Catrope) [18:29:15] (03Merged) 10jenkins-bot: Add JsonConfig as a dependency for Chart [integration/config] - 10https://gerrit.wikimedia.org/r/1059146 (owner: 10Catrope) [18:39:40] !log Zuul: Add JsonConfig as a dependency for Chart [18:39:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:40:20] !log Zuul: Rename ruby 3.1 jobs to 2.7 [merged but not deployed?] [18:40:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:42:03] Oh, meh. [18:44:06] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/Chart] Add JsonConfig as a phan dep too [integration/config] - 10https://gerrit.wikimedia.org/r/1059148 [18:44:33] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/Chart] Add JsonConfig as a phan dep too [integration/config] - 10https://gerrit.wikimedia.org/r/1059148 (owner: 10Jforrester) [18:45:43] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Chart] Add JsonConfig as a phan dep too [integration/config] - 10https://gerrit.wikimedia.org/r/1059148 (owner: 10Jforrester) [18:46:02] !log Zuul: [mediawiki/extensions/Chart] Add JsonConfig as a phan dep too [18:46:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:49:35] 10Continuous-Integration-Infrastructure, 07Jenkins, 10Release-Engineering-Team (Seen), 06collaboration-services, and 2 others: Upgrade CI Jenkins ssh key to ecdsa - https://phabricator.wikimedia.org/T177826#10036531 (10Dzahn) >>! In T177826#10034509, @hashar wrote: > I would append the ECDSA key there. Loo... [18:50:52] 10Continuous-Integration-Infrastructure, 07Jenkins, 10Release-Engineering-Team (Seen), 06collaboration-services, and 2 others: Upgrade CI Jenkins ssh key to ecdsa - https://phabricator.wikimedia.org/T177826#10036534 (10Dzahn) 05Open→03In progress [18:51:14] 10Deployments: Burst of curlFactory cURL errors during mediawiki deployments - https://phabricator.wikimedia.org/T371633 (10dancy) 03NEW [18:52:51] 10GitLab, 06collaboration-services: gitlab/devtools: send logs to a new disk - https://phabricator.wikimedia.org/T371066#10036549 (10Dzahn) [18:55:36] 10Deployments: Burst of curlFactory cURL errors during mediawiki deployments - https://phabricator.wikimedia.org/T371633#10036552 (10dancy) [19:21:48] 10Release-Engineering-Team (Seen), 10Scap, 06serviceops-radar: Transient httpbb errors from on mwdebug boxes - https://phabricator.wikimedia.org/T371427#10036618 (10brennen) 05Open→03Resolved a:03brennen > @thcipriani Mystery solved, in this specific case -- up to you whether to leave the task open... [19:22:10] 06Project-Admins, 07Tracking-Neverending: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706#10036622 (10MBinder_WMF) Please add @JayCano , EM for the #trust_and_safety_product_team , so that he can make sprint boards. :) [19:26:42] 10GitLab, 06collaboration-services: gitlab/devtools: send logs to a new disk - https://phabricator.wikimedia.org/T371066#10036625 (10Dzahn) Alright: - created new 20 GB cinder volume (Horizon) - attached new volume to instance (Horizon) - moved existing contents of /var/log/gitlab temp. to /root/ - ran wmcs-p... [19:28:01] 10GitLab, 06collaboration-services: gitlab/devtools: send logs to a new disk - https://phabricator.wikimedia.org/T371066#10036626 (10Dzahn) This way we don't even need a config change about where logs are going. All services are running per `gitlab-ctl status`. Except I am still waiting for https://gitlab.d... [19:31:01] 10GitLab, 06collaboration-services: gitlab/devtools: send logs to a new disk - https://phabricator.wikimedia.org/T371066#10036629 (10Dzahn) 05Open→03In progress [20:10:56] 06Project-Admins, 07Tracking-Neverending: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706#10036683 (10bd808) >>! In T706#10036621, @MBinder_WMF wrote: > Please add @JayCano , EM for the #trust_and_safety_product_team , so that he can make sprint boa... [20:27:28] !log Adding a volume to deployment-deploy04 to hold /srv [20:27:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:38:42] (03open) 10dancy: llbsolver: avoid nil releaser on error [repos/releng/buildkit] (wmf/v0.15) - 10https://gitlab.wikimedia.org/repos/releng/buildkit/-/merge_requests/67 [20:39:09] (03merge) 10dancy: llbsolver: avoid nil releaser on error [repos/releng/buildkit] (wmf/v0.15) - 10https://gitlab.wikimedia.org/repos/releng/buildkit/-/merge_requests/67 [20:51:15] 10Phabricator (Upstream), 07Upstream: When I delete a comment, I don't want to be added as a subscriber - https://phabricator.wikimedia.org/T338306#10036758 (10Aklapper) a:03Aklapper [21:01:24] 06Release-Engineering-Team: Buildkit 0.15.1 released - https://phabricator.wikimedia.org/T371641 (10dancy) 03NEW [21:01:54] 06Release-Engineering-Team: Buildkit 0.15.1 released - https://phabricator.wikimedia.org/T371641#10036776 (10dancy) [21:03:18] (03open) 10dancy: use buildkit 0.15.1 in staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/394 (https://phabricator.wikimedia.org/T371641) [21:03:19] (03update) 10dancy: use buildkit 0.15.1 in staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/394 (https://phabricator.wikimedia.org/T371641) [21:04:15] (03merge) 10dancy: use buildkit 0.15.1 in staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/394 (https://phabricator.wikimedia.org/T371641) [21:06:17] (03open) 10dancy: Use buildkit 0.15.1 in production [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/395 (https://phabricator.wikimedia.org/T371641) [21:06:19] (03update) 10dancy: Use buildkit 0.15.1 in production [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/395 (https://phabricator.wikimedia.org/T371641) [21:13:09] (03merge) 10dancy: Use buildkit 0.15.1 in production [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/395 (https://phabricator.wikimedia.org/T371641) [21:21:14] 06Release-Engineering-Team, 13Patch-For-Review: Buildkit 0.15.1 released - https://phabricator.wikimedia.org/T371641#10036835 (10dancy) p:05Triage→03Medium [21:29:51] 06Release-Engineering-Team, 06Data Products, 06Data-Platform-SRE, 10Dumps-Generation, and 2 others: Migrate current-generation dumps to run from our containerized images - https://phabricator.wikimedia.org/T352650#10036924 (10dr0ptp4kt) To confirm understanding, did we have a leaning on whether the contain... [21:53:49] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T366961#10036974 (10brennen) 05Open→03Resolved Seems stable on all wikis. Optimistically resolving. [22:10:22] 10Phabricator (Upstream), 07Upstream: When I delete a comment, I don't want to be added as a subscriber - https://phabricator.wikimedia.org/T338306#10036997 (10Aklapper) 05Open→03Stalled [22:15:14] 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Buildkit 0.15.1 released - https://phabricator.wikimedia.org/T371641#10037012 (10Dzahn) [22:23:53] 10GitLab, 06collaboration-services: gitlab/devtools: send logs to a new disk - https://phabricator.wikimedia.org/T371066#10037026 (10Dzahn) Had quite the trouble getting gitlab to start up again and debugged for a while. Turned out to be permission issues after all (though I had only used "mv" to move thi... [22:23:56] 10GitLab, 06collaboration-services: gitlab/devtools: send logs to a new disk - https://phabricator.wikimedia.org/T371066#10037027 (10Dzahn) 05In progress→03Resolved [22:32:06] 10Fresnel: For metrics which shouldn't change between runs (e.g. bytes transmitted), don't show "(± 0 B)" - https://phabricator.wikimedia.org/T220632#10037075 (10Pppery) [22:57:56] !log Reboot-testing deployment-deploy04 [22:57:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:08:06] Project beta-scap-sync-world build #165943: 04FAILURE in 2 min 6 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/165943/ [23:08:10] 06Release-Engineering-Team, 06collaboration-services: Buildkit 0.15.1 released - https://phabricator.wikimedia.org/T371641#10037149 (10dancy) 05Open→03Resolved [23:15:21] !log Re-armed keyholder on deployment-deploy04 [23:15:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:16:44] Yippee, build fixed! [23:16:44] Project beta-scap-sync-world build #165944: 09FIXED in 2 min 2 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/165944/ [23:56:15] 10Phabricator maintenance bot: Automatically generate patch for initial wiki configuration - https://phabricator.wikimedia.org/T371649 (10Zabe) 03NEW [23:57:03] 10Phabricator maintenance bot: Automatically generate patch for initial wiki configuration - https://phabricator.wikimedia.org/T371649#10037242 (10Zabe) [23:57:04] 10Release-Engineering-Team (Seen), 10LPL Technical Support, 10MediaWiki-Configuration: Automate WMF wiki creation - https://phabricator.wikimedia.org/T158730#10037243 (10Zabe)