[01:09:26] 10Gerrit, 10Phabricator, 06Release-Engineering-Team (Seen), 06Diffusion-Repository-Administrators, 10Wikimedia-GitHub: Create some automation for management of gerrit/phabricator/github mirrors - https://phabricator.wikimedia.org/T235874#11625880 (10bd808) 05Stalled→03Open Removing Stalled status as... [02:39:34] 10Beta-Cluster-Infrastructure, 06DBA, 10MediaWiki-libs-Rdbms, 07Epic: Enable MariaDB/MySQL's Strict Mode - https://phabricator.wikimedia.org/T108255#11626005 (10Reedy) [03:13:29] (03PS1) 10Zabe: Add msk to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1240108 [03:13:41] (03PS2) 10Zabe: zuul: Add msk to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1240108 [03:14:56] (03PS3) 10Zabe: zuul: Add msk to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1240108 [06:58:11] !log Updating development images on contint primary for T415922 [06:58:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:58:14] T415922: Allow limited editing of Patch Demo wikis - https://phabricator.wikimedia.org/T415922 [08:20:58] 10Gerrit, 10Projects-Cleanup, 10Wikimedia-GitHub: Archive no longer used WMDE Gerrit repositories - https://phabricator.wikimedia.org/T376982#11626364 (10Aklapper) [08:24:16] 10Gerrit, 10Projects-Cleanup, 10Wikimedia-GitHub: Archive no longer used WMDE Gerrit repositories - https://phabricator.wikimedia.org/T376982#11626369 (10Aklapper) [08:25:11] 10Gerrit, 10Projects-Cleanup, 10Wikimedia-GitHub: Archive no longer used WMDE Gerrit repositories - https://phabricator.wikimedia.org/T376982#11626370 (10Aklapper) 05Open→03Resolved Not sure why "Deactivate the Diffusion repository" in the desc above was crossed out; I also deactivate the repos in Di... [09:20:56] (03CR) 10VolkerE: [C:03+1] Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [09:28:34] 10Gerrit, 06collaboration-services: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11626670 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by arnaudb@cumin1003 for host gerrit1003.wikimedia.org with OS bookworm [09:50:55] 10Gerrit, 06collaboration-services: Gerrit switchover between secondary instances - https://phabricator.wikimedia.org/T406334#11626755 (10ABran-WMF) p:05Medium→03High This task is a blocker for {T417247} [09:52:18] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#11626767 (10ABran-WMF) [09:52:35] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470#11626770 (10ABran-WMF) [09:58:38] 10Gerrit, 06collaboration-services: Gerrit switchover between secondary instances - https://phabricator.wikimedia.org/T406334#11626792 (10ABran-WMF) >>! In T406334#11242622, @Dzahn wrote: > This would only be a thing though if we permanently keep 3 or more gerrit servers around. The process will also become o... [10:14:45] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: Create an OpenSearch dashboard for Gerrit sshd logs - https://phabricator.wikimedia.org/T417753 (10hashar) 03NEW [10:32:54] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11626961 (10ABran-WMF) I missed a spot in puppet: ` $ grep gerrit /etc/passwd 27:gerrit2:x:925:925::/var/lib/gerrit2:/bin/bash ` [10:41:34] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11626996 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by arnaudb@cumin1003 for host gerrit1003.wikimedia.org with OS bookworm executed with errors: - gerrit1003 (**FAI... [10:56:54] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11627086 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by arnaudb@cumin1003 for host gerrit1003.wikimedia.org with OS bookworm [11:39:18] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11627233 (10ABran-WMF) missed spot cleaned up: ` arnaudb@gerrit1003:~ $ grep gerrit /etc/passwd 27:gerrit:x:925:925::/var/lib/gerrit:/bin/bash ` [11:58:16] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767 (10ABran-WMF) 03NEW [11:58:42] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767#11627311 (10ABran-WMF) p:05Triage→03High setting high priority to mirror the parent task [11:59:25] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767#11627315 (10ABran-WMF) [12:15:47] !log zuul-1001.zuul3.eqiad1.wikimedia.cloud: added keepalive=20 to the scheduler Gerrit driver and restarted scheduler container # T417497 [12:15:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:15:50] T417497: Gerrit events not received by Zuul (CI is not triggered for some patches) - https://phabricator.wikimedia.org/T417497 [12:27:02] 10Continuous-Integration-Infrastructure, 10Gerrit, 07Zuul, 13Patch-For-Review: Gerrit events not received by Zuul (CI is not triggered for some patches) - https://phabricator.wikimedia.org/T417497#11627407 (10hashar) There are other accounts listening for stream-events. They can be seen from `gerrit show-c... [13:11:41] 06Release-Engineering-Team (Doing), 10MediaWiki-extensions-InterwikiSorting, 13Patch-For-Review, 07Technical-Debt, 07User-notice: Undeploy the InterwikiSorting extension from Wikipedia production - https://phabricator.wikimedia.org/T253764#11627541 (10STei-WMF) Hello @Ladsgroup , I did some reading on th... [13:25:36] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767#11627596 (10ABran-WMF) [14:21:24] (03merge) 10jelto: remove manual job trigger for apply job [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/145 [14:24:55] (03open) 10jelto: remove diff job from ci [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/146 [14:25:12] (03update) 10jelto: remove manual job trigger for apply job [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/145 [14:54:11] 06Release-Engineering-Team (Doing), 10MediaWiki-extensions-InterwikiSorting, 13Patch-For-Review, 07Technical-Debt, 07User-notice: Undeploy the InterwikiSorting extension from Wikipedia production - https://phabricator.wikimedia.org/T253764#11628043 (10Ladsgroup) Hi, it's a bit different. We are not forci... [15:13:35] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11628110 (10Dzahn) I would not expect a reimage to work while the puppet role is applied. Recommend moving to "insetup" role, reimage and then apply gerrit prod role again. Errors like the... [15:25:46] (03CR) 10Phuedx: [C:03+1] Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:30:05] (03CR) 10Jforrester: "Is EventLogging definitely no longer needed? Remember that these aren't recursive." [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:33:16] (03CR) 10Phuedx: [C:03+1] "The PHP part of TestKitchen extension no longer depends on EventLogging. The JS part won't depend on it after https://gerrit.wikimedia.org" [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:33:23] (03CR) 10Phuedx: Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:35:09] (03CR) 10Jforrester: "Either is fine — though note that until I459a1da8aa4bf0505ec275ba5e725900ab334b80 is in all actively-supported branches (wmf/*, REL1_*), w" [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:36:31] (03PS2) 10Aude: Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) [15:36:59] (03CR) 10Aude: "added EventLogging back as a dependency for now" [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:37:32] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:39:22] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1240071 (https://phabricator.wikimedia.org/T417706) (owner: 10Aude) [15:42:23] (03approved) 10dancy: remove diff job from ci [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/146 (owner: 10jelto) [15:43:48] !log Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency for T417706 [15:44:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:44:39] T417706: CI ReadingLists / TestKitchen error - Could not find the registration file for the extension "EventBus" - https://phabricator.wikimedia.org/T417706 [15:58:44] (03PS2) 10Hashar: jjb: add EXT_DEPENDENCIES to Quibble jobs parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 [16:00:38] (03CR) 10Hashar: [C:04-1] "I need to review the resulting full diff." [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 (owner: 10Hashar) [16:12:35] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11628382 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by arnaudb@cumin1003 for host gerrit1003.wikimedia.org with OS bookworm executed with errors: - gerrit1003 (**FAI... [16:12:57] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11628384 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by arnaudb@cumin1003 for host gerrit1003.wikimedia.org with OS bookworm [16:20:03] maintenance-disconnect-full-disks build 782212 integration-agent-docker-1046 (/: 36%, /srv: 98%, /var/lib/docker: 31%): OFFLINE due to disk space [16:22:46] (03CR) 10Hashar: [C:04-1] "Jobs that have Zuul parameters added back:" [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 (owner: 10Hashar) [16:25:03] maintenance-disconnect-full-disks build 782213 integration-agent-docker-1046 (/: 36%, /srv: 89%, /var/lib/docker: 31%): RECOVERY disk space OK [16:29:02] (03PS3) 10Hashar: jjb: add EXT_DEPENDENCIES to Quibble jobs parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 [16:29:02] (03PS1) 10Hashar: jjb: add missing Zuul parameters on mwext-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1240333 [16:30:17] !log Triggered https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/ with empty Zuul parameters introduced by https://gerrit.wikimedia.org/r/1240333 | https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/4893/console [16:30:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:30:24] 17:29:26 WARNING:quibble.cmd:ZUUL_PROJECT not set. Assuming mediawiki/core [16:30:25] easy [16:32:09] (03PS4) 10Hashar: jjb: add EXT_DEPENDENCIES to Quibble jobs parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 [16:32:51] (03PS1) 10Harroyo-wmf: Include AbuseFilter as a dependency for WikimediaEvents [integration/config] - 10https://gerrit.wikimedia.org/r/1240335 (https://phabricator.wikimedia.org/T417799) [16:33:59] (03CR) 10Hashar: [C:03+2] "`" [integration/config] - 10https://gerrit.wikimedia.org/r/1240333 (owner: 10Hashar) [16:34:58] (03CR) 10Dreamy Jazz: [C:03+1] Include AbuseFilter as a dependency for WikimediaEvents [integration/config] - 10https://gerrit.wikimedia.org/r/1240335 (https://phabricator.wikimedia.org/T417799) (owner: 10Harroyo-wmf) [16:35:56] (03Merged) 10jenkins-bot: jjb: add missing Zuul parameters on mwext-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1240333 (owner: 10Hashar) [16:40:16] (03CR) 10Hashar: "Parent I658901bb7b764129189e5ef8d1ff1f5270caf48f adds the Zuul parameters to `mwext-phpunit-coverage` and this mean I need to remove `trig" [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 (owner: 10Hashar) [16:48:59] (03CR) 10Hashar: "s" [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 (owner: 10Hashar) [16:50:29] (03PS5) 10Hashar: jjb: add EXT_DEPENDENCIES to Quibble jobs parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 [16:53:16] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Reimage gerrit1003 - https://phabricator.wikimedia.org/T417246#11628551 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by arnaudb@cumin1003 for host gerrit1003.wikimedia.org with OS bookworm completed: - gerrit1003 (**PASS**) - Do... [16:53:30] (03PS5) 10Hashar: jjb: add EXT_DEPENDENCIES to Quibble jobs parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 [17:04:09] (03PS6) 10Hashar: jjb: add EXT_DEPENDENCIES to Quibble jobs parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 [17:04:09] (03CR) 10Hashar: [C:03+2] "Ok the diff looks good now, it looked mangled because CI does a merge commit (IIRC)." [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 (owner: 10Hashar) [17:04:54] !log Added EXT_DEPENDENCIES to Quibble Jenkins jobs parameters so we can manually trigger them from the Web UI using a different set of deps # https://gerrit.wikimedia.org/r/c/integration/config/+/1240254/ [17:04:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:05:13] I forgot SKIN_DEPENDENCIES, but that will be easy to add if needed [17:05:57] !log Regenerating Jenkins jobs with JJB based on https://gerrit.wikimedia.org/r/c/integration/config/+/1240254/ [17:05:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:07:24] (03Merged) 10jenkins-bot: jjb: add EXT_DEPENDENCIES to Quibble jobs parameters [integration/config] - 10https://gerrit.wikimedia.org/r/1240254 (owner: 10Hashar) [17:17:45] 06Release-Engineering-Team, 10Scap, 06collaboration-services: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767#11628684 (10hashar) ` warning: error running /usr/lib/git-core/git 'config' '--includes' '--global' '--replace-all' 'filter.lfs.clean' 'git-lfs clean -- %f': 'fatal: $HOM... [17:25:03] maintenance-disconnect-full-disks build 782225 integration-agent-docker-1041 (/: 37%, /srv: 99%, /var/lib/docker: 36%): OFFLINE due to disk space [17:30:03] maintenance-disconnect-full-disks build 782226 integration-agent-docker-1041 (/: 37%, /srv: 18%, /var/lib/docker: 35%): RECOVERY disk space OK [17:41:15] 06Release-Engineering-Team, 10MediaWiki-Vendor: Delete some branches from mediawiki/vendor - https://phabricator.wikimedia.org/T417804 (10Reedy) 03NEW [17:41:37] 06Release-Engineering-Team, 10MediaWiki-Vendor: Delete some branches from mediawiki/vendor - https://phabricator.wikimedia.org/T417804#11628742 (10Reedy) p:05Triage→03Low [17:45:10] 06Release-Engineering-Team, 10Scap, 06collaboration-services: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767#11628752 (10Dzahn) also see T257317#9851645 [17:58:08] 06Release-Engineering-Team, 10MediaWiki-Vendor: Delete some branches from mediawiki/vendor - https://phabricator.wikimedia.org/T417804#11628782 (10hashar) I think those were for ElasticSearch upgrades and probably created/used by @EBernhardson / @dcausse. I guess the branches can be deleted. I don't think th... [18:01:28] 06Release-Engineering-Team, 10MediaWiki-Vendor: Delete some branches from mediawiki/vendor - https://phabricator.wikimedia.org/T417804#11628807 (10Reedy) https://gerrit.wikimedia.org/g/mediawiki/vendor/+/refs/heads/es6 (Jan 2019) https://gerrit.wikimedia.org/g/mediawiki/vendor/+/refs/heads/es710 (Aug 2022) We... [18:10:03] maintenance-disconnect-full-disks build 782234 integration-agent-docker-1046 (/: 36%, /srv: 99%, /var/lib/docker: 31%): OFFLINE due to disk space [18:11:05] 06Release-Engineering-Team, 10MediaWiki-Vendor: Delete some branches from mediawiki/vendor - https://phabricator.wikimedia.org/T417804#11628827 (10EBernhardson) The es6 and es710 branches can indeed be cleared out, IIRC they were needed for CI to line things up, but with it deployed they are now unused. [18:15:03] maintenance-disconnect-full-disks build 782235 integration-agent-docker-1046 (/: 36%, /srv: 62%, /var/lib/docker: 31%): RECOVERY disk space OK [18:20:03] maintenance-disconnect-full-disks build 782236 integration-agent-docker-1057 (/: 37%, /srv: 100%, /var/lib/docker: 31%): OFFLINE due to disk space [18:25:03] maintenance-disconnect-full-disks build 782237 integration-agent-docker-1057 (/: 37%, /srv: 48%, /var/lib/docker: 31%): RECOVERY disk space OK [18:40:26] !log Unblock 46.59.0.0/17 (T417747) [18:40:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:57:56] 06Release-Engineering-Team, 10Scap, 06collaboration-services: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767#11629274 (10thcipriani) Do you have the full log of the puppet run to see more context? [19:04:42] 10Gerrit (Gerrit 3.12), 06collaboration-services: Upgrade Gerrit hosts from Bullseye to Bookworm - https://phabricator.wikimedia.org/T392464#11629299 (10Dzahn) > we should reimage the Gerrit hosts from Bullseye to Bookworm, keeping java 17. Arnaud reimaged gerrit1003 to bookworm. Java is not installed yet but... [19:07:10] (03PS1) 10Bvibber: Drop MobileFrontend from ReaderExperiments deps [integration/config] - 10https://gerrit.wikimedia.org/r/1240360 (https://phabricator.wikimedia.org/T416405) [19:35:25] 10GitLab (CI & Job Runners), 10ReaderExperiments: (CI) MobileFrontend test failures when used as a dep for ReaderExperiments - https://phabricator.wikimedia.org/T417818 (10bvibber) 03NEW [19:38:04] (03PS2) 10Bvibber: Drop MobileFrontend from ReaderExperiments deps [integration/config] - 10https://gerrit.wikimedia.org/r/1240360 (https://phabricator.wikimedia.org/T416405) [19:40:35] 10GitLab (CI & Job Runners), 10ReaderExperiments, 13Patch-For-Review: (CI) MobileFrontend test failures when used as a dep for ReaderExperiments - https://phabricator.wikimedia.org/T417818#11629419 (10bvibber) [19:40:38] (03CR) 10Reedy: [C:03+2] Drop MobileFrontend from ReaderExperiments deps [integration/config] - 10https://gerrit.wikimedia.org/r/1240360 (https://phabricator.wikimedia.org/T416405) (owner: 10Bvibber) [19:40:46] \o/ [19:42:15] (03Merged) 10jenkins-bot: Drop MobileFrontend from ReaderExperiments deps [integration/config] - 10https://gerrit.wikimedia.org/r/1240360 (https://phabricator.wikimedia.org/T416405) (owner: 10Bvibber) [19:44:54] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1240360 [19:44:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:49:42] thx [20:14:41] (03open) 10jhuneidi: catalyst-code-server: repair entrypoint [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/90 (https://phabricator.wikimedia.org/T415922) [20:15:13] (03merge) 10jhuneidi: catalyst-code-server: repair entrypoint [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/90 (https://phabricator.wikimedia.org/T415922) [20:17:36] !log Updating development images on contint primary for T415922 [20:17:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:17:38] T415922: Allow limited editing of Patch Demo wikis - https://phabricator.wikimedia.org/T415922 [21:14:17] 06Release-Engineering-Team, 10Scap, 06collaboration-services, 13Patch-For-Review: scap error on Gerrit first setup - https://phabricator.wikimedia.org/T417767#11629801 (10thcipriani) @dancy found a reason `$HOME` would not be set in the initial puppet run: the [[https://github.com/puppetlabs/puppet/blob/d0... [22:22:06] 10GitLab (CI & Job Runners), 10ReaderExperiments: (CI) MobileFrontend test failures when used as a dep for ReaderExperiments - https://phabricator.wikimedia.org/T417818#11630050 (10bvibber) may be a dupe of T415451 [22:49:23] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T413807#11630083 (10dancy) [23:03:51] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T413807#11630134 (10LucasWerkmeister) [23:37:38] Project beta-scap-sync-world build #245910: 04FAILURE in 2 min 19 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/245910/ [23:38:38] 23:37:37 [28 hits] Error: Interface "MediaWiki\Extension\ParserMigration\Hook\ShouldUseParsoidHook" not found [23:42:58] bvibber: ^ may just be opcache-y (see if the next one passes) [23:44:00] possibly. are there any wikis where ParserMigration is literally not present/loaded? [23:44:35] commons, seemingly [23:44:38] https://commons.wikimedia.beta.wmcloud.org/wiki/Special:Version is broken [23:44:44] hah [23:45:02] it should be safe to pull ReaderExperiments from commons afaik [23:45:04] https://noc.wikimedia.org/conf/highlight.php?file=InitialiseSettings-labs.php doesn't do a full override [23:46:51] readerexperiments is on on all beta, but only a handful in prod [23:47:11] parsermigration is off on commons [23:47:55] Yippee, build fixed! [23:47:55] Project beta-scap-sync-world build #245911: 09FIXED in 2 min 9 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/245911/ [23:48:19] Reedy: try https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1240406 in a pinch [23:49:25] that would be same as the default.. it's the InitialiseSettings-labs.php override that's the problem [23:49:26] wait no i have to fix that on beta lol [23:49:29] yeah [23:49:34] brainfart lol [23:52:17] Reedy: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1240406 now corrected