[04:19:45] 10Beta-Cluster-Infrastructure, 06Product Safety and Integrity: beta: /enwiki/wiki/Special:AccountSecurity UnexpectedValueException: More than one recovery codes instance was found in the database for this user. - https://phabricator.wikimedia.org/T429278 (10Chlod) 03NEW [06:04:46] 10Beta-Cluster-Infrastructure, 06Growth-Team, 10GrowthExperiments-NewcomerTasks, 06Discovery-Search (2026.06.01 - 2026.07.03): [beta-cluster] Fetching task suggestions failed: cirrussearch-backend-error - https://phabricator.wikimedia.org/T427196#12022053 (10ArthurTaylor) I'm still seeing broken search on... [06:10:20] (03PS1) 10Robert Vogel: Add new dependency to BlueSpiceProDistributionConnector [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 [06:25:23] (03PS1) 10Hslater: Zuul: [mediawiki/extensions/BlueSpiceProDistributionConnector] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1302640 [06:30:38] (03PS2) 10Hslater: Zuul: [mediawiki/extensions/BlueSpiceProDistributionConnector] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1302640 [06:34:11] (03PS2) 10Hashar: Add new dependency to BlueSpiceProDistributionConnector [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 (owner: 10Robert Vogel) [06:34:27] (03PS3) 10Hashar: Zuul: add dependency BlueSpiceProDistributionConnector [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 (owner: 10Robert Vogel) [06:36:39] (03PS4) 10Hashar: Zuul: add dependency BlueSpiceProDistributionConnector [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 (owner: 10Robert Vogel) [06:37:08] (03CR) 10Hashar: [C:03+2] "I have added VueJSPLus which is required by PDFCreator." [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 (owner: 10Robert Vogel) [06:39:24] (03Abandoned) 10Robert Vogel: Zuul: add dependency BlueSpiceProDistributionConnector [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 (owner: 10Robert Vogel) [06:40:23] (03CR) 10Hashar: "Oops :) no worries!" [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 (owner: 10Robert Vogel) [06:43:13] (03PS3) 10Hashar: Zuul: [BlueSpiceProDistributionConnector] add PDFCreator [integration/config] - 10https://gerrit.wikimedia.org/r/1302640 (owner: 10Hslater) [06:43:37] (03CR) 10Hashar: [C:03+2] "I have added VueJSPlus which is a requirement of PDFCreator." [integration/config] - 10https://gerrit.wikimedia.org/r/1302640 (owner: 10Hslater) [06:46:44] (03Merged) 10jenkins-bot: Zuul: [BlueSpiceProDistributionConnector] add PDFCreator [integration/config] - 10https://gerrit.wikimedia.org/r/1302640 (owner: 10Hslater) [06:47:58] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/Workflows] Re-enable master jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1302228 (owner: 10Hslater) [06:48:49] (03CR) 10Hashar: [C:03+2] build: Updating js-yaml to 4.2.0 [integration/docroot] - 10https://gerrit.wikimedia.org/r/1302287 (owner: 10Libraryupgrader) [06:49:43] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Workflows] Re-enable master jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1302228 (owner: 10Hslater) [06:50:11] (03CR) 10Hashar: [C:03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/1302228 (owner: 10Hslater) [06:51:00] (03CR) 10Robert Vogel: "Sorry! I didn't see this was merged already when I abandoned it!" [integration/config] - 10https://gerrit.wikimedia.org/r/1302634 (owner: 10Robert Vogel) [07:02:32] (03PS1) 10Hslater: Zuul: [mediawiki/extensions/BlueSpiceProDistributionConnector] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1302656 [07:03:21] (03CR) 10Hslater: "Nice catch, but VueJSPlus -> VueJsPlus ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/1302640 (owner: 10Hslater) [07:03:38] (03PS2) 10Hslater: Zuul: [mediawiki/extensions/BlueSpiceProDistributionConnector] Add VueJsPlus [integration/config] - 10https://gerrit.wikimedia.org/r/1302656 [07:13:49] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: Request for adding to WMF-NDA for JerryWang-WMF - https://phabricator.wikimedia.org/T429240#12022204 (10Aklapper) 05Open→03Resolved a:03Aklapper Done after verifying via https://meta.wikimedia.org/wiki/Special:Log?page=User:JerryWang-WMF [07:24:01] (03CR) 10Hashar: [C:03+2] "I swear I have copy pasted it!!! Sorry -:-]" [integration/config] - 10https://gerrit.wikimedia.org/r/1302640 (owner: 10Hslater) [07:24:08] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/BlueSpiceProDistributionConnector] Add VueJsPlus [integration/config] - 10https://gerrit.wikimedia.org/r/1302656 (owner: 10Hslater) [07:26:13] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/BlueSpiceProDistributionConnector] Add VueJsPlus [integration/config] - 10https://gerrit.wikimedia.org/r/1302656 (owner: 10Hslater) [07:27:44] (03CR) 10Hashar: [C:03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/1302656 (owner: 10Hslater) [07:34:43] (03PS1) 10Hslater: Zuul: [mediawiki/extensions/EnhancedUpload] Re-enable master jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1302666 [07:59:17] 06Release-Engineering-Team (Radar), 10Cloud-VPS (Quota-requests): Quota increase request for zuul - https://phabricator.wikimedia.org/T428515#12022311 (10Volans) @dduvall did you had a chance to test if it works without it? [08:02:20] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/EnhancedUpload] Re-enable master jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1302666 (owner: 10Hslater) [08:04:56] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/EnhancedUpload] Re-enable master jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1302666 (owner: 10Hslater) [08:05:58] (03CR) 10Hashar: [C:03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/1302666 (owner: 10Hslater) [08:38:52] (03PS2) 10Hashar: Zuul: [TestKitchen] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1237861 (https://phabricator.wikimedia.org/T389998) [08:54:34] FIRING: InstanceDown: Project deployment-prep instance deployment-cirrussearch14 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [08:54:43] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cirrussearch14 is down - https://phabricator.wikimedia.org/T429296 (10wmcs-alerts) 03NEW [08:59:34] RESOLVED: [3x] InstanceDown: Project deployment-prep instance deployment-cirrussearch12 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [09:06:04] FIRING: [3x] InstanceDown: Project deployment-prep instance deployment-cirrussearch12 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [09:06:10] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cirrussearch12 is down - https://phabricator.wikimedia.org/T429299 (10wmcs-alerts) 03NEW [09:09:04] 10Beta-Cluster-Infrastructure, 06Growth-Team, 10GrowthExperiments-NewcomerTasks, 06Discovery-Search (2026.06.01 - 2026.07.03): [beta-cluster] Fetching task suggestions failed: cirrussearch-backend-error - https://phabricator.wikimedia.org/T427196#12022575 (10dcausse) a:05dcausse→03bking @bking is I thi... [09:18:12] 10Continuous-Integration-Infrastructure, 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, and 2 others: Fetches from Gerrit aborted due to: GnuTLS recv error (-54): Error in the pull function - https://phabricator.wikimedia.org/T420865#12022630 (10ArthurTaylor) I'm still seeing a bunch of Re... [09:23:18] (03PS1) 10Hashar: Zuul: [TestKitchen] replace EventLogging by EventBus [integration/config] - 10https://gerrit.wikimedia.org/r/1302756 [09:23:18] (03PS1) 10Hashar: Zuul: [TestKitchen] remove EventStreamConfig [integration/config] - 10https://gerrit.wikimedia.org/r/1302757 (https://phabricator.wikimedia.org/T422358) [09:33:57] 10Continuous-Integration-Infrastructure, 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, and 2 others: Fetches from Gerrit aborted due to: GnuTLS recv error (-54): Error in the pull function - https://phabricator.wikimedia.org/T420865#12022723 (10ArthurTaylor) https://integration.wikimedia.... [09:39:49] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cirrussearch12 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [09:45:08] (03CR) 10Phuedx: [C:03+1] Zuul: [TestKitchen] replace EventLogging by EventBus [integration/config] - 10https://gerrit.wikimedia.org/r/1302756 (owner: 10Hashar) [09:45:32] (03CR) 10Phuedx: [C:03+1] Zuul: [TestKitchen] remove EventStreamConfig [integration/config] - 10https://gerrit.wikimedia.org/r/1302757 (https://phabricator.wikimedia.org/T422358) (owner: 10Hashar) [09:46:53] (03CR) 10Hashar: [C:03+2] "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/1302756 (owner: 10Hashar) [09:46:57] (03CR) 10Hashar: [C:03+2] "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/1302757 (https://phabricator.wikimedia.org/T422358) (owner: 10Hashar) [09:50:35] (03Merged) 10jenkins-bot: Zuul: [TestKitchen] replace EventLogging by EventBus [integration/config] - 10https://gerrit.wikimedia.org/r/1302756 (owner: 10Hashar) [09:51:10] (03Merged) 10jenkins-bot: Zuul: [TestKitchen] remove EventStreamConfig [integration/config] - 10https://gerrit.wikimedia.org/r/1302757 (https://phabricator.wikimedia.org/T422358) (owner: 10Hashar) [10:15:26] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Disable unused @Wmdephabbot and @WMDE-Phabricator-Board-Statistics-Bot Phabricator accounts - https://phabricator.wikimedia.org/T418141#12022854 (10Aklapper) a:03Aklapper [10:15:48] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Disable unused @Wmdephabbot and @WMDE-Phabricator-Board-Statistics-Bot Phabricator accounts - https://phabricator.wikimedia.org/T418141#12022857 (10Aklapper) 05Open→03Resolved Thanks for the clarification! I now disabled @Wmdephabbot and @WMDE-Phabri... [11:14:28] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-OATHAuth, 06Product Safety and Integrity: beta: /enwiki/wiki/Special:AccountSecurity UnexpectedValueException: More than one recovery codes instance was found in the database for this user. - https://phabricator.wikimedia.org/T429278#12023118 (10Reedy) Do... [11:16:29] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-OATHAuth, 06Product Safety and Integrity: beta: /enwiki/wiki/Special:AccountSecurity UnexpectedValueException: More than one recovery codes instance was found in the database for this user. - https://phabricator.wikimedia.org/T429278#12023120 (10Chlod) Ye... [11:32:46] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-OATHAuth, 06Product Safety and Integrity: beta: /enwiki/wiki/Special:AccountSecurity UnexpectedValueException: More than one recovery codes instance was found in the database for this user. - https://phabricator.wikimedia.org/T429278#12023203 (10Reedy) It... [11:34:17] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-OATHAuth, 06Product Safety and Integrity: beta: /enwiki/wiki/Special:AccountSecurity UnexpectedValueException: More than one recovery codes instance was found in the database for this user. - https://phabricator.wikimedia.org/T429278#12023209 (10Reedy) Th... [11:39:14] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-OATHAuth, 06Product Safety and Integrity: beta: /enwiki/wiki/Special:AccountSecurity UnexpectedValueException: More than one recovery codes instance was found in the database for this user. - https://phabricator.wikimedia.org/T429278#12023222 (10Chlod) Th... [11:42:27] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-OATHAuth, 06Product Safety and Integrity: beta: /enwiki/wiki/Special:AccountSecurity UnexpectedValueException: More than one recovery codes instance was found in the database for this user. - https://phabricator.wikimedia.org/T429278#12023240 (10Reedy) I... [12:27:25] (03PS3) 10Hashar: Zuul: [TestKitchen] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1237861 (https://phabricator.wikimedia.org/T389998) [12:42:41] 10Continuous-Integration-Infrastructure, 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, and 2 others: Fetches from Gerrit aborted due to: GnuTLS recv error (-54): Error in the pull function - https://phabricator.wikimedia.org/T420865#12023559 (10ABran-WMF) 05Open→03In progress [13:12:28] (03open) 10kamila: Add a bookworm image flavour [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/246 (https://phabricator.wikimedia.org/T418200 https://phabricator.wikimedia.org/T429030) [13:12:34] (03update) 10kamila: Add a bookworm image flavour [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/246 (https://phabricator.wikimedia.org/T418200 https://phabricator.wikimedia.org/T429030) [13:20:49] (03PS1) 10Arnaudb: zuul: retry on connection resets, not only GnuTLS errors [integration/quibble] - 10https://gerrit.wikimedia.org/r/1302819 (https://phabricator.wikimedia.org/T420865) [13:20:49] (03CR) 10Arnaudb: "To handle https://phabricator.wikimedia.org/T420865#12022630 where another pattern emerged, I broadened the scope of retryable network err" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1302819 (https://phabricator.wikimedia.org/T420865) (owner: 10Arnaudb) [13:44:40] (03approved) 10aghirelli: Update summary rules based on feedback [repos/ci-tools/wikimedia-spectral-ruleset] - 10https://gitlab.wikimedia.org/repos/ci-tools/wikimedia-spectral-ruleset/-/merge_requests/12 (https://phabricator.wikimedia.org/T428150) (owner: 10kbach) [13:44:49] (03merge) 10aghirelli: Update summary rules based on feedback [repos/ci-tools/wikimedia-spectral-ruleset] - 10https://gitlab.wikimedia.org/repos/ci-tools/wikimedia-spectral-ruleset/-/merge_requests/12 (https://phabricator.wikimedia.org/T428150) (owner: 10kbach) [14:35:37] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services, 07Essential-Work: Deploy Phab/Phorge 2026-06-16 - https://phabricator.wikimedia.org/T429350 (10brennen) 03NEW [14:36:12] (03open) 10brennen: update submodules for 2026-06-16 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/108 (https://phabricator.wikimedia.org/T429350) [14:39:17] (03approved) 10swfrench: Add a bookworm image flavour [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/246 (https://phabricator.wikimedia.org/T418200 https://phabricator.wikimedia.org/T429030) (owner: 10kamila) [14:44:55] (03open) 10aokoth: scap: replace phab2002 with phab2003 [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/109 (https://phabricator.wikimedia.org/T423727) [14:46:55] (03CR) 10Hashar: [C:03+2] "It is a transitive dependency to a dev dependency, no need to rebuild the extension." [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1302843 (owner: 10Libraryupgrader) [14:54:58] (03merge) 10brennen: update submodules for 2026-06-16 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/108 (https://phabricator.wikimedia.org/T429350) [14:57:17] (03approved) 10dancy: Add a bookworm image flavour [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/246 (https://phabricator.wikimedia.org/T418200 https://phabricator.wikimedia.org/T429030) (owner: 10kamila) [15:00:54] (03CR) 10Ahmon Dancy: [C:03+1] zuul: retry on connection resets, not only GnuTLS errors [integration/quibble] - 10https://gerrit.wikimedia.org/r/1302819 (https://phabricator.wikimedia.org/T420865) (owner: 10Arnaudb) [15:05:02] (03open) 10dancy: config.py: Bump version_update_patch_timeout to 10 minutes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1204 [15:05:14] (03update) 10dancy: config.py: Bump version_update_patch_timeout to 10 minutes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1204 [15:07:02] 10Phabricator (2026-06-16), 06Release-Engineering-Team, 06collaboration-services, 07Essential-Work, 13Patch-For-Review: Deploy Phab/Phorge 2026-06-16 - https://phabricator.wikimedia.org/T429350#12024368 (10Aklapper) Changelog for this deployment: * Core: ** Work around {T361459} and hopefully fix {T42830... [15:08:16] 10MediaWiki-Releasing, 05MW-1.46-release: Write release announcement for 1.46.0-rc.0 - https://phabricator.wikimedia.org/T423268#12024379 (10MSantos) 05Open→03Resolved a:03MSantos [15:21:01] (03open) 10dancy: log.py: Fix JSONFormatter to use 0 for lineno instead of None [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1205 [15:21:04] (03update) 10dancy: log.py: Fix JSONFormatter to use 0 for lineno instead of None [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1205 [15:24:05] (03merge) 10dancy: log.py: Fix JSONFormatter to use 0 for lineno instead of None [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1205 [15:24:40] 10Phabricator, 10Phabricator maintenance bot: Replace deprecated Phabricator Conduit API calls with their stable equivalents - https://phabricator.wikimedia.org/T428847#12024526 (10Ladsgroup) https://gitlab.wikimedia.org/ladsgroup/Phabricator-maintenance-bot/-/commit/31c8062309e7d6d6efd62f49c865eb46012e8d63 D... [15:25:01] (03update) 10dancy: config.py: Bump version_update_patch_timeout to 10 minutes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1204 [15:27:23] (03merge) 10dancy: config.py: Bump version_update_patch_timeout to 10 minutes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1204 [15:28:12] (03update) 10dancy: Release 4.269.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1206 [15:28:14] (03open) 10dancy: Release 4.269.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1206 [15:28:24] 10Phabricator, 10Phabricator maintenance bot: Replace deprecated Phabricator Conduit API calls with their stable equivalents - https://phabricator.wikimedia.org/T428847#12024589 (10Ladsgroup) Had to revert. The output format is completely different and it breaks basically everything. I give it to claude or som... [15:30:55] (03merge) 10dancy: Release 4.269.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1206 [15:31:14] !log Turning off deployment-db11 and deployment-db14 [15:31:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:45:04] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team: Increase size of deployment-deploy04.deployment-prep - https://phabricator.wikimedia.org/T429364 (10dancy) 03NEW [15:48:20] !log Resizing deployment-deploy04.deployment-prep from g4.cores4.ram8.disk20 to g4.cores8.ram16.disk20 (T429364) [15:48:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:48:23] T429364: Increase size of deployment-deploy04.deployment-prep - https://phabricator.wikimedia.org/T429364 [15:48:37] 06Release-Engineering-Team (Radar), 10QS-Test-Automation, 07Epic: [EPIC] Create Critical User Journey test automation for CI - https://phabricator.wikimedia.org/T422555#12024711 (10zeljkofilipin) A few related links @thcipriani shared with me: - https://wikitech.wikimedia.org/wiki/Httpbb - https://gerrit.wi... [16:00:46] !log sudo keyholder arm on deployment-deploy04.deployment-prep [16:00:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:01:52] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team: Increase size of deployment-deploy04.deployment-prep - https://phabricator.wikimedia.org/T429364#12024850 (10dancy) 05Open→03Resolved Done. [16:07:21] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Remove remaining diffusion.ssh-host entries from Phab config and Puppet - https://phabricator.wikimedia.org/T429367 (10Aklapper) 03NEW p:05Triage→03Low [16:14:28] (03CR) 10Dzahn: [C:03+1] zuul: retry on connection resets, not only GnuTLS errors [integration/quibble] - 10https://gerrit.wikimedia.org/r/1302819 (https://phabricator.wikimedia.org/T420865) (owner: 10Arnaudb) [16:15:29] 10Phabricator (2026-06-16), 06Release-Engineering-Team (Doing 😎), 06collaboration-services, 07Essential-Work: Deploy Phab/Phorge 2026-06-16 - https://phabricator.wikimedia.org/T429350#12024948 (10Aklapper) 05Open→03Resolved a:03brennen [16:16:01] 10Phabricator (2026-06-16), 06Release-Engineering-Team (Doing 😎): PhabricatorDataNotAttachedException in ManiphestTaskDetailController when anonymously accessing a task created by email - https://phabricator.wikimedia.org/T428640#12024951 (10Aklapper) 05Stalled→03Resolved Fixed by today's deployment in... [16:57:56] 10Phabricator-Bot-Requests: Phabricator Bot Request: Test Kitchen Experiments Status Reporter - https://phabricator.wikimedia.org/T429377 (10cjming) 03NEW [17:00:54] 06Release-Engineering-Team (Radar), 10Cloud-VPS (Quota-requests): Quota increase request for zuul - https://phabricator.wikimedia.org/T428515#12025314 (10dduvall) 05Open→03Resolved a:03dduvall Sorry, yes! We have a working setup that uses CAPI and a custom SANs entry for haproxy to give us a publicly... [17:01:14] 10Phabricator: Requesting permissions to bulk move tasks from one project to another - https://phabricator.wikimedia.org/T429378 (10DMburugu) 03NEW [17:03:07] 06Release-Engineering-Team (Radar), 10Cloud-VPS (Quota-requests): Quota increase request for zuul - https://phabricator.wikimedia.org/T428515#12025333 (10Volans) Great, thanks for closing the loop. [17:06:41] 10Phabricator (2026-06-16), 06Release-Engineering-Team (Doing 😎), 10Wikimedia-Phabricator-Extensions: Remove custom "Expert Mode" - https://phabricator.wikimedia.org/T351289#12025344 (10Aklapper) 05Open→03Resolved This got resolved today in the deployment in T429350. [17:08:16] 10Phabricator: Requesting permissions to bulk move tasks from one project to another - https://phabricator.wikimedia.org/T429378#12025358 (10Urbanecm) I reviewed the situation and the permission was removed on Jun 04 by @Aklapper (previously granted by yours truly on Oct 7, 2025): {F88927690} Presumably, this... [17:14:36] 10Continuous-Integration-Infrastructure, 10Browser Test Platform, 10QS-Test-Automation, 07ci-test-error (WMF-deployed Build Failure): Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#12025381 (10Daimona) FWIW, this is still happening.... [17:27:34] 10Continuous-Integration-Infrastructure, 10Browser Test Platform, 10QS-Test-Automation, 07ci-test-error (WMF-deployed Build Failure): Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#12025447 (10vaughnwalters) >>! In T389536#12025381,... [18:10:39] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily] Fetch & report Catalyst image [integration/config] - 10https://gerrit.wikimedia.org/r/1302933 (https://phabricator.wikimedia.org/T424815) [18:16:54] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily] Fetch & report Catalyst image [integration/config] - 10https://gerrit.wikimedia.org/r/1302933 (https://phabricator.wikimedia.org/T424815) [18:41:22] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 07Zuul, 06collaboration-services: Increase memory for Zuul VMs - https://phabricator.wikimedia.org/T425679#12026010 (10Dzahn) Hi all. So.. since we are now getting hardware for zuul.. does it mean this ticket is outdated? [18:48:17] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Remove acl*Batch-Editors members who have not executed bulk jobs in the last 4 years - https://phabricator.wikimedia.org/T428167#12026023 (10Aklapper) Well, turns out in T429378 that this also affects the "Move Tasks to Project" option in workboard colum... [18:55:50] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Requesting permissions to bulk move tasks from one project to another - https://phabricator.wikimedia.org/T429378#12026049 (10Aklapper) 05Open→03Resolved a:03Aklapper > Presumably, this happened due to an inactivity sweep. Correct, and that was m... [19:01:19] (03PS2) 10Jforrester: jjb: [catalyst-daily] Fetch & report Catalyst image [integration/config] - 10https://gerrit.wikimedia.org/r/1302933 (https://phabricator.wikimedia.org/T424815) (owner: 10Vaughn Walters) [19:01:56] (03CR) 10Jforrester: [C:03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/1302933 (https://phabricator.wikimedia.org/T424815) (owner: 10Vaughn Walters) [19:04:17] (03Merged) 10jenkins-bot: jjb: [catalyst-daily] Fetch & report Catalyst image [integration/config] - 10https://gerrit.wikimedia.org/r/1302933 (https://phabricator.wikimedia.org/T424815) (owner: 10Vaughn Walters) [19:05:40] (03PS1) 10Dduvall: zuul: Remove base parent from echo test job [integration/config] - 10https://gerrit.wikimedia.org/r/1302942 [19:06:28] 10Phabricator-Bot-Requests, 06Release-Engineering-Team (Doing 😎): Phabricator Bot Request: Test Kitchen Experiments Status Reporter - https://phabricator.wikimedia.org/T429377#12026104 (10Aklapper) 05Open→03In progress p:05Triage→03Medium a:03Aklapper Thanks for the ticket! @Test-Kitchen-Experiments-... [19:12:06] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily] Remove redundant ingress overrides [integration/config] - 10https://gerrit.wikimedia.org/r/1302943 (https://phabricator.wikimedia.org/T429390) [19:19:17] (03CR) 10Dduvall: [C:03+2] zuul: Remove base parent from echo test job [integration/config] - 10https://gerrit.wikimedia.org/r/1302942 (owner: 10Dduvall) [19:21:51] (03Merged) 10jenkins-bot: zuul: Remove base parent from echo test job [integration/config] - 10https://gerrit.wikimedia.org/r/1302942 (owner: 10Dduvall) [19:31:52] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily] Remove redundant ingress overrides [integration/config] - 10https://gerrit.wikimedia.org/r/1302943 (https://phabricator.wikimedia.org/T429390) [19:32:12] (03PS1) 10Dduvall: zuul: Remove base parent from directory-listing [integration/config] - 10https://gerrit.wikimedia.org/r/1302947 [19:32:22] (03CR) 10Dduvall: [C:03+2] zuul: Remove base parent from directory-listing [integration/config] - 10https://gerrit.wikimedia.org/r/1302947 (owner: 10Dduvall) [19:35:03] (03Merged) 10jenkins-bot: zuul: Remove base parent from directory-listing [integration/config] - 10https://gerrit.wikimedia.org/r/1302947 (owner: 10Dduvall) [19:36:10] (03PS2) 10Vaughn Walters: jjb: [catalyst-daily] Remove redundant ingress overrides [integration/config] - 10https://gerrit.wikimedia.org/r/1302943 (https://phabricator.wikimedia.org/T429390) [20:57:19] (03CR) 10Arnaudb: [C:03+1] "thanks for the +1s! I don't have enough permission to submit this, could someone take care of it?" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1302819 (https://phabricator.wikimedia.org/T420865) (owner: 10Arnaudb) [21:03:44] should "running" go dow while "waiting" goes up? https://grafana.wikimedia.org/d/ad656c66-d8b5-4b09-a54b-61e7df71fb17/zuul-3a-3a-gearman-prometheus?orgId=1&from=now-3h&to=now&timezone=utc&viewPanel=panel-21 [21:07:37] looks like the zuul postmerge queue is stuck again? [21:36:48] (03CR) 10Jeena Huneidi: [C:03+2] jjb: [catalyst-daily] Remove redundant ingress overrides [integration/config] - 10https://gerrit.wikimedia.org/r/1302943 (https://phabricator.wikimedia.org/T429390) (owner: 10Vaughn Walters) [21:40:19] (03Merged) 10jenkins-bot: jjb: [catalyst-daily] Remove redundant ingress overrides [integration/config] - 10https://gerrit.wikimedia.org/r/1302943 (https://phabricator.wikimedia.org/T429390) (owner: 10Vaughn Walters) [21:42:53] (03CR) 10Jeena Huneidi: [C:03+2] "Updated the jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/1302943 (https://phabricator.wikimedia.org/T429390) (owner: 10Vaughn Walters) [22:01:15] 10Beta-Cluster-Infrastructure: Eliminate sudo rm -rf /var/lib/puppet/ssl step in new deployment-prep WMCS project (and others) - https://phabricator.wikimedia.org/T429413 (10dancy) 03NEW [22:07:37] 10Beta-Cluster-Infrastructure: Eliminate sudo rm -rf /var/lib/puppet/ssl step in new deployment-prep WMCS project (and others) - https://phabricator.wikimedia.org/T429413#12027025 (10dancy) @Andrew Do you anticipate any issues if we set `profile::puppetserver::autosign` to `true` (or something) in the `deploymen... [23:37:47] (03CR) 10Vaughn Walters: "Okay thx, tests are all green too" [integration/config] - 10https://gerrit.wikimedia.org/r/1302943 (https://phabricator.wikimedia.org/T429390) (owner: 10Vaughn Walters)