[07:44:28] FIRING: PuppetSyncFailure: Failed to update Puppet repository /srv/git/operations/puppet on instance deployment-puppetserver-1 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetSyncFailure [07:44:34] 10Beta-Cluster-Infrastructure: Failed to update Puppet repository /srv/git/operations/puppet on instance deployment-puppetserver-1 in project deployment-prep - https://phabricator.wikimedia.org/T405547 (10wmcs-alerts) 03NEW [07:56:00] (03CR) 10Jforrester: [C:03+2] "It's not loaded on any production wiki. Let's do this now." [integration/config] - 10https://gerrit.wikimedia.org/r/1184838 (https://phabricator.wikimedia.org/T362317) (owner: 10Jforrester) [07:57:33] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Graph] Drop from production listing [integration/config] - 10https://gerrit.wikimedia.org/r/1184838 (https://phabricator.wikimedia.org/T362317) (owner: 10Jforrester) [07:57:56] !log Zuul: [mediawiki/extensions/Graph] Drop from production listing, for T362317 [07:57:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:58:00] T362317: Undeploy Graph from Wikimedia production wikis - https://phabricator.wikimedia.org/T362317 [08:00:54] 06Release-Engineering-Team, 10MediaWiki-extensions-Graph, 10MW-1.45-notes (1.45.0-wmf.18; 2025-09-09), 13Patch-For-Review, 07Technical-Debt: Undeploy Graph from Wikimedia production wikis - https://phabricator.wikimedia.org/T362317#11212942 (10Jdforrester-WMF) >>! In T362317#11197845, @Lejonel wrote: > W... [08:01:40] (03update) 10jforrester: Draft: Stop branching Graph for production, no longer used [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/156 (https://phabricator.wikimedia.org/T362317) [08:01:54] (03update) 10jforrester: Stop branching Graph for production, no longer used [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/156 (https://phabricator.wikimedia.org/T362317) [08:07:12] (03merge) 10jforrester: Stop branching Graph for production, no longer used [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/156 (https://phabricator.wikimedia.org/T362317) [08:14:41] 06Release-Engineering-Team, 10MediaWiki-extensions-Graph, 06translatewiki.net, 10MW-1.45-notes (1.45.0-wmf.18; 2025-09-09), and 2 others: Undeploy Graph from Wikimedia production wikis - https://phabricator.wikimedia.org/T362317#11212983 (10A_smart_kitten) [08:35:13] 06Release-Engineering-Team, 10MediaWiki-extensions-Graph, 06translatewiki.net, 10MW-1.45-notes (1.45.0-wmf.18; 2025-09-09), and 2 others: Undeploy Graph from Wikimedia production wikis - https://phabricator.wikimedia.org/T362317#11213053 (10Jdforrester-WMF) 05Openβ†’03Resolved a:03Jdforrester-WMF O... [08:46:43] 10Continuous-Integration-Infrastructure, 07Jenkins: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11213115 (10jnuche) @Dzahn thanks as always for your help. During the package installation Scap applies its own configuration to Jenkins, also moving... [08:49:59] 10Continuous-Integration-Config, 10Librarization, 07Test-Coverage: Run a phpunit coverage job pre-merge for libraries - https://phabricator.wikimedia.org/T147093#11213131 (10Reedy) Still would be useful to have ;) [08:50:30] 10Continuous-Integration-Config, 07Test-Coverage: Run proactive coverage checking on more repos - https://phabricator.wikimedia.org/T267781#11213136 (10Reedy) See also: {T147093} [11:00:31] (03CR) 10Jforrester: [C:03+2] zuul: Simplify WebAuthn removal to only be on REL1_39 [integration/config] - 10https://gerrit.wikimedia.org/r/1190769 (owner: 10Reedy) [11:02:16] (03Merged) 10jenkins-bot: zuul: Simplify WebAuthn removal to only be on REL1_39 [integration/config] - 10https://gerrit.wikimedia.org/r/1190769 (owner: 10Reedy) [11:07:53] !log Zuul: Simplify WebAuthn removal to only be on REL1_39 [11:07:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:20:09] (03PS1) 10Huajing: Zuul: [mediawiki/extensions/AdhocTranslation] Add BlueSpiceDiscovery skin dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1191366 [12:22:08] (03CR) 10CI reject: [V:04-1] Zuul: [mediawiki/extensions/AdhocTranslation] Add BlueSpiceDiscovery skin dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1191366 (owner: 10Huajing) [12:27:07] (03PS2) 10Huajing: Zuul: [mediawiki/extensions/AdhocTranslation] Add BlueSpiceDiscovery dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1191366 [12:49:43] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11214112 (10Dzahn) [13:07:45] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11214211 (10Dzahn) Hey @jnuche trying to debug this more I wanted to run the actual `update_jenkins.sh` script that you l... [13:15:53] Hi, I just used my first build argument in blubber and then discovered that it's a brand new feature: https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-5-0 [13:16:04] Wanted to drop by and say thanks, this is really useful. [13:17:04] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11214251 (10Dzahn) Notably the command in the sudoers.d file looks like this: `deploy-jenkins ALL=(root) NOPASSWD: /usr/bi... [13:22:01] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11214276 (10Dzahn) Might also be that just that the " need more escaping, so like: `deploy-jenkins ALL=(root) NOPASSWD: /u... [13:43:49] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11214355 (10jnuche) @Dzahn I really like the idea of putting the apt command in `/usr/local/bin/apt_update_jenkins`. Let's... [13:45:21] (03open) 10jnuche: update_jenkins.sh: move apt install command to a local file [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/107 (https://phabricator.wikimedia.org/T405352) [13:52:17] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T396381#11214404 (10Michael) I'm not sure if we consider {T405511} and/or {T405514} train blockers, but they now both have fixes attached... [13:52:34] (03update) 10jnuche: update_jenkins.sh: move apt install command to a local file [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/107 (https://phabricator.wikimedia.org/T405352) [13:56:37] (03update) 10jnuche: update_jenkins.sh: move apt install command to a local file [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/107 (https://phabricator.wikimedia.org/T405352) [13:57:24] (03update) 10jnuche: update_jenkins.sh: move apt install command to a local file [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/107 (https://phabricator.wikimedia.org/T405352) [14:17:55] 10Diffusion, 10Phabricator: Disable IO for diffusion repositories - https://phabricator.wikimedia.org/T405596 (10thcipriani) 03NEW [14:25:29] 06Project-Admins: Create project tag for Wikisource Reader App - https://phabricator.wikimedia.org/T405599 (10Saiphani02) 03NEW [14:26:59] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [YOUR DEVELOPER ACCOUNT USERNAME HERE] - https://phabricator.wikimedia.org/T405600 (10Alabnglangit1015512reas-collab) 03NEW [14:43:21] btullis: You can thank dduvall for that! [14:57:52] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services, 13Patch-For-Review: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11214895 (10Dzahn) @jnuche On the inactive host, releases2003, I have: - disabled puppet with a link... [15:01:45] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 13Patch-For-Review: Merge Phorge's upstream master (2025-09-08) into our wmf/stable - https://phabricator.wikimedia.org/T404134#11214917 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=2cd14582-9d2... [15:02:04] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 13Patch-For-Review: Merge Phorge's upstream master (2025-09-08) into our wmf/stable - https://phabricator.wikimedia.org/T404134#11214919 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=b53b2083-98f... [15:05:51] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 13Patch-For-Review: Merge Phorge's upstream master (2025-09-08) into our wmf/stable - https://phabricator.wikimedia.org/T404134#11214939 (10brennen) [15:18:53] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T396381#11214982 (10pmiazga) [15:19:01] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [YOUR DEVELOPER ACCOUNT USERNAME HERE] - https://phabricator.wikimedia.org/T405600#11214983 (10Aklapper) 05Openβ†’03Stalled Hi @Alabnglangit1015512reas-collab and welcome! What is your GitLab username? Which pro... [16:00:39] btullis: wow, you're the first user of it i think. let me know if you have issues :) [16:01:55] (03merge) 10dduvall: ci: Use .kokkuri:bake to build and publish image [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/160 [16:29:11] 10Phabricator (phabricator-next), 06Release-Engineering-Team (Doing 😎), 10Wikimedia-Phabricator-Extensions: Unnamed configuration source listed in all Settings - https://phabricator.wikimedia.org/T403909#11215383 (10Aklapper) [16:58:26] 10Phabricator (Upstream), 07Upstream: Authored include uploaded timestamp - https://phabricator.wikimedia.org/T391638#11215557 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [16:58:31] 10Phabricator, 06Release-Engineering-Team (Priority Backlog πŸ“₯): Upstream src/herald/SetSubtypeHeraldAction.php - https://phabricator.wikimedia.org/T389686#11215559 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [16:58:34] 10Phabricator (Upstream), 07Upstream: Document project picture dimension constraints - https://phabricator.wikimedia.org/T383083#11215561 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [16:58:35] 10Phabricator (Upstream), 07Upstream: Small typo in PhabricatorApplicationEditHTTPParameterHelpView.php - https://phabricator.wikimedia.org/T380365#11215563 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [16:58:37] 10Phabricator (Upstream), 07Upstream: Modified files not counted in total when attaching files - https://phabricator.wikimedia.org/T380361#11215565 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [16:58:38] 10Phabricator (Upstream), 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Upstream: DomainException: Attempting to iterate an object which is not iterable - https://phabricator.wikimedia.org/T373316#11215567 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.... [16:58:41] 10Phabricator (Upstream), 07Upstream: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857#11215569 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's s... [16:58:48] 10Phabricator (Upstream), 07Upstream: Upstream rPHABf2fd14dc1edeb41aa2874336548cfaa7fa0e87a0 (maniphest.gettasktransactions API) - https://phabricator.wikimedia.org/T364728#11215571 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software depl... [16:58:52] 10Phabricator (Upstream), 07Performance Issue, 07Upstream: ManiphestReportController uses a heavy/slow ManiphestTransaction table query - https://phabricator.wikimedia.org/T372219#11215572 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's soft... [16:58:56] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Exception: "Call to phutil_nonempty_string() expected null or a string, got: int" in PhabricatorRepository.php (due to Importing) - https://phabricator.wikimedia.org/T347483#11215575 (10Aklapper) 05Stalledβ†’03Resolved This iss... [16:59:00] 10Phabricator (Upstream), 07Upstream: Writing {{#something:}} into a Phab comment makes text disappear - https://phabricator.wikimedia.org/T326398#11215579 (10Aklapper) 05Stalledβ†’03Resolved a:03Aklapper This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `... [16:59:04] 10Phabricator (Upstream), 06Release-Engineering-Team (Seen), 07Upstream: Hovercards: (RuntimeException) Undefined index: objectPHID (due to RequestURI all lowercase) - https://phabricator.wikimedia.org/T346876#11215577 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wi... [16:59:17] 10Phabricator, 06Release-Engineering-Team (Priority Backlog πŸ“₯): Update to Phorge upstream 2025.18 release - https://phabricator.wikimedia.org/T386558#11215583 (10Aklapper) 05Stalledβ†’03Resolved a:03Aklapper Done via today's software deployment in `T404134`. [17:00:57] 10Gerrit, 07Upstream: Gerrit fudges multiple suggestions in comments - https://phabricator.wikimedia.org/T405625 (10Reedy) 03NEW [17:03:57] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Phabricator workboard import: Array for %Ls conversion is empty. Query: projectPHID IN (%Ls) - https://phabricator.wikimedia.org/T392168#11215629 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator... [17:04:01] 10Phabricator: Allow Security Team members to see MFA status of a Phab user account - https://phabricator.wikimedia.org/T401106#11215631 (10Aklapper) 05Stalledβ†’03Resolved a:03Aklapper This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [17:04:02] 10Phabricator (Upstream), 07Upstream: After archiving a Phabricator project, all future edits to it generate an extra "[username] set this project's color to Red" entry in the project's history - https://phabricator.wikimedia.org/T398735#11215634 (10Aklapper) 05Stalledβ†’03Resolved This issue should now... [17:04:05] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Performance Issue, 07Upstream: Ferret Fulltext search queries all apps' databases (also those of uninstalled apps) - https://phabricator.wikimedia.org/T397499#11215636 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed... [17:04:07] 10Phabricator (Upstream), 07Upstream: Unclear how to update email address of a Phabricator bot account - https://phabricator.wikimedia.org/T396881#11215638 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [17:04:11] 10Phabricator (Upstream), 10Wikibugs, 07Upstream: Wikibugs reports color of milestones wrong - https://phabricator.wikimedia.org/T395250#11215641 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`. [17:04:12] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Phabricator's dark mode produces white patches on collapsed sections - https://phabricator.wikimedia.org/T378408#11215643 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after tod... [17:04:14] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: New Column Trigger: When moving to a column, assign to the "current user" (user that is performing the move) - https://phabricator.wikimedia.org/T393268#11215645 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixe... [17:04:18] 10Phabricator (Upstream), 07Upstream: Alternative hashtag of milestone has no primary slug, thus project URL redirects to 404 /tag// - https://phabricator.wikimedia.org/T186173#11215652 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org after today's software... [17:04:21] 10Phabricator (Upstream), 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06Traffic, 07Upstream: Use preconnect for https://phab.wmfusercontent.org CDN - https://phabricator.wikimedia.org/T367290#11215648 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wikimedia.org... [17:04:28] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Query for global Feed Transaction Logs: Exception: ConpherenceTransactionQuery overheated - https://phabricator.wikimedia.org/T344232#11215650 (10Aklapper) 05Stalledβ†’03Resolved This issue should now be fixed on phabricator.wi... [17:04:38] 10Phabricator, 06Release-Engineering-Team (Priority Backlog πŸ“₯): Update to Phorge upstream rPca176821 / Arcanist upstream rARCd2d2afd0 (2025-09-08) - https://phabricator.wikimedia.org/T393840#11215654 (10Aklapper) 05Stalledβ†’03Resolved a:03Aklapper Done via today's software deployment in `T404134`. [17:07:18] 10Phabricator: Allow Security Team members to see MFA status of a Phab user account - https://phabricator.wikimedia.org/T401106#11215692 (10A_smart_kitten) >>! In T401106#11215631, @Aklapper wrote: > This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in `T404134`.... [17:08:20] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services: Upgrade phab (phorge) hosts to bookworm - https://phabricator.wikimedia.org/T372619#11215700 (10Aklapper) Today's upgrade of the Phabricator production instance in T404134 included many PHP8 fixes in the codebase, so I am personally fin... [17:11:41] (03update) 10aklapper: .arclint: Revert to standard phutil.xhpast [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/100 [17:12:42] (03merge) 10aklapper: .arclint: Revert to standard phutil.xhpast [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/100 [17:16:25] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Phabricator's dark mode produces white patches on collapsed sections - https://phabricator.wikimedia.org/T378408#11215756 (10Dzahn) Thanks, Dark Andre [17:17:30] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services: Upgrade phab (phorge) hosts to bookworm - https://phabricator.wikimedia.org/T372619#11215757 (10Dzahn) :)) that's great! yay [17:18:04] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services: Upgrade phab (phorge) hosts to bookworm - https://phabricator.wikimedia.org/T372619#11215759 (10Dzahn) @LSobanski should we raise the prio a bit from low? [17:27:20] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T396381#11215835 (10brennen) Planning to deploy backports prior to rolling train; currently waiting on all-clear for deployment server DC... [17:28:48] 10Phabricator: Allow Security Team members to see MFA status of a Phab user account - https://phabricator.wikimedia.org/T401106#11215839 (10Aklapper) 05Resolvedβ†’03Open Oops, yes. Thanks! [17:39:15] 10Phabricator, 06Release-Engineering-Team (Priority Backlog πŸ“₯): Adjust our Global Search dropdown options after fixing T393840 - https://phabricator.wikimedia.org/T401840#11215906 (10Aklapper) [17:40:30] 10Phabricator (2025-09-25), 06Release-Engineering-Team (Doing 😎), 06collaboration-services: Merge Phorge's upstream master (2025-09-08) into our wmf/stable - https://phabricator.wikimedia.org/T404134#11215914 (10Aklapper) 05Openβ†’03Resolved * List of most relevant changes: {J321} * Announcement: https... [17:56:45] 10Phabricator, 06Release-Engineering-Team (Priority Backlog πŸ“₯): Add recipient for automated weekly Phabricator data for Tech News - https://phabricator.wikimedia.org/T405638 (10Quiddity) 03NEW [18:04:19] 10Gerrit, 07Upstream: Gerrit fudges multiple suggestions in comments - https://phabricator.wikimedia.org/T405625#11216201 (10Paladox) This is fixed on gerrit @ master (I can reproduce on gerrit.wikimedia.orgbut not on gerrit-review). I dunno what fixed it. [18:08:55] 10Gerrit, 07Upstream: Gerrit fudges multiple suggestions in comments - https://phabricator.wikimedia.org/T405625#11216314 (10Paladox) Oh I think https://github.com/GerritCodeReview/gerrit/commit/3ffc3b13b6dea42079d1f98aa8c899df29832c6b fixes it. Which I backported here https://gerrit-review.googlesource.com/c... [18:15:23] 10Phabricator, 06Release-Engineering-Team (Doing 😎), 13Patch-For-Review: Add recipient for automated weekly Phabricator data for Tech News - https://phabricator.wikimedia.org/T405638#11216414 (10Aklapper) p:05Triageβ†’03Medium a:03Aklapper [18:35:51] 10Phabricator: Allow Security Team members to see MFA status of a Phab user account - https://phabricator.wikimedia.org/T401106#11216640 (10A_smart_kitten) > members of #acl_security_secteam. (I don't know which would be the best to use here, but FWIW #acl_security_team is another ACL project that seems to repr... [18:36:39] 10Phabricator: Request for acl*Batch-Editors for JVanderhoop_WMF - https://phabricator.wikimedia.org/T405649 (10JVanderhoop-WMF) 03NEW [18:44:48] 10Diffusion, 10Phabricator, 06collaboration-services: Disable IO for diffusion repositories - https://phabricator.wikimedia.org/T405596#11216692 (10Aklapper) Does this refer to all and any IO? For example, we mirror 26 repositories from Diffusion into Github (see also queries in T347577 and T349921). [18:46:23] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Request for acl*Batch-Editors for JVanderhoop_WMF - https://phabricator.wikimedia.org/T405649#11216697 (10Aklapper) 05Openβ†’03Resolved a:03Aklapper Here you go [18:54:24] 10GitLab (CI & Job Runners), 06Release-Engineering-Team (Doing 😎): Enable use of `.kokkuri:bake` on trusted runners - https://phabricator.wikimedia.org/T405651 (10dduvall) 03NEW [18:54:35] 10GitLab (CI & Job Runners), 06Release-Engineering-Team (Doing 😎): Enable use of `.kokkuri:bake` on trusted runners - https://phabricator.wikimedia.org/T405651#11216730 (10dduvall) 05Openβ†’03In progress a:03dduvall [18:55:15] 10Diffusion, 10Phabricator, 06collaboration-services: Disable IO for diffusion repositories - https://phabricator.wikimedia.org/T405596#11216732 (10Tacsipacsi) What does β€œIO” mean in this context at all? Is it some kind of Phabricator jargon? In a general sense, I/O would include everything that happens with... [19:08:37] 10Diffusion, 10Phabricator, 06collaboration-services: Disable IO for diffusion repositories - https://phabricator.wikimedia.org/T405596#11216788 (10Dzahn) The I/O term is in the Diffusion (Phabricator) settings. If you go to Diffusion and hit "Manage Repository" one of the repos there are a couple settings.... [19:20:51] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 13Patch-For-Review, 05Release, 05Train Deployments: 1.45.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T396381#11216850 (10brennen) Blocker resolved, logs clean-ish, rolling to all wikis. [19:46:20] 10Diffusion, 10Phabricator, 06collaboration-services: Disable IO for diffusion repositories - https://phabricator.wikimedia.org/T405596#11216949 (10Tacsipacsi) Thanks for the pointers! At https://phabricator.wikimedia.org/source/mediawiki/manage/uris/, I see * one //No I/O// entry pointing to https://github... [20:24:43] 06Release-Engineering-Team (Radar), 06serviceops-radar, 06SRE: deployment server - low disk space on /srv - https://phabricator.wikimedia.org/T387796#11217157 (10Dzahn) We got a new alert today. It was deploy1003 at 98% on /srv/. Still about /srv/docker. [20:26:56] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T396381#11217190 (10brennen) Currently stable on all wikis. [20:33:04] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Add recipient for automated weekly Phabricator data for Tech News - https://phabricator.wikimedia.org/T405638#11217214 (10Aklapper) 05Openβ†’03Resolved [21:56:19] 06Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T405677#11217470 (10thcipriani) p:05Triageβ†’03Medium a:03hashar [21:56:51] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T405678#11217475 (10thcipriani) p:05Triageβ†’03Medium a:03jnuche [21:57:25] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T405679#11217481 (10thcipriani) p:05Triageβ†’03Medium a:03Aklapper [21:58:14] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T405680#11217486 (10thcipriani) p:05Triageβ†’03Medium a:03dancy [21:58:46] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T405681#11217491 (10thcipriani) p:05Triageβ†’03Medium a:03dduvall [23:31:25] 10Continuous-Integration-Infrastructure, 07Jenkins, 06collaboration-services, 13Patch-For-Review: APT error when installing Jenkins package in releases instances - https://phabricator.wikimedia.org/T405352#11217717 (10Dzahn) @jnuche I re-enabled puppet because that was the worst timing. Did that right befo...