[00:57:49] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06SRE, 10SRE-Access-Requests: Requesting access to contint-roots for Corvus - https://phabricator.wikimedia.org/T395167#10866331 (10KFrancis) Hello all, the NDA has been sent for signatures. I'll confirm when it's complete. [07:23:47] 06Project-Admins: Create a new SLO Phabricator tag - https://phabricator.wikimedia.org/T395537 (10elukey) 03NEW [09:24:27] 06Project-Admins: Create a new SLO Phabricator tag - https://phabricator.wikimedia.org/T395537#10866761 (10Aklapper) Hi, please see https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects and provide * Project Name * Public description (ideally understandable for anyone out there): When to add... [09:37:27] 06Project-Admins: Create a new SLO Phabricator tag - https://phabricator.wikimedia.org/T395537#10866801 (10elukey) The project is part of the efforts stated in "PES1.5 SLO Culture: defining ownership and commitments", and it should be used by SRE to tag the work related to infrastructure improvements and for oth... [12:28:53] 10Beta-Cluster-Infrastructure, 10Temporary accounts, 06Trust and Safety Product Team, 13Patch-For-Review, 10Trust and Safety Product Sprint (Sprint Carrot Cake (May 26 - June 13)): IP reveal rights: Beta wikis should not give rights to admins, bureaucrat... - https://phabricator.wikimedia.org/T395560#10867334 [12:33:14] 10Beta-Cluster-Infrastructure, 10Temporary accounts, 06Trust and Safety Product Team, 10Trust and Safety Product Sprint (Sprint Carrot Cake (May 26 - June 13)): IP reveal rights: Beta wikis should not give rights to admins, bureaucrats, and oversighters - https://phabricator.wikimedia.org/T395560#10867351 (... [12:33:22] 10Beta-Cluster-Infrastructure, 10Temporary accounts, 06Trust and Safety Product Team, 10Trust and Safety Product Sprint (Sprint Carrot Cake (May 26 - June 13)): IP reveal rights: Beta wikis should not give rights to admins, bureaucrats, and oversighters - https://phabricator.wikimedia.org/T395560#10867355... [13:19:41] 06Project-Admins: Create a new SLO Phabricator tag - https://phabricator.wikimedia.org/T395537#10867490 (10Jdforrester-WMF) I wonder if this is going to be best represented as a project or a tag. Project should be fine, though this is more a cross-cutting, on-going theme than a specific project? Eh. [13:23:45] 10Scap (SpiderPig πŸ•ΈοΈ): spiderpig: Prompt before closing windows with active deployments going on - https://phabricator.wikimedia.org/T395567 (10taavi) 03NEW [15:08:11] (03open) 10btullis: Fix the paths of some of the dumps scripts and config files [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/181 (https://phabricator.wikimedia.org/T394389) [15:18:14] 10Scap (SpiderPig πŸ•ΈοΈ): Spiderpig should HTML-escape the patch title - https://phabricator.wikimedia.org/T395574 (10cscott) 03NEW [15:44:33] 10Release-Engineering-Team (Radar), 10observability: https://logstash.wikimedia.org/app/dashboards#/view/mediawiki-errors not showing new data - https://phabricator.wikimedia.org/T395579#10868028 (10brennen) [15:45:14] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), 10Release Pipeline (Blubber): error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected char... - https://phabricator.wikimedia.org/T393856#10868031 [15:45:30] 10Release-Engineering-Team (Radar), 10observability, 10Wikimedia-Logstash: https://logstash.wikimedia.org/app/dashboards#/view/mediawiki-errors not showing new data - https://phabricator.wikimedia.org/T395579#10868033 (10brennen) [15:55:46] 10Release-Engineering-Team (Radar), 10observability, 10Wikimedia-Logstash: https://logstash.wikimedia.org/app/dashboards#/view/mediawiki-errors not showing new data - https://phabricator.wikimedia.org/T395579#10868067 (10brennen) β†’14Duplicate dup:03T390215 [16:03:51] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), 10Release Pipeline (Blubber): error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected char... - https://phabricator.wikimedia.org/T393856#10868102 [16:04:39] 10Scap (SpiderPig πŸ•ΈοΈ): Spiderpig should HTML-escape the patch title - https://phabricator.wikimedia.org/T395575#10868104 (10A_smart_kitten) [16:29:53] 06Release-Engineering-Team, 10MediaWiki-Debug-Logger, 10Wikimedia-Logstash: Kibana dashboard mediawiki-errors lacks channel errors and exceptions - https://phabricator.wikimedia.org/T341815#10868232 (10Krinkle) This is intentional. The way it is meant to work is that uncaught errors or errors that otherwise... [16:29:58] 06Release-Engineering-Team, 10MediaWiki-Debug-Logger, 10Wikimedia-Logstash, 07Developer Productivity, 10MediaWiki-Platform-Team (Radar): Kibana dashboard mediawiki-errors lacks channel errors and exceptions - https://phabricator.wikimedia.org/T341815#10868234 (10Krinkle) [16:53:25] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work: Buildkit v0.22.0 released - https://phabricator.wikimedia.org/T394931#10868380 (10dancy) [17:01:09] 06Release-Engineering-Team, 10MediaWiki-Debug-Logger, 10Wikimedia-Logstash, 07Developer Productivity, 10MediaWiki-Platform-Team (Radar): Logstash dashboard mediawiki-errors lacks "error"-level messages from diagnostic channels - https://phabricator.wikimedia.org/T341815#10868395 (10Krinkle) [17:38:19] (03update) 10addshore: Draft: Rewrite (maybe version 1)? [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/628 (https://phabricator.wikimedia.org/T395580) [17:38:27] 10GitLab (Project Migration), 10BDC-Implementation, 06collaboration-services, 10fundraising-tech-ops: All fundraising analytics host to access gitlab in addition to gerrit - https://phabricator.wikimedia.org/T393423#10868533 (10Dwisehaupt) 05Openβ†’03Resolved This was pushed a bit ago in conjunction... [17:39:41] (03open) 10addshore: Add novnc service [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/629 (https://phabricator.wikimedia.org/T395580) [17:44:18] (03update) 10addshore: Add novnc service [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/629 (https://phabricator.wikimedia.org/T395580) [17:44:35] (03update) 10addshore: Draft: Rewrite (maybe version 1)? [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/628 [17:48:50] 10GitLab (Pipeline Services Migration🐀), 06Release-Engineering-Team, 10dev-images, 10MediaWiki-Docker, 13Patch-For-Review: MySQL/MariaDB images for development environments - https://phabricator.wikimedia.org/T238925#10868595 (10Addshore) I'm removing the #mwcli tag from this now as this isn't really imp... [17:55:18] (03update) 10addshore: Add novnc service [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/629 (https://phabricator.wikimedia.org/T395580) [18:02:46] (03update) 10addshore: Draft: Rewrite (maybe version 1)? [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/628 [18:03:02] (03open) 10addshore: get-code dry run [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/630 [18:18:18] (03merge) 10addshore: Add novnc service [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/629 (https://phabricator.wikimedia.org/T395580) [18:27:34] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), 10Release Pipeline (Blubber): error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected char... - https://phabricator.wikimedia.org/T393856#10868722 [18:31:01] (03merge) 10addshore: get-code dry run [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/630 [18:37:07] (03update) 10addshore: Draft: Rewrite (maybe version 1)? [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/628 [19:19:10] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06SRE, 10SRE-Access-Requests: Requesting access to contint-roots for Corvus - https://phabricator.wikimedia.org/T395167#10868893 (10Arnoldokoth) [19:19:29] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), 10Release Pipeline (Blubber): error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected char... - https://phabricator.wikimedia.org/T393856#10868897 [19:29:40] (03open) 10cjming: Add deployer to UTC late backport window [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/182 [19:39:36] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work: Buildkit v0.22.0 released - https://phabricator.wikimedia.org/T394931#10868969 (10dancy) Note: I'm running the build process for v0.22.0 right now so that I can see if it resolves T393856. [19:41:55] (03merge) 10dancy: Add deployer to UTC late backport window [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/182 (owner: 10cjming) [19:44:42] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work: Buildkit v0.22.0 released - https://phabricator.wikimedia.org/T394931#10868977 (10dancy) >>! In T394931#10868969, @dancy wrote: > Note: I'm running the build process for v0.22.0 right now so that I can see if it r... [19:45:16] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work: Buildkit v0.22.0 released - https://phabricator.wikimedia.org/T394931#10868979 (10dancy) 05Openβ†’03Stalled Blocking this until T393856 is resolved. [19:50:11] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work: buildkit v0.21.1 released - https://phabricator.wikimedia.org/T393731#10868987 (10dancy) T393856 is a new problem since deploying buildkit 0.21.1. [19:52:13] 06Project-Admins, 06Release-Engineering-Team, 10MediaWiki-extensions-General, 10MediaWiki-General, and 2 others: Create a security pre-release Phabricator policy manageable by the Security Team - https://phabricator.wikimedia.org/T393403#10868992 (10Mstyles) 05Openβ†’03Resolved If you 're still able... [19:53:34] (03update) 10brennen: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [19:53:40] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), and 2 others: error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected character (at "ns-recu... - https://phabricator.wikimedia.org/T393856#10869001 [19:57:45] (03update) 10brennen: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [19:58:37] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work: buildkit v0.21.1 released - https://phabricator.wikimedia.org/T393731#10869005 (10dancy) Rolled back to v0.20.0 via https://gerrit.wikimedia.org/r/1152138 [20:32:17] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), 10Release Pipeline (Blubber): error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected char... - https://phabricator.wikimedia.org/T393856#10869115 [20:46:11] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), 10Release Pipeline (Blubber): error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected char... - https://phabricator.wikimedia.org/T393856#10869170 [20:51:38] (03update) 10brennen: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [21:07:49] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), and 2 others: error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected character (at "ns-recu... - https://phabricator.wikimedia.org/T393856#10869275 [21:08:08] (03update) 10brennen: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [21:13:39] 10Phabricator, 10Wikibugs: Wikibugs reports color of milestones wrong - https://phabricator.wikimedia.org/T395250#10869316 (10bd808) The current wikibugs logic [[https://gitlab.wikimedia.org/toolforge-repos/wikibugs2/-/blob/afbea805fbb6f7794e7cd4b308355fdb9f4d99ef/src/wikibugs2/messagebuilder.py#L194-207|maps... [22:11:58] !log Submitted WikimediaDebug v3.1.0 to Chrome Web Store for review (T395190, T315111) [22:12:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:12:01] T395190: WikimediaBackend ignores the last change to "backend" dropdown - https://phabricator.wikimedia.org/T395190 [22:12:01] T315111: Make WikimediaDebug status more sticky - https://phabricator.wikimedia.org/T315111 [22:18:08] !log Submitted WikimediaDebug v3.1.0 to addons.mozilla.org for review (T395190, T315111) [22:18:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:18:11] T395190: WikimediaBackend ignores the last change to "backend" dropdown - https://phabricator.wikimedia.org/T395190 [22:18:12] T315111: Make WikimediaDebug status more sticky - https://phabricator.wikimedia.org/T315111 [22:45:04] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10Metrics Platform, 10ci-test-error (WMF-deployed Build Failure), and 2 others: error: failed to solve: ParseAddr("ns-recursor.openstack.eqiad1.wikimediacloud.org"): unexpected character (at "ns-recu... - https://phabricator.wikimedia.org/T393856#10869615 [23:35:40] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T392173#10869728 (10thcipriani) 05Openβ†’03Resolved [23:50:17] 10Release-Engineering-Team (Doing 😎), 06Java-Scala-Standardization, 07Essential-Work: Renew Gitlab Maven CI_RELEASE_TOKEN - https://phabricator.wikimedia.org/T394827#10869776 (10thcipriani) [23:50:47] 10GitLab (Account Approval), 06Release-Engineering-Team, 07Essential-Work: Requesting GitLab account activation for 1f616emo - https://phabricator.wikimedia.org/T394668#10869777 (10thcipriani) [23:51:18] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 07Essential-Work: Only trigger Herald rule H339 when a task is created - https://phabricator.wikimedia.org/T394625#10869778 (10thcipriani) [23:52:21] 10Gerrit, 10Release-Engineering-Team (Doing 😎), 07Essential-Work: Incorrect "CI has completed checks" popup appears when navigating from a change with tests in progress to one with no tests in progress - https://phabricator.wikimedia.org/T394485#10869779 (10thcipriani) [23:52:53] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 06Trust-and-Safety, 07Essential-Work: Reset 2FA for Phabricator account @LaMagiaaa - https://phabricator.wikimedia.org/T393280#10869782 (10thcipriani) [23:53:31] 10Gerrit (Gerrit 3.10), 10Release-Engineering-Team (Doing 😎), 07Essential-Work: Upgrade to Gerrit 3.10.6 - https://phabricator.wikimedia.org/T390666#10869783 (10thcipriani) [23:53:47] 06Project-Admins, 10Release-Engineering-Team (Doing 😎), 06Language and Product Localization, 07Essential-Work: Archive #wikimedia-language-setup Phabricator project - https://phabricator.wikimedia.org/T389453#10869784 (10thcipriani) [23:54:25] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 07Essential-Work: Please remove an MFA factor from my Phabricator account - https://phabricator.wikimedia.org/T394763#10869785 (10thcipriani) [23:55:02] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 07Essential-Work: Restrict visibility of form 91 to #trusted-contributors ("Request creation of a new Wiki") - https://phabricator.wikimedia.org/T394845#10869786 (10thcipriani) [23:55:11] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 07Essential-Work: Call to a member function updateDatasourceTokens() on null at PhabricatorProjectFulltextEngine.php:20 (due to DB corruption) - https://phabricator.wikimedia.org/T345563#10869787 (10thcipriani)