[00:27:16] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10collaboration-services, 10Patch-For-Review, and 2 others: Many (all?) of the phabricator/tools scripts are in Python 2 - https://phabricator.wikimedia.org/T355574 (10Dzahn) The dump script works again. It was succesfully converted by Brennen... [00:29:50] 10Phabricator (2024-02-13), 10Dumps-Generation, 10collaboration-services, 10Patch-For-Review, 10User-brennen: phabricator_task_dump.service Failed on phab1004 - https://phabricator.wikimedia.org/T355502 (10Dzahn) T355574#9540664 The dump script works again. It was succesfully converted by Brennen. Afte... [00:33:10] 10Phabricator (2024-02-13), 10Dumps-Generation, 10collaboration-services, 10Patch-For-Review, 10User-brennen: phabricator_task_dump.service Failed on phab1004 - https://phabricator.wikimedia.org/T355502 (10Dzahn) [00:36:46] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10collaboration-services, 10Patch-For-Review, and 2 others: Many (all?) of the phabricator/tools scripts are in Python 2 - https://phabricator.wikimedia.org/T355574 (10Dzahn) [00:36:57] 10Phabricator (2024-02-13), 10Dumps-Generation, 10collaboration-services, 10Patch-For-Review, 10User-brennen: phabricator_task_dump.service Failed on phab1004 - https://phabricator.wikimedia.org/T355502 (10Dzahn) [00:37:05] 10Phabricator (2024-02-13), 10Dumps-Generation, 10collaboration-services, 10Patch-For-Review, 10User-brennen: phabricator_task_dump.service Failed on phab1004 - https://phabricator.wikimedia.org/T355502 (10Dzahn) 05In progressβ†’03Resolved ` 00:30 < jinxer-wm> (SystemdUnitFailed) firing: phabricator_ta... [00:37:14] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10collaboration-services, 10Patch-For-Review, and 2 others: Many (all?) of the phabricator/tools scripts are in Python 2 - https://phabricator.wikimedia.org/T355574 (10Dzahn) I think next is we can delete the "trello" scripts. [01:50:38] 10Project-Admins, 10Security-Team: Create new subproject for acl*security - https://phabricator.wikimedia.org/T357487 (10DannyS712) [01:51:13] 10Project-Admins, 10Security-Team: Create new subproject for acl*security - https://phabricator.wikimedia.org/T357487 (10DannyS712) [09:30:09] * hashar shakes fists at `make` [09:45:48] today I have learned about `:::=` as an assignment operator in GNU make ... [09:48:20] o_O [09:49:15] wow, `!=` as shell assignment sounds almost more evil [09:59:57] I have an abusive relationship with Make :) [11:17:48] 10Release-Engineering-Team, 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Move 50% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T357507 (10Clement_Goubert) [11:18:03] `beta-scap-sync-world` either takes ~2 minutes or 11 minutes, and will always take 11 minutes when I'm actively waiting for something to test (: https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-sync-world/buildTimeTrend [11:18:14] Thanks Beta. [11:20:12] 10Release-Engineering-Team, 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Move 50% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T357507 (10Clement_Goubert) p:05Triageβ†’03High [11:20:55] 10Release-Engineering-Team, 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Move 60% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T357508 (10Clement_Goubert) [11:21:31] 10Release-Engineering-Team, 10Scap, 10MW-on-K8s, 10SRE, 10serviceops: Scap should check errors coming from mw-on-k8s canaries during deployments - https://phabricator.wikimedia.org/T357402 (10Clement_Goubert) [11:21:37] 10Release-Engineering-Team, 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Move 60% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T357508 (10Clement_Goubert) [11:23:43] 10Release-Engineering-Team, 10Scap, 10MW-on-K8s, 10SRE, 10serviceops: Scap should check errors coming from mw-on-k8s canaries during deployments - https://phabricator.wikimedia.org/T357402 (10Clement_Goubert) [11:23:49] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [11:25:29] 10Release-Engineering-Team, 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Move 60% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T357508 (10Clement_Goubert) 05Openβ†’03Stalled p:05Triageβ†’03High We'll stay at 50% or under because we want scap to check canaries before... [11:26:51] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [11:38:34] (03PS1) 10Zfilipin: jjb: Cleanup of selenium-daily-beta-* jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1003396 (https://phabricator.wikimedia.org/T188742) [11:44:25] (03PS1) 10Zfilipin: jjb: Delete selenium-daily-beta-Wikistories [integration/config] - 10https://gerrit.wikimedia.org/r/1003398 (https://phabricator.wikimedia.org/T357510) [11:46:34] (03PS1) 10Zfilipin: jjb: Delete selenium-daily-beta-CirrusSearch [integration/config] - 10https://gerrit.wikimedia.org/r/1003399 (https://phabricator.wikimedia.org/T357509) [11:55:17] (03CR) 10Jbol: [C: 03+1] "Looks good to me, but someone else should also confirm since I'm not so closely familiar with this project" [integration/config] - 10https://gerrit.wikimedia.org/r/1003396 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [12:52:58] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Add-Link, 10Growth-Team, and 2 others: research/mwaddlink has failing CI on the main branch - https://phabricator.wikimedia.org/T357217 (10kevinbazira) Following T357217#9534633, I had a chat with @MGerlach and option 1 is the most feasible at... [13:00:25] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/1003398 (https://phabricator.wikimedia.org/T357510) (owner: 10Zfilipin) [13:01:34] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/1003399 (https://phabricator.wikimedia.org/T357509) (owner: 10Zfilipin) [13:02:16] (03Merged) 10jenkins-bot: jjb: Delete selenium-daily-beta-Wikistories [integration/config] - 10https://gerrit.wikimedia.org/r/1003398 (https://phabricator.wikimedia.org/T357510) (owner: 10Zfilipin) [13:03:14] (03Merged) 10jenkins-bot: jjb: Delete selenium-daily-beta-CirrusSearch [integration/config] - 10https://gerrit.wikimedia.org/r/1003399 (https://phabricator.wikimedia.org/T357509) (owner: 10Zfilipin) [13:04:52] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/1003396 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [13:06:20] (03Merged) 10jenkins-bot: jjb: Cleanup of selenium-daily-beta-* jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1003396 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [13:48:57] 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10MediaWiki-Core-Tests, 10Code-Health, and 8 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Tgr) I think there are two common use-cases where slow CI is annoying:... [13:57:33] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) It's getting routed to WMCS's main mailservers: ` 2024-02-14 13:01:57 1raEth-0006mk-Lf <= wiki-enwiki-s9-s8uk79-se/dKyYlA6mL... [13:58:04] 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10MediaWiki-Core-Tests, 10Code-Health, and 8 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Daimona) Agreed that early feedback can help. One more thing to consid... [14:04:50] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) ` Notice: /Stage[main]/Exim4/File[/etc/exim4/exim4.conf]/content: --- /etc/exim4/exim4.conf 2023-12-11 16:45:46.519337352 +... [14:05:48] FWIW, I fixed a puppet rebase issue and now a lot of puppet changes are trickling in [14:05:59] (in beta cluster) [14:11:02] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) Prgoress? ` 2024-02-14 14:09:13 1raFwn-0005fc-Me <= wiki@wikimedia.beta.wmflabs.org U=www-data P=local S=2898 id=enwiki.65cc... [14:11:55] 10GitLab (Pipeline Services Migration🐀), 10collaboration-services: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab - https://phabricator.wikimedia.org/T300171 (10Jelto) [14:23:16] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) Replaced the host with IP and now I'm getting this: ` 2024-02-14 14:21:08 1raG8J-0002l3-Tc ** ladsgroup+beta@gmail.com R=wik... [14:28:12] (03PS1) 10Jaime Nuche: jjb: overhaul Zuul deploy jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1003437 [14:28:46] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) I love this, the relay allows internal IPs ` hostlist relay_from_hosts = <; @[] ; 127.0.0.1 ; ::1 ; 172.16.0.0/12 ; 127.0.0.... [14:40:38] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Add-Link, 10Growth-Team, and 2 others: research/mwaddlink has failing CI on the main branch - https://phabricator.wikimedia.org/T357217 (10kevinbazira) Finally got the test build to succeed. @kostajh and @Urbanecm_WMF please review whenever you... [14:50:19] !log Zuul: [mediawiki/vendor] Test composer lint with PHP 8.0+ too for T300463 T316078 T352085 T353362 [14:50:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:50:26] T300463: Make PHP 8.0 voting on MW master - https://phabricator.wikimedia.org/T300463 [14:50:27] T316078: Make PHP 8.1 voting on development (master) branch of MW ecosystem (core, extensions, skins, libraries) - https://phabricator.wikimedia.org/T316078 [14:50:27] T352085: Make PHP 8.2 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T352085 [14:50:27] T353362: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362 [14:52:06] 10Continuous-Integration-Config, 10PHP 8.3 support, 10Patch-For-Review: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362 (10Jdforrester-WMF) [14:52:08] 10Continuous-Integration-Config, 10PHP 8.2 support, 10Patch-For-Review: Make PHP 8.2 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T352085 (10Jdforrester-WMF) [14:52:15] Progress! [14:54:16] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) The whitespace change fixed the lookup and **now it should be working**. I got the mail as well. Headers look fine too: ` R... [15:00:45] jnuche: the more I have looked at the Zuul deployment Makefiles, the more I think that those should be included in the python-build docker image [15:01:17] this way it would be shared among repositories. I have looked at a few more /deploy repositories and each are a web of forked copy/pastas :-( [15:02:29] hashar: I would like to get something working first with the current setup we have, then we can think of improvements if that's ok [15:12:09] 10Project-Admins, 10Security-Team, 10SecTeam-Processed: Create new subproject for acl*security - https://phabricator.wikimedia.org/T357487 (10sbassett) 05Openβ†’03Resolved p:05Triageβ†’03Low **Done:** https://phabricator.wikimedia.org/project/view/6986/ I believe this should automatically inherit the pe... [15:23:56] 10Project-Admins, 10Security-Team, 10SecTeam-Processed: Create new subproject for acl*security - https://phabricator.wikimedia.org/T357487 (10DannyS712) 05Resolvedβ†’03Open p:05Lowβ†’03Unbreak! https://phabricator.wikimedia.org/project/members/6986/ reports "Joinable By" as "All Users" Moving to unbreak... [15:27:38] 10Project-Admins, 10Security-Team, 10SecTeam-Processed: Create new subproject for acl*security - https://phabricator.wikimedia.org/T357487 (10sbassett) 05Openβ†’03Resolved p:05Unbreak!β†’03Low >>! In T357487#9542727, @DannyS712 wrote: > https://phabricator.wikimedia.org/project/members/6986/ reports "Joi... [15:58:29] 10Release-Engineering-Team (Quid Pro Crow πŸ¦ƒ): Aklapper to watch / learn Phabricator deployment - https://phabricator.wikimedia.org/T343466 (10thcipriani) 05In progressβ†’03Resolved I think @Aklapper and @brennen have paired a few times on this and that will likely continue. Declaring this complete. [15:59:12] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10serviceops-radar, 10Python3-Porting: git-fat replacement/removal - https://phabricator.wikimedia.org/T279509 (10thcipriani) [15:59:41] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10MW-on-K8s, 10SRE, 10serviceops: Scap should check errors coming from mw-on-k8s canaries during deployments - https://phabricator.wikimedia.org/T357402 (10thcipriani) [15:59:55] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Upstream: Maniphest Advanced Search form should display "100" in its "Page Size" field by default - https://phabricator.wikimedia.org/T288915 (10thcipriani) [16:01:16] 10Phabricator (2024-01-16), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Unused custom maniphest.project.task.transactions Conduit API has issues - https://phabricator.wikimedia.org/T353042 (10thcipriani) [16:07:54] 10Fresnel, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10patch-welcome: Add scenario options to Fresnel comparison output - https://phabricator.wikimedia.org/T217856 (10thcipriani) [16:08:44] 10Release-Engineering-Team (Now this 🫠), 10Security-Team, 10SecTeam-Processed: Notify MediaWiki security tasks as soon as an uploaded patch fails to apply - https://phabricator.wikimedia.org/T350065 (10thcipriani) [16:08:59] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10CAS-SSO, 10Infrastructure-Foundations, 10Patch-For-Review: Correct IDP login page Privacy Policy - https://phabricator.wikimedia.org/T350129 (10thcipriani) [16:09:14] 10Phabricator-Sprint-Extension, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Diffusion-Repository-Administrators, 10Projects-Cleanup, and 2 others: Archive broken and unmaintained Phabricator Sprint extension - https://phabricator.wikimedia.org/T275325 (10thcipriani) [16:10:00] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10collaboration-services: Get GitLab to render `T{\d}+` in MR overviews, comments, etc. as links to Phabricator - https://phabricator.wikimedia.org/T337570 (10thcipriani) [16:10:26] 10Continuous-Integration-Config, 10Release-Engineering-Team (Now this 🫠), 10Language-Team (Language-2024-January-March), 10affects-translatewiki.net: jenkins.job=mediawiki-i18n-check-docker fails without a real error - https://phabricator.wikimedia.org/T331098 (10thcipriani) [16:10:47] 10Gerrit, 10GitLab (Misc), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Technical-Tool-Request, and 2 others: A tool for displaying the canonical location of all Wikimedia repositories and any known mirrors - https://phabricator.wikimedia.org/T330347 (10thcipriani) [16:11:51] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Zuul: Proof-of-concept: Run MediaWiki tests in latest Zuul - https://phabricator.wikimedia.org/T349214 (10thcipriani) [16:11:59] 10GitLab (Misc), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Puppetize Zuul v9 - https://phabricator.wikimedia.org/T350814 (10thcipriani) [16:15:11] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10Patch-For-Review: Scap backporting a patch that gets a -2 hangs - https://phabricator.wikimedia.org/T350628 (10thcipriani) [16:16:05] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) An update: Gmail is happy now but our own wikimedia.org servers don't get the mails: ` 2024-02-14 16:02:17 H=deployment-med... [16:19:24] 10Release-Engineering-Team (Backlog), 10SRE: mwdebug2001 and mwdebug2002 "/" almost full - https://phabricator.wikimedia.org/T219989 (10thcipriani) [16:19:28] 10Release-Engineering-Team (Seen), 10Scap: `scap clean` failure - https://phabricator.wikimedia.org/T218783 (10thcipriani) 05Openβ†’03Invalid We've refactored this a few times, now no one runs `scap clean` manually, it's a part of `scap train` and it runs every week. I don't think this specific task is still... [16:25:43] 10Release-Engineering-Team (Seen), 10Scap: scap clean leaving lots of empty directories on mw hosts - https://phabricator.wikimedia.org/T252177 (10thcipriani) 05Openβ†’03Resolved a:03thcipriani This is resolved at some point in our refactoring of train deploys. I spot-checked a couple instances and I'm no... [16:56:53] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Add-Link, 10Growth-Team, and 2 others: research/mwaddlink has failing CI on the main branch - https://phabricator.wikimedia.org/T357217 (10Urbanecm_WMF) Hi @kevinbazira, thanks for working on this issue and for submitting a patch. I'm, however,... [16:58:58] 10Release-Engineering-Team, 10Release Pipeline (Blubber): Blubber Python builder: Support virtualenv setup - https://phabricator.wikimedia.org/T357548 (10dancy) [17:00:31] 10Release-Engineering-Team, 10Release Pipeline (Blubber): Blubber Python builder: Support virtualenv setup - https://phabricator.wikimedia.org/T357548 (10dancy) [17:00:38] 10Release-Engineering-Team, 10Release Pipeline (Blubber): Blubber: Add workaround for Bookworm's Python pip restrictions - https://phabricator.wikimedia.org/T346090 (10dancy) [17:02:04] 10Continuous-Integration-Infrastructure, 10EntitySchema, 10Wikidata, 10wmde-wikidata-tech: [ES] Investigate OutOfMemory error in browser test, remove workaround - https://phabricator.wikimedia.org/T356402 (10Lucas_Werkmeister_WMDE) In T356896 it looks like we’re hitting a similar issue in QUnit tests – at... [17:02:22] legoktm: just a note that I spent ~30 minutes yesterday trying to figure out why "import phabricator" worked in wikibugs.py before eventually realizing that "fab" was your phabricator package and not something related to https://www.fabfile.org/ 🀣 [17:53:58] 10Continuous-Integration-Config, 10Release-Engineering-Team (Now this 🫠), 10Language-Team (Language-2024-January-March), 10affects-translatewiki.net: jenkins.job=mediawiki-i18n-check-docker fails without a real error - https://phabricator.wikimedia.org/T331098 (10thcipriani) a:03hashar [17:56:53] bd808: oh no :((( I'm so sorry [18:00:14] 10Release-Engineering-Team (Now this 🫠), 10Release Pipeline (Blubber): Blubber Python builder: Support virtualenv setup - https://phabricator.wikimedia.org/T357548 (10thcipriani) [18:15:19] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10collaboration-services, 10User-brennen: scap install fails on new Phabricator/Phorge host due to missing user - https://phabricator.wikimedia.org/T357572 (10brennen) [18:17:38] legoktm: naming is hard :) [18:23:59] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Emails sent from @wikimedia.beta.wmflabs.org do not reach intended recipients - https://phabricator.wikimedia.org/T343925 (10Ladsgroup) Cherry-picked this in beta cluster and it works. Basically now this is fixed, I only keep it open to get the puppet patches... [18:32:34] 10Release-Engineering-Team (Onboarding πŸš€), 10Scap: scap backport check prevents using it to fix an about-to-be-live branch - https://phabricator.wikimedia.org/T317795 (10dancy) [18:38:05] 10Release-Engineering-Team (Onboarding πŸš€), 10Scap: scap backport check prevents using it to fix an about-to-be-live branch - https://phabricator.wikimedia.org/T317795 (10dancy) [18:55:31] 10Release-Engineering-Team (Onboarding πŸš€), 10Scap: scap backport check prevents using it to fix an about-to-be-live branch - https://phabricator.wikimedia.org/T317795 (10dancy) I ran this scenario in train-dev today and the behavior has changed since this ticket was filed: ` debian@deploy:~$ scap backport 101... [19:49:28] 10Release-Engineering-Team, 10Scap: scap backport: Unexpected "not found in any deployed wikiversion" warning - https://phabricator.wikimedia.org/T357578 (10dancy) [19:58:06] 10Release-Engineering-Team, 10Scap: scap backport: Unexpected "not found in any deployed wikiversion" warning - https://phabricator.wikimedia.org/T357578 (10dancy) 05Openβ†’03Resolved a:03dancy Not a bug! I forgot that train-dev's `operations/mediawiki-config` uses branch `train-dev`. [20:03:18] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps opened https://gitlab.wikimedia.org/repos/releng/gitlab-terraform-images/-/merge_reques... [20:23:19] 10Release-Engineering-Team, 10Scap: scap wikiversions-inuse: Always read wikiversions from the staging directory - https://phabricator.wikimedia.org/T357581 (10dancy) [20:30:07] 10Release-Engineering-Team, 10Scap, 10Patch-For-Review: scap wikiversions-inuse: Always read wikiversions from the staging directory - https://phabricator.wikimedia.org/T357581 (10CodeReviewBot) dancy opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/212 scap wikiversions-inuse: Always... [20:33:44] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps merged https://gitlab.wikimedia.org/repos/releng/gitlab-terraform-images/-/merge_reques... [20:37:43] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps opened https://gitlab.wikimedia.org/repos/releng/gitlab-terraform-images/-/merge_reques... [20:43:20] 10Release-Engineering-Team (Now this 🫠), 10Scap: scap backport check prevents using it to fix an about-to-be-live branch - https://phabricator.wikimedia.org/T317795 (10dancy) [20:43:41] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps merged https://gitlab.wikimedia.org/repos/releng/gitlab-terraform-images/-/merge_reques... [21:22:47] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10Patch-For-Review: scap backport check prevents using it to fix an about-to-be-live branch - https://phabricator.wikimedia.org/T317795 (10CodeReviewBot) dancy opened https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/34 deploy/scripts/scap-... [21:23:36] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10Patch-For-Review: scap backport check prevents using it to fix an about-to-be-live branch - https://phabricator.wikimedia.org/T317795 (10CodeReviewBot) dancy merged https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/34 deploy/scripts/scap-... [21:38:45] 10Project-Admins, 10Movement-Insights: Create project tags for Movement Insights teams - https://phabricator.wikimedia.org/T356553 (10Mayakp.wiki) p:05Triageβ†’03Medium [22:02:00] 10Release-Engineering-Team (Now this 🫠), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10dancy) [22:08:05] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Release, 10Train Deployments, 10User-brennen: 1.42.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T354436 (10Msz2001) There's a potentially highly visible regression for wikitables on mobile: {T357589} I didn't mark it as blocker but if... [22:09:16] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10Patch-For-Review: scap backport check prevents using it to fix an about-to-be-live branch - https://phabricator.wikimedia.org/T317795 (10CodeReviewBot) dancy opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/214 Add test case for backpo... [22:24:29] 10Release-Engineering-Team (Now this 🫠), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps merged https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/337 upda... [22:43:09] 10Release-Engineering-Team (Now this 🫠), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps opened https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/338 inlu... [22:47:50] 10Release-Engineering-Team (Now this 🫠), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps updated https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/338 upd... [22:53:17] 10Release-Engineering-Team (Now this 🫠), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps merged https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/338 upda... [23:32:15] 10Release-Engineering-Team (Now this 🫠), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps opened https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/339 set... [23:52:58] 10Release-Engineering-Team (Now this 🫠), 10Patch-For-Review: gitlab-cloud-runner: Roll back pending helm releases before running terraform apply - https://phabricator.wikimedia.org/T354787 (10CodeReviewBot) sandeeps merged https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/339 set...