[00:50:06] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for adstein - https://phabricator.wikimedia.org/T439655#12398596 (10Adstein1) I plan to work on [[ https://www.mediawiki.org/wiki/Mwbot-rs#Contributing | mwbot-rs. ]] [03:50:51] (03PS1) 10Phedenskog: Add an option to delete node_modules after each browser test projec [integration/quibble] - 10https://gerrit.wikimedia.org/r/1351621 (https://phabricator.wikimedia.org/T439717) [07:28:49] 10Phabricator, 06Infrastructure-Foundations: Removing an offboarded user from privileged Phab ACL projects should not require admins - https://phabricator.wikimedia.org/T392111#12399062 (10MoritzMuehlenhoff) 05Open→03Resolved >>! In T392111#10835600, @Aklapper wrote: > Thank you! If @MoritzMuehlenhoff... [07:34:30] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: CI build slow and timing out sometimes - https://phabricator.wikimedia.org/T434024#12399093 (10Peter) I did some quick tests on a test agent (9999)... [08:21:02] 10Diffusion, 10Phabricator: Bad repo data: PhabricatorDataNotAttachedException browsing particular git repositories - https://phabricator.wikimedia.org/T421682#12399387 (10Aklapper) I merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/commit/3528a8b38b6ccc1534ca47719d41447b07abff2b as a simple... [08:46:38] (03CR) 10Gkyziridis: inference-services: Add the revertrisk-wikidata test pipeline (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1351199 (https://phabricator.wikimedia.org/T433699) (owner: 10Gkyziridis) [08:56:37] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Potentially disable H489 - https://phabricator.wikimedia.org/T431735#12399461 (10Aklapper) 05Open→03Resolved a:03Aklapper @Jony: No reply; I've deactivated H489 for now. [09:01:32] 10Phabricator (Upstream), 10Wikimedia-Phabricator-Extensions, 07Upstream: Add sortable headers for Related Changes in Gerrit - https://phabricator.wikimedia.org/T422867#12399507 (10Aklapper) 05Open→03Stalled Stalled on finishing / merging an upstream patch for https://we.phorge.it/T16699 before this coul... [10:19:58] 10Phabricator (Upstream), 07Upstream: EXCEPTION: (AphrontSchemaQueryException) #1054: Unknown column 'projectGroupName.indexedObjectName' in 'HAVING' - https://phabricator.wikimedia.org/T428155#12399819 (10Aklapper) NB: My assumption above is wrong as it breaks results paging [10:31:21] 10Fresh, 06Release-Engineering-Team, 10Growth Team Maintenance, 06Growth-Team, 10GrowthExperiments: npm run cy:open does not open a browser from fresh-node - https://phabricator.wikimedia.org/T439551#12399840 (10EMcFarland-WMF) [10:32:56] 10Fresh, 06Release-Engineering-Team, 10Growth Team Maintenance, 06Growth-Team, 10GrowthExperiments: npm run cy:open does not open a browser from fresh-node - https://phabricator.wikimedia.org/T439551#12399849 (10EMcFarland-WMF) Hi @hashar, I wanted to make you aware of this task because I saw you tagged... [11:42:16] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Potentially disable H489 - https://phabricator.wikimedia.org/T431735#12400106 (10Jony) @Aklapper Apologies for the delayed response; for the time being, it is possible for me to monitor the matter as a member of these projects. [12:09:31] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T438219#12400180 (10ihurbain) [12:10:18] 10Continuous-Integration-Infrastructure, 07Composer: Update composer on CI from 2.10.2 to 2.10.3 - https://phabricator.wikimedia.org/T440297 (10hashar) 03NEW [12:10:48] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T438219#12400198 (10ihurbain) I added https://phabricator.wikimedia.org/T440296 to this because I'm reasonably sure that it's a new behavi... [12:11:56] 10Continuous-Integration-Infrastructure, 07Composer: Update composer on CI from 2.10.2 to 2.10.3 - https://phabricator.wikimedia.org/T440297#12400212 (10hashar) [12:12:06] 10Continuous-Integration-Config, 07Composer: Update composer on CI from 2.10.2 to 2.10.3 - https://phabricator.wikimedia.org/T440297#12400213 (10hashar) [12:12:59] 10Phabricator, 06Security-Team, 07SecTeam-Processed: Audit Phabricator security policies and groups membership - https://phabricator.wikimedia.org/T391150#12400216 (10Aklapper) I assume this ticket is supposed to track some //automation//, so tickets like {T397734} don't get forgotten? It's not clear from th... [12:13:13] 10Phabricator, 06Security-Team, 07SecTeam-Processed: Audit Phabricator security policies and groups membership - https://phabricator.wikimedia.org/T391150#12400220 (10Aklapper) [12:15:53] 10Phabricator: Disabled workboards are still linked by tags (requires removing the Pin icon for the workboard under "Project Details") - https://phabricator.wikimedia.org/T127908#12400235 (10Aklapper) FYI, upstream "When archiving a project, reset its default menu item to Profile" in https://we.phorge.it/T15882... [12:17:49] 10Phabricator: Disabled workboards are still linked by tags (requires removing the Pin icon for the workboard under "Project Details") - https://phabricator.wikimedia.org/T127908#12400244 (10Aklapper) We could deploy https://we.phorge.it/D25731 as a downstream patch if really wanted (and after more testing). Ho... [12:19:17] (03PS1) 10Hashar: dockerfiles: update composer from 2.10.2 to 2.10.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1352045 (https://phabricator.wikimedia.org/T440297) [12:21:04] 10Continuous-Integration-Config, 07Composer, 13Patch-For-Review: Update composer on CI from 2.10.2 to 2.10.3 - https://phabricator.wikimedia.org/T440297#12400258 (10hashar) [12:31:17] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Update to Phorge/Arcanist upstream 2026.xx - https://phabricator.wikimedia.org/T428547#12400299 (10Aklapper) [12:31:19] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Missing space between two Diffusion strings / messages - https://phabricator.wikimedia.org/T304205#12400298 (10Aklapper) [12:32:23] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1350083 [12:32:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:33:58] 10Phabricator (Upstream), 07Upstream: Allow varying notification settings (eg: notify vs email) by tag/project - https://phabricator.wikimedia.org/T165832#12400306 (10Aklapper) 05Open→03Declined I don't see this getting implemented in upstream. Closing to reflect reality. [12:34:48] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Collaboration-Services, 07Upstream: zuul-client encrypt fails to retrieve public key - https://phabricator.wikimedia.org/T431003#12400310 (10hashar) 05Stalled→03Open >>! In T431003#12088758, @hashar wrote: > Another thing I found on Thursday eveni... [12:41:06] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Collaboration-Services, 07Upstream: zuul-client encrypt fails to retrieve public key - https://phabricator.wikimedia.org/T431003#12400333 (10hashar) 05Open→03Resolved a:03hashar **Verified** by: Repeatedly calling zuul-client which now sets... [12:47:51] 10GitLab, 06Release-Engineering-Team: kokkuri:setup-variables failing for repos/releng/release!268 (due to being a fork?) - https://phabricator.wikimedia.org/T436899#12400357 (10hashar) A recent example, though that is still related to patchdemo: https://gitlab.wikimedia.org/hashar/patchdemo/-/jobs/986165 I... [13:08:13] 10Phabricator (Upstream), 07Upstream: Calendar text overlap - https://phabricator.wikimedia.org/T159092#12400460 (10Aklapper) [13:38:16] 06Release-Engineering-Team, 10Ceph, 06Infrastructure-Foundations, 10SRE-swift-storage, 13Patch-For-Review: Test the Docker Registry's GC with Releng/MediaWiki images - https://phabricator.wikimedia.org/T437297#12400733 (10elukey) After ~130 mins the mark and sweep completed, and ~400GB of data has been d... [13:44:13] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Calendar text overlap - https://phabricator.wikimedia.org/T159092#12400775 (10Aklapper) 05Open→03Stalled a:03Aklapper [13:50:49] 10Phabricator, 10Phabricator (Upstream), 07Upstream: Phabricator needs to handle bounces/errors from non-existent email addresses - https://phabricator.wikimedia.org/T100400#12400819 (10Aklapper) 05Open→03Resolved Not seen for ages in the logs, optimistically resolving. [13:57:32] We will do some maintenance on GitLab CI. The cloud runners will be unavailable between 15:00 to 16:00 UTC [13:58:45] 10Phabricator (Upstream), 07Upstream: Email Preferences should inform of default values - https://phabricator.wikimedia.org/T182172#12400908 (10Aklapper) 05Stalled→03Declined Whether the default settings for notifications are too much or not enough notification noise feels rather subjective. In case y... [14:06:03] 10Scap, 06SRE Observability, 06MediaWiki-Core-Platform-Team (Radar): Scap's logstash checker for channels outside "error" and "exception" - https://phabricator.wikimedia.org/T438570#12401037 (10herron) For reference there are two logstash throttle thresholds in place today: 1) is a throttle_warning tag at 4... [14:13:25] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Potentially disable H489 - https://phabricator.wikimedia.org/T431735#12401098 (10Aklapper) No problem! Ah, even better. Yay! :) [14:34:59] 10Gerrit, 06Collaboration-Services, 07Incident Severity 1, 13Patch-For-Review, 07Sustainability (Incident Followup): Add the the ability to warm caches to Gerrit switchover cookbook - https://phabricator.wikimedia.org/T440177#12401312 (10ABran-WMF) 05Open→03In progress a:03ABran-WMF @hashar the 3 p... [14:38:18] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12401323 (10ssastry) Hello all, I wanted to flag this write up -- https://www.mediawiki.org/wiki/Media... [14:46:34] dduvall: I've been experimenting with cert rotation for long-lived magnum clusters. Here is what I've learned so far: https://wikitech.wikimedia.org/wiki/Help:Managed_Kubernetes#Certificate_rotation [14:46:41] We badly need some kind of monitoring and alerting for this but in the meantime you probably want to set calendar reminders for your clusters :/ [14:46:51] please let me know if that process turns out to only work for me! [14:49:11] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06Collaboration-Services: Deploy Phab/Phorge 2026-10-06 - https://phabricator.wikimedia.org/T440340 (10brennen) 03NEW [14:49:49] (03open) 10brennen: update submodules for 2026-10-06 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/121 (https://phabricator.wikimedia.org/T440340) [14:56:43] (03merge) 10brennen: update submodules for 2026-10-06 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/121 (https://phabricator.wikimedia.org/T440340) [14:58:07] 10Phabricator (2026-10-06), 06Release-Engineering-Team (Doing 😎): Add WEBP to files.viewable-mime-types - https://phabricator.wikimedia.org/T438712#12401520 (10Aklapper) webp: {F106656918} [15:01:34] !log Rebuilding gitlab-cloud-runners (prod) (T425441) [15:01:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:01:37] T425441: gitlab behind CDN - https://phabricator.wikimedia.org/T425441 [15:09:06] 10Phabricator (2026-10-06), 06Release-Engineering-Team (Doing 😎), 06Collaboration-Services: Deploy Phab/Phorge 2026-10-06 - https://phabricator.wikimedia.org/T440340#12401612 (10Aklapper) 05Open→03Resolved Changes supposed to be included in this deployment: * {T438712} / https://gitlab.wikimedia.org/... [15:17:29] andrewbogott: thanks I’ll have a look! [15:26:29] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12401724 (10Bugreporter) I believe using Produnto for templates is a bad idea: - Wiki can have much m... [15:30:33] 06Release-Engineering-Team (Doing 😎), 07Epic: [Epic] Migrate gitlab-cloud-runner to WMCS - https://phabricator.wikimedia.org/T416256#12401832 (10BLiviero-WMF) @Andrew can you comment here on whether your previous comment still holds? There is an ask on whether the team can move gitlab runners to WMF from Di... [15:42:57] (03CR) 10Santiago Faci: "I would say that, in general, looks good. And after testing this a bit locally, it works as expected. Just a couple of comments about some" [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1344123 (https://phabricator.wikimedia.org/T438283) (owner: 10Clare Ming) [15:47:44] (03merge) 10dancy: digitaocean: use new vpc egress gateway and isolated_workers in production [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/653 (https://phabricator.wikimedia.org/T425441) (owner: 10jelto) [15:48:19] !log gitlab-cloud-runners (prod) rebuild complete (T425441) [15:48:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:48:22] T425441: gitlab behind CDN - https://phabricator.wikimedia.org/T425441 [15:49:20] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12401951 (10Pigsonthewing) >>! In T440142#12401724, @Bugreporter wrote: > - A second issue is some t... [15:57:19] GitLab maintenance is done, CI is available again [16:05:52] 10GitLab (Project and group requests), 06Release-Engineering-Team: GitLab Private Repository Request for: pcendrer/plantilla-chatbot-patrones - https://phabricator.wikimedia.org/T440370 (10PCendrer) 03NEW [16:08:52] 10Phabricator (2026-10-06), 06Release-Engineering-Team (Doing 😎), 06Collaboration-Services, 07Essential-Work: Deploy Phab/Phorge 2026-10-06 - https://phabricator.wikimedia.org/T440340#12402194 (10brennen) [16:13:50] 10GitLab (Project and group requests), 06Release-Engineering-Team, 06App Growth, 07Essential-Work: Finding right group for microsite project in GitLab - https://phabricator.wikimedia.org/T439904#12402254 (10brennen) 05Open→03Stalled p:05Triage→03Low Pasting in discussion from WMF Slack: > @brennen... [16:14:49] 10GitLab (Project and group requests), 06Release-Engineering-Team, 06App Growth, 07Essential-Work: Finding right group for microsite project in GitLab - https://phabricator.wikimedia.org/T439904#12402284 (10Cklimas) Adding the `memory-optimized` tag to the CI jobs is resolving the OOM issues we had been se... [16:21:21] 10Scap, 06SRE Observability, 06MediaWiki-Core-Platform-Team (Radar): Scap's logstash checker for channels outside "error" and "exception" - https://phabricator.wikimedia.org/T438570#12402390 (10Tgr) >>! In T438570#12401037, @herron wrote: > Re: false alarms: having a look at throttle_warning tagged logs ove... [16:21:50] 10GitLab, 06Release-Engineering-Team: kokkuri:setup-variables failing for repos/releng/release!268 (due to being a fork?) - https://phabricator.wikimedia.org/T436899#12402397 (10dancy) This issue is documented here: https://support.gitlab.com/hc/en-us/articles/28598294286364-CI-ID-token-issuance-is-disabled-af... [16:28:03] 10Scap, 06SRE Observability, 06MediaWiki-Core-Platform-Team (Radar): Scap's logstash checker for channels outside "error" and "exception" - https://phabricator.wikimedia.org/T438570#12402435 (10Tgr) (Wrt the current high number of memcached errors, there is an investigation ongoing in {T420223}. For replag,... [16:39:52] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12402506 (10NguoiDungKhongDinhDanh) I for one dislike writing any kind of HTML in Lua. Lua, minimal a... [16:43:30] 10GitLab (Project and group requests), 06Release-Engineering-Team, 10Tool-plantilla-chatbot: GitLab Private Repository Request for: pcendrer/plantilla-chatbot-patrones - https://phabricator.wikimedia.org/T440370#12402586 (10brennen) a:03brennen Could you describe the plantilla-chatbot project? [16:50:45] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12402615 (10ssastry) > I for one dislike writing any kind of HTML in Lua. ... > In simple cases, wikit... [16:54:16] 06Release-Engineering-Team (Doing 😎), 07Epic: [Epic] Migrate gitlab-cloud-runner to WMCS - https://phabricator.wikimedia.org/T416256#12402648 (10Andrew) Regarding Magnum: out support is not fantastic but we are in much better shape than we were when I wrote that comment in April. As long as CI folks are fine d... [16:55:30] 10GitLab (Project and group requests), 06Release-Engineering-Team, 10Tool-plantilla-chatbot: GitLab Private Repository Request for: pcendrer/plantilla-chatbot-patrones - https://phabricator.wikimedia.org/T440370#12402664 (10PCendrer) plantilla-chatbot is a spanish wikipedia bot for fight against the AI in th... [16:56:29] 10GitLab (Project and group requests), 06Release-Engineering-Team, 10Tool-plantilla-chatbot: GitLab Private Repository Request for: pcendrer/plantilla-chatbot-patrones - https://phabricator.wikimedia.org/T440370#12402671 (10Aklapper) [16:59:52] 10GitLab (Project and group requests), 06Release-Engineering-Team, 10Tool-plantilla-chatbot, 07Essential-Work: GitLab Private Repository Request for: pcendrer/plantilla-chatbot-patrones - https://phabricator.wikimedia.org/T440370#12402693 (10brennen) 05Open→03Resolved Thanks - it sounds like this f... [17:04:27] (03PS1) 10Neriah: Zuul: [mediawiki/extensions/HAWelcome] Add optional dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/1352192 [17:31:09] 10GitLab (Project and group requests), 06Release-Engineering-Team, 06App Growth (AppG Sprint [FY2627 Sep 30 - Oct 13]), 07Essential-Work: Finding right group for microsite project in GitLab - https://phabricator.wikimedia.org/T439904#12402846 (10ABendall-WMF) [17:44:01] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: CI build slow and timing out sometimes - https://phabricator.wikimedia.org/T434024#12402905 (10Lars) Wow, thanks so much @Peter, that's a huge impr... [17:49:52] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: CI build slow and timing out sometimes - https://phabricator.wikimedia.org/T434024#12402935 (10Lars) @Peter or @pwangai For parallelizing these tes... [17:50:53] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, and 2 others: CI build slow and timing out sometimes - https://phabricator.wikimedia.org/T434024#12402941 (10Lars) A few notes for tests: VerifyEmployerFileTest is fixed. Omnigroupm... [18:21:00] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12403091 (10NguoiDungKhongDinhDanh) > I am not and I don't think anyone else is proposing that we shou... [18:24:05] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12403096 (10Doc_James) Sure we would be happy to meet and discuss. I was under the impression that the... [18:29:37] 10Beta-Cluster-Infrastructure, 06Data-Engineering, 10Kafka-Infrastructure: beta - Run a single event platform Kafka cluster in beta - https://phabricator.wikimedia.org/T437541#12403106 (10tchin) Peeking into the data folder, there's no new activity in any of the topic partitions since Sept 17th, with the onl... [18:40:05] FIRING: InstanceDown: Project deployment-prep instance deployment-kafka-jumbo-9 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [18:40:16] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-kafka-jumbo-9 is down - https://phabricator.wikimedia.org/T440387 (10wmcs-alerts) 03NEW [18:43:08] 10Beta-Cluster-Infrastructure, 06Data-Engineering, 10Kafka-Infrastructure: beta - Run a single event platform Kafka cluster in beta - https://phabricator.wikimedia.org/T437541#12403139 (10tchin) a:03tchin [18:43:17] 10Beta-Cluster-Infrastructure, 06Data-Engineering, 10Kafka-Infrastructure: beta - Run a single event platform Kafka cluster in beta - https://phabricator.wikimedia.org/T437541#12403140 (10tchin) Cleaned up `deployment-kafka-jumbo-5`, `deployment-kafka-jumbo-8`, `and deployment-kafka-jumbo-9` instances using... [18:45:05] RESOLVED: InstanceDown: Project deployment-prep instance deployment-kafka-jumbo-9 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [18:45:45] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12403145 (10ssastry) Right, Produnto use for templates is not under active consideration now, but ther... [18:47:03] FIRING: InstanceDown: Project deployment-prep instance deployment-kafka-jumbo-5 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [18:47:09] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-kafka-jumbo-5 is down - https://phabricator.wikimedia.org/T440388 (10wmcs-alerts) 03NEW [18:52:03] RESOLVED: [2x] InstanceDown: Project deployment-prep instance deployment-kafka-jumbo-5 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [18:53:43] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12403184 (10Doc_James) @ssastry Agree we want to avoid working at cross purposes. Yes lets work on sch... [19:44:50] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06Fundraising-Backlog: Add jenkins job to automerge +2ed patches on fundraising-dev repo - https://phabricator.wikimedia.org/T439854#12403381 (10SBurnett-WMF) [19:59:25] 10GitLab, 06Release-Engineering-Team: kokkuri:setup-variables failing for repos/releng/release!268 (due to being a fork?) - https://phabricator.wikimedia.org/T436899#12403423 (10hashar) That is great @dancy thank you!!!! I went to https://gitlab.wikimedia.org/-/profile/preferences On the sidebar {nav Access... [20:11:25] !log Disable duplicate publishing noise from extension-ElectronPdfService, rEEPS, T143162 [20:11:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:11:28] T143162: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162 [20:35:44] It seems the beta cluster requires 2FA for interface admins, like prod, as of last month, and a maintenance script was run to revoke this from existing users. [20:36:11] To enable 2FA requires re-auth which falls back to email. [20:36:25] It seems outgoing email doesn't work maybe? [20:36:50] https://beta-logs.wmcloud.org/goto/bd76bd529b4a504b7a08bf4cc76d115b [20:37:10] ` MediaWiki\Mail\UserMailer::send: sending mail to "Krinkle"` [20:37:12] `Sending mail via internal mail() function` [20:37:22] (this was with WikimediaDebug logging enabled) [20:40:05] s/a month/two months ago/ [21:44:33] 06Release-Engineering-Team, 10Catalyst (PatchDemo), 07Essential-Work: Clean up remaining Patch demo legacy wikis, remove config for /wikis - https://phabricator.wikimedia.org/T440405 (10brennen) 03NEW [21:45:44] 06Release-Engineering-Team, 10Catalyst (PatchDemo), 07Essential-Work: Clean up remaining Patch demo legacy wikis, remove config for /wikis - https://phabricator.wikimedia.org/T440405#12403914 (10brennen) [22:29:59] (03CR) 10Santiago Faci: Add Test Kitchen experiment overrides to the header and popup (031 comment) [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1344123 (https://phabricator.wikimedia.org/T438283) (owner: 10Clare Ming) [23:00:54] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12404177 (10Bugreporter) > In simple cases, wikitext is no doubt the preferred language over Lua. But... [23:12:45] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T438219#12404201 (10Aklapper) [23:22:27] 10Beta-Cluster-Infrastructure, 07Crosswiki, 10Wiki-Setup (Create): Create templates wiki to hold global templates and documentation during development and testing - https://phabricator.wikimedia.org/T440142#12404210 (10Bugreporter) >>! In T440142#12401951, @Pigsonthewing wrote: >>>! In T440142#12401724, @Bug...