[11:25:34] 06serviceops, 06DC-Ops, 10decommission-hardware, 10ops-eqiad, and 2 others: decommission mw135[8-9], mw136[4-6], mw137[2-3], mw140[0-4], mw1406, mw14[11-13] - https://phabricator.wikimedia.org/T383227#11254329 (10Jclark-ctr) [11:31:38] 06serviceops, 06DC-Ops, 10decommission-hardware, 10ops-eqiad, and 2 others: decommission mw135[8-9], mw136[4-6], mw137[2-3], mw140[0-4], mw1406, mw14[11-13] - https://phabricator.wikimedia.org/T383227#11254364 (10Jclark-ctr) 05Open→03Resolved [11:32:24] 06serviceops, 06DC-Ops, 10decommission-hardware, 10ops-eqiad, and 2 others: decommission mw135[8-9], mw136[4-6], mw137[2-3], mw140[0-4], mw1406, mw14[11-13] - https://phabricator.wikimedia.org/T383227#11254367 (10Jclark-ctr) [11:35:03] 06serviceops, 14Growth Team Maintenance, 06Growth-Team, 10GrowthExperiments-Mentorship, 07Wikimedia-production-error: growthexperiments-updatementeedata-s1 errors out on DB timeout - https://phabricator.wikimedia.org/T395299#11254387 (10DMburugu) [11:50:36] 06serviceops, 07Epic, 06MW-Interfaces-Team (MWI-Roadmap), 07OKR-Work, 13Patch-For-Review: rest.php via rest-gateway production rollout - https://phabricator.wikimedia.org/T406318#11254657 (10Clement_Goubert) 05Open→03In progress p:05Triage→03High [12:09:00] 06serviceops, 07Epic, 06MW-Interfaces-Team (MWI-Roadmap), 07OKR-Work, 13Patch-For-Review: rest.php via rest-gateway production rollout - https://phabricator.wikimedia.org/T406318#11254769 (10Clement_Goubert) group0 is now routed at 10% to the `rest-gateway` [12:33:36] 06serviceops, 07Epic, 06MW-Interfaces-Team (MWI-Roadmap), 07OKR-Work, 13Patch-For-Review: rest.php via rest-gateway production rollout - https://phabricator.wikimedia.org/T406318#11254867 (10Clement_Goubert) [12:36:36] 06serviceops, 07Epic, 06MW-Interfaces-Team (MWI-Roadmap), 07OKR-Work, 13Patch-For-Review: rest.php via rest-gateway production rollout - https://phabricator.wikimedia.org/T406318#11254902 (10Clement_Goubert) Note that we rolled `test2wiki` and `testwikidata` back into the normal rollout. - `test2wiki` is... [12:47:25] 06serviceops, 07Epic, 06MW-Interfaces-Team (MWI-Roadmap), 07OKR-Work, 13Patch-For-Review: rest.php via rest-gateway production rollout - https://phabricator.wikimedia.org/T406318#11254953 (10Clement_Goubert) [15:19:18] 06serviceops, 06SRE: eqiad (2) memcached host for wikifunctions service implementation tracking - https://phabricator.wikimedia.org/T313965#11255650 (10kamila) 05Open→03Resolved a:03kamila Looks done to me, correct me if I'm wrong :-) [15:20:30] 06serviceops, 07Epic, 06MW-Interfaces-Team (MWI-Roadmap), 07OKR-Work, 13Patch-For-Review: rest.php via rest-gateway production rollout - https://phabricator.wikimedia.org/T406318#11255659 (10Clement_Goubert) [15:26:16] 06serviceops, 13Patch-For-Review: MediaWiki on PHP 8.3 production workload migration - https://phabricator.wikimedia.org/T405955#11255687 (10Scott_French) [15:41:29] 06serviceops, 10Prod-Kubernetes, 07Kubernetes: charlie wiped cluster redeployment use-case - https://phabricator.wikimedia.org/T406212#11255722 (10RLazarus) a:03RLazarus [15:48:35] FYI folks I pushed the outstanding Homer change to the CRs in eqiad to remove the BGP for wikikube-ctrl1001 [16:09:32] 06serviceops, 07Datacenter-Switchover: Remove references to mw-maintenance hosts in switchdc - https://phabricator.wikimedia.org/T404538#11255916 (10jasmine_) {{done}} References to the maintenance hosts have been removed from the switchdc cookbooks. Separately noted: We will follow this up by removing refer... [16:09:57] 06serviceops, 07Datacenter-Switchover: Remove references to mw-maintenance hosts in switchdc - https://phabricator.wikimedia.org/T404538#11255917 (10jasmine_) 05Open→03Resolved [16:27:02] 06serviceops: wikikube-ctrl200[4-5] implementation tracking - https://phabricator.wikimedia.org/T390861#11255990 (10kamila) [17:58:55] 06serviceops, 13Patch-For-Review: MediaWiki on PHP 8.3 production workload migration - https://phabricator.wikimedia.org/T405955#11256410 (10Scott_French) [18:58:49] 06serviceops, 10GitLab (CI & Job Runners): failed to push docker-registry.discovery.wmnet/repos/data-engineering/airflow-dags:airflow-2.10.5-py3.11-2025-10-03-192132-3003d4328df66a0086a350fdd2ba1dbd80a235c5: unknown: blob upload invalid - https://phabricator.wikimedia.org/T406392#11256660 (10dancy) >>! In T406... [21:14:42] 06serviceops: `sextant update` doesn't like growthbook - https://phabricator.wikimedia.org/T406786#11257072 (10RLazarus) [21:27:46] 06serviceops: `sextant update` doesn't like growthbook - https://phabricator.wikimedia.org/T406786#11257128 (10RLazarus) This dates from https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1056137, where growthbook used to be one chart but was split into a frontend and a backend. That commit copied `... [21:38:38] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#11257166 (10Krinkle) [21:40:45] 06serviceops: `sextant update` doesn't like growthbook - https://phabricator.wikimedia.org/T406786#11257171 (10RLazarus) Oops, I see @brouberol is OOO for a bit. @RKemper can I talk you into taking a look? [21:41:07] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#11257172 (10Krinkle) [21:41:10] 06serviceops, 06Data-Platform-SRE: `sextant update` doesn't like growthbook - https://phabricator.wikimedia.org/T406786#11257175 (10RLazarus) [21:41:40] 06serviceops, 06Data-Platform-SRE: `sextant update` doesn't like growthbook - https://phabricator.wikimedia.org/T406786#11257176 (10RLazarus)