[10:40:56] 06serviceops, 10Prod-Kubernetes, 07Kubernetes: charlie wiped cluster redeployment use-case - https://phabricator.wikimedia.org/T406212#11245659 (10Clement_Goubert) >>! In T406212#11238224, @RLazarus wrote: > What's the best query to check the cluster is wiped? One option is just a command-line flag like `--i... [13:30:31] 06serviceops, 06MediaWiki-Platform-Team, 07OKR-Work: Test api rate limiting on staging cluster - https://phabricator.wikimedia.org/T406490 (10daniel) 03NEW [13:31:22] 06serviceops, 06MediaWiki-Platform-Team, 07OKR-Work: Test api rate limiting on staging cluster - https://phabricator.wikimedia.org/T406490#11246074 (10daniel) [14:44:05] 06serviceops, 06MediaWiki-Platform-Team, 07OKR-Work: Test api rate limiting on staging cluster - https://phabricator.wikimedia.org/T406490#11246394 (10daniel) [14:51:17] 06serviceops, 06MediaWiki-Platform-Team, 07OKR-Work: Test api rate limiting on staging cluster - https://phabricator.wikimedia.org/T406490#11246443 (10daniel) [14:53:17] 06serviceops, 06MediaWiki-Platform-Team, 07OKR-Work: Test api rate limiting on production cluster - https://phabricator.wikimedia.org/T406498 (10daniel) 03NEW [14:53:47] 06serviceops, 06MediaWiki-Platform-Team, 07OKR-Work: Test api rate limiting on production cluster - https://phabricator.wikimedia.org/T406498#11246464 (10daniel) [15:21:43] 06serviceops, 06MediaWiki-Platform-Team, 07OKR-Work: Test api rate limiting on production cluster - https://phabricator.wikimedia.org/T406498#11246577 (10daniel) Candiate routes for enabling rate limiting... Very low traffic: * /api/rest_v1/page/pdf/ (< 1 req/sec) * /api/rest_v1/page/talk/ (< 1 req/sec) * /... [16:22:34] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE: Q1:rack/setup/install mc-misc200[12] - https://phabricator.wikimedia.org/T372800#11246868 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jhancock@cumin1002 for host mc-misc2001.codfw.wmnet with OS bookworm [16:55:50] 06serviceops, 10Prod-Kubernetes, 06Traffic, 07Kubernetes, 13Patch-For-Review: Handling inbound IPIP traffic on low traffic LVS k8s based realservers - https://phabricator.wikimedia.org/T352956#11247042 (10ssingh) Hi @akosiaris: Following up on this after a discussion during Traffic's planning with @Vguti... [17:29:46] 06serviceops, 10decommission-hardware, 13Patch-For-Review: decommission mw135[8-9], mw136[4-6], mw137[2-3], mw140[0-4], mw1406, mw14[11-13] - https://phabricator.wikimedia.org/T383227#11247144 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=270ad3ea-fd56-4b48-9229-f9ae3073d358) set by jas... [18:29:46] 06serviceops, 06MW-Interfaces-Team (MWI-Sprint-19 (2025-09-23 to 2025-10-07)), 07OKR-Work: Execute test plan for rest gateway rerouting for rest.php requests and report findings - https://phabricator.wikimedia.org/T405368#11247310 (10aaron) p:05Triage→03High [18:30:52] 06serviceops, 06MW-Interfaces-Team (MWI-Sprint-19 (2025-09-23 to 2025-10-07)), 07OKR-Work: Execute test plan for rest gateway rerouting for rest.php requests and report findings - https://phabricator.wikimedia.org/T405368#11247313 (10aaron) 05Open→03Resolved Everything passes and I don't see any rema... [20:04:11] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE: Q1:rack/setup/install mc-misc200[12] - https://phabricator.wikimedia.org/T372800#11247671 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jhancock@cumin1002 for host mc-misc2001.codfw.wmnet with OS bookworm [20:40:21] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE: Q1:rack/setup/install mc-misc200[12] - https://phabricator.wikimedia.org/T372800#11247785 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jhancock@cumin1002 for host mc-misc2001.codfw.wmnet with OS bookworm completed: - mc-misc2001 (**P... [20:51:42] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE, 13Patch-For-Review: Q1:rack/setup/install wikikube-ctrl2006 - https://phabricator.wikimedia.org/T400661#11247796 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jhancock@cumin1002 for host wikikube-ctrl2006.codfw.wmnet with OS bookworm ex... [21:28:31] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE, 13Patch-For-Review: Q1:rack/setup/install wikikube-ctrl2006 - https://phabricator.wikimedia.org/T400661#11247892 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jhancock@cumin1002 for host wikikube-ctrl2006.codfw.wmnet with OS bookworm [21:53:32] 06serviceops, 10GitLab (CI & Job Runners): failed to push docker-registry.discovery.wmnet/repos/data-engineering/airflow-dags:airflow-2.10.5-py3.11-2025-10-03-192132-3003d4328df66a0086a350fdd2ba1dbd80a235c5: unknown: blob upload invalid - https://phabricator.wikimedia.org/T406392#11247933 (10Scott_French) @ama... [22:00:10] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE, 13Patch-For-Review: Q1:rack/setup/install wikikube-ctrl2006 - https://phabricator.wikimedia.org/T400661#11247946 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jhancock@cumin1002 for host wikikube-ctrl2006.codfw.wmnet with OS bookworm ex... [23:04:02] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE, 13Patch-For-Review: Q1:rack/setup/install wikikube-ctrl2006 - https://phabricator.wikimedia.org/T400661#11248078 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jhancock@cumin1002 for host wikikube-ctrl2006.codfw.wmnet with OS bookworm