[08:12:34] 10netops, 06Infrastructure-Foundations, 06SRE, 13Patch-For-Review: Core router error logs: "sshd: Did not receive identification string" from prometheus hosts - https://phabricator.wikimedia.org/T368513#10021674 (10ayounsi) [08:33:46] 06Traffic: Route FR to esams - https://phabricator.wikimedia.org/T371216 (10Fabfur) 03NEW [08:37:01] 06Traffic, 13Patch-For-Review: Route FR to esams - https://phabricator.wikimedia.org/T371216#10021785 (10ayounsi) [08:39:22] 06Traffic, 13Patch-For-Review: Route FR to esams - https://phabricator.wikimedia.org/T371216#10021800 (10ayounsi) [09:02:51] 06Traffic, 13Patch-For-Review: Route FR to esams - https://phabricator.wikimedia.org/T371216#10021894 (10Volans) If deemed useful those are the [[ https://phabricator.wikimedia.org/P66993 | subdivisions in maxmind db for France ]] (nda-only) and the related docs for gdnsd are here (assuming up to date ;)): ht... [10:17:58] 06Traffic, 10conftool: Allow integrating requestctl rules into haproxy - https://phabricator.wikimedia.org/T369606#10022273 (10Joe) [10:33:15] 06Traffic, 10conftool: support the haproxy silent-drop hysteresis gadget from requestctl - https://phabricator.wikimedia.org/T371144#10022386 (10Joe) Before I think of implementations, I'd like to understand better what you want to offer: * A way to limit concurrency of any request corresponding to a specific... [10:46:57] 06Traffic, 10conftool: support the haproxy silent-drop hysteresis gadget from requestctl - https://phabricator.wikimedia.org/T371144#10022454 (10Joe) To remove even more doubt, can you make an example of what you'd expect in haproxy dsl with the following rule: ` expression: ipblock@cloud/aws AND pattern@ua/r... [10:47:28] 06Traffic, 10MW-on-K8s, 06serviceops, 06SRE, and 2 others: Spin down api_appserver and appserver clusters - https://phabricator.wikimedia.org/T367949#10022456 (10Volans) [11:04:52] 06Traffic, 10MW-on-K8s, 06serviceops, 06SRE, and 2 others: Spin down api_appserver and appserver clusters - https://phabricator.wikimedia.org/T367949#10022490 (10Clement_Goubert) [13:24:56] 06Traffic: Route FR to esams - https://phabricator.wikimedia.org/T371216#10022883 (10Fabfur) 05Open→03In progress p:05Triage→03Low [13:24:57] 06Traffic: Route FR to esams - https://phabricator.wikimedia.org/T371216#10022885 (10Volans) FYI 86% of maxmind's french prefixes have the subdivisions information. Then how much they are accurate both in terms of geolocation and as a consequence routing for this specific case is all to be seen. Some example co... [13:25:06] 06Traffic: Route FR to esams - https://phabricator.wikimedia.org/T371216#10022886 (10Fabfur) Leaving this open as reminder to revert the geo-maps modification [13:44:34] 10netops, 06Infrastructure-Foundations, 06SRE: Netbox automation to move selected hosts from ASW to LSW - https://phabricator.wikimedia.org/T370846#10022944 (10cmooney) [14:28:55] 10netops, 06Traffic, 06Infrastructure-Foundations, 06SRE: Upgrade anycast-healthchecker to 0.9.8 (from 0.9.1-1+wmf12u1) - https://phabricator.wikimedia.org/T370068#10023237 (10cmooney) p:05Triage→03Medium [14:30:28] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, and 3 others: lvs2013: move uplink to lsw1-c2-codfw and connect to per-rack vlan - https://phabricator.wikimedia.org/T370927#10023247 (10cmooney) p:05Triage→03Medium [14:48:43] 10Acme-chief, 06Traffic, 06Infrastructure-Foundations, 10Puppet-Infrastructure, and 2 others: Revert back to fleet-wide acmechief config once all ACME consumers are on Puppet 7 - https://phabricator.wikimedia.org/T365799#10023337 (10SLyngshede-WMF) a:03SLyngshede-WMF [14:56:00] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Request additional mgmt IP range for frack servers - https://phabricator.wikimedia.org/T370164#10023429 (10cmooney) 05Open→03Resolved Gonna close this one, I see hosts have been assigned to the new range and are reachable ` cmo... [15:27:08] 06Traffic, 10MW-on-K8s, 06serviceops, 06SRE, and 2 others: Spin down api_appserver and appserver clusters - https://phabricator.wikimedia.org/T367949#10023539 (10Clement_Goubert) 05In progress→03Resolved [16:19:52] 06Traffic, 13Patch-For-Review: Package and deploy ATS 9.2.5 - https://phabricator.wikimedia.org/T339134#10023813 (10ssingh) We are missing the `debian/patches/0004-logging-broken-pipe-no-spam.patch` in the 9.2.5 build so we will need to do another build. ` Jul 29 16:16:46 cp4052 trafficserver[2483817]: [Jul 2... [16:33:10] 06Traffic, 10conftool: support the haproxy silent-drop hysteresis gadget from requestctl - https://phabricator.wikimedia.org/T371144#10023879 (10CDanis) > A way to limit concurrency from specific IPs, only for requests corresponding to a specific pattern? So limit concurrency for a user over the threshold only... [17:14:06] 06Traffic, 10MW-on-K8s, 06serviceops, 06SRE, 10Release-Engineering-Team (Seen): Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536#10024160 (10jijiki) [17:45:49] 06Traffic, 13Patch-For-Review: Package and deploy ATS 9.2.5 - https://phabricator.wikimedia.org/T339134#10024344 (10ssingh) `cp4052` is running `ATS 9.2.5` with our build of `9.2.5-1wm2` as of `Mon 2024-07-29 17:17:37 UTC`. [18:05:38] 06Traffic, 06Data-Persistence, 06SRE, 10SRE-swift-storage, and 6 others: Change default image thumbnail size - https://phabricator.wikimedia.org/T355914#10024486 (10NBaca-WMF) Created three related tickets to track work for this: * https://phabricator.wikimedia.org/T371295 for running synthetic performanc... [18:51:11] 06Traffic: Adding IP Addresses to SPF (Dayforce) - https://phabricator.wikimedia.org/T371304#10024762 (10ssingh)