[03:32:15] <komla>	 Can we have a +2 on this quota request: T425905
[03:32:16] <stashbot>	 T425905: Quota increase request for project language - https://phabricator.wikimedia.org/T425905
[03:34:08] <komla>	 We also have 2 more quota requests pending
[06:44:12] <godog>	 greetings
[07:59:25] <godog>	 I'll kick off the work in T420565
[07:59:26] <stashbot>	 T420565: Audit tools memory requests vs actual usage - https://phabricator.wikimedia.org/T420565
[08:11:51] <godog>	 ok change merged, bumping version and deploying to toolsbeta
[08:24:02] <godog>	 ok new webservice-cli is on toolsbeta now, I have verified that e.g. restarting fourohfour does the right thing
[08:24:17] <godog>	 -      memory:  256Mi
[08:24:17] <godog>	 +      memory:  128Mi
[08:29:32] <godog>	 I'm test-running the roll-restart script on toolsbeta
[08:35:31] <godog>	 ok deploying new webservice-cli to tools
[08:45:59] <godog>	 will begin with restarting a sample of 10 tools https://phabricator.wikimedia.org/P92497
[09:05:56] <godog>	 I'm not seeing any disaster ensuing
[09:07:42] <godog>	 I'll be resuming restarting webservices using default requests later today
[14:04:50] <andrewbogott>	 dhinus, komla: horizontal scaling works for wiki-tts. So now they have a different, more possible quota request.  https://phabricator.wikimedia.org/T425804
[14:17:19] <dhinus>	 andrewbogott: nice! it was worth asking in the end :)
[14:18:00] <andrewbogott>	 yeah!
[18:11:31] <Raymond_Ndibe>	 Not sure what is wrong. https://test.beta.toolforge.org/healthz is unreachable, but a webservice is currently running on the test tool.
[18:58:32] <andrewbogott>	 Reedy: all traffic is failing isn't it?  I'm going to try restarting the proxy
[19:00:41] <andrewbogott>	 um... sorry Reedy, mistype, disregard
[19:00:51] <Reedy>	 heh
[19:00:53] <andrewbogott>	 The toolsbeta haproxies say "Server k8s-ingress-http/toolsbeta-test-k8s-gateway-2.toolsbeta.eqiad1.wikimedia.cloud"
[20:56:58] <andrewbogott>	 LOL I did not paste the useful part of that log message!  It is "Server k8s-ingress-http/toolsbeta-test-k8s-gateway-[12].toolsbeta.eqiad1.wikimedia.cloud is DOWN, reason: Layer4 timeout"