[00:53:24] <wikibugs>	 06serviceops, 10Cassandra, 06Growth-Team, 13Patch-For-Review: mediawiki: migrate from image-suggestion to data-gateway - https://phabricator.wikimedia.org/T368096#11084390 (10Scott_French)
[00:57:42] <wikibugs>	 06serviceops, 10Cassandra, 06Growth-Team, 13Patch-For-Review: mediawiki: migrate from image-suggestion to data-gateway - https://phabricator.wikimedia.org/T368096#11084391 (10Eevans)
[01:06:48] <wikibugs>	 06serviceops, 10Cassandra, 06Growth-Team, 13Patch-For-Review: mediawiki: migrate from image-suggestion to data-gateway - https://phabricator.wikimedia.org/T368096#11084402 (10Eevans) The 500s are indeed the result of query read timeouts at the coordinator nodes, and for the queries in question, they all re...
[01:17:55] <wikibugs>	 06serviceops, 10Cassandra, 06Growth-Team, 13Patch-For-Review: mediawiki: migrate from image-suggestion to data-gateway - https://phabricator.wikimedia.org/T368096#11084428 (10Scott_French)
[08:31:58] <wikibugs>	 06serviceops, 10MediaWiki-extensions-WikimediaMaintenance, 10MW-on-K8s: mwscript-k8s does not include an environment variable with the username of the executing user - https://phabricator.wikimedia.org/T401803#11084906 (10akosiaris) Re-reading my first response, I realize I might have been a bit unclear. Ind...
[08:45:05] <wikibugs>	 06serviceops, 13Patch-For-Review: docker-registry.wikimedia.org keeps serving bad blobs - https://phabricator.wikimedia.org/T390251#11084940 (10akosiaris) >>! In T390251#11084257, @thcipriani wrote: > @akosiaris looks like you had a lot of changes for making a new registry, is that registry ready? Or is it sti...
[09:10:12] <wikibugs>	 06serviceops, 10MW-on-K8s: Running maintenance scripts in screen on `deploy1003` appears to fail, but is still running - https://phabricator.wikimedia.org/T400962#11085029 (10Clement_Goubert) I think this happens because the logs follow is a kubelet-backed read, not a kube-apiserver-backed one, and as such it...
[09:10:43] <wikibugs>	 06serviceops, 10MW-on-K8s: Running maintenance scripts in screen on `deploy1003` appears to fail, but is still running - https://phabricator.wikimedia.org/T400962#11085030 (10Clement_Goubert) 05Open→03Resolved a:03Clement_Goubert
[09:44:06] <wikibugs>	 06serviceops, 06Abstract Wikipedia team, 13Patch-For-Review: Cannot deploy function-orchestrator in staging environment due to insufficient quotas - https://phabricator.wikimedia.org/T401833#11085165 (10Clement_Goubert) >>! In T401833#11083397, @Jdforrester-WMF wrote: > Unfortunately the new 20GiB limit does...
[11:22:51] <wikibugs>	 06serviceops, 06Abstract Wikipedia team: Cannot deploy function-orchestrator in staging environment due to insufficient quotas - https://phabricator.wikimedia.org/T401833#11085891 (10akosiaris) Thanks @claime. Stupid typo on my side.
[13:48:07] <wikibugs>	 06serviceops, 10Prod-Kubernetes, 06Traffic, 07Kubernetes, 13Patch-For-Review: Handling inbound IPIP traffic on low traffic LVS k8s based realservers - https://phabricator.wikimedia.org/T352956#11086307 (10akosiaris) >>! In T352956#11083016, @Vgutierrez wrote: >>>! In T352956#11016142, @akosiaris wrote: >...
[14:02:27] <wikibugs>	 06serviceops, 10Prod-Kubernetes, 06Traffic, 07Kubernetes, 13Patch-For-Review: Handling inbound IPIP traffic on low traffic LVS k8s based realservers - https://phabricator.wikimedia.org/T352956#11086373 (10Vgutierrez) you got that available as part of the sre.loadbalancer.migrate-service-ipip cookbook on...