[07:41:24] kubestagemaster2004 is running on a Ganeti server which is going to be decommed, to move it to the new node, it'll briefly be run on a DRBD backend, so if any alert kicks in for latency or similar, just ignore it and it will resolve 30 mins later [07:47:51] ack [08:02:14] move complete and back on "plain" storage [08:33:19] thanks! [08:40:41] 16 [08:40:42] err [09:38:55] 06serviceops, 10MW-on-K8s, 07Datacenter-Switchover: Control mw-on-k8s periodic maintenance jobs with an etcd value - https://phabricator.wikimedia.org/T367118#10261932 (10Clement_Goubert) Since the spawned Jobs are not part of the release, they won't be stopped, so that would need to be added as well. [09:56:51] 06serviceops, 10Deployments, 06Release-Engineering-Team: sync-testservers-k8s takes 4 minutes when deploying a mediawiki-config change - https://phabricator.wikimedia.org/T374907#10261994 (10Clement_Goubert) For the couple of pods, and taking into account @akosiaris' findings above (supported by a quick tha... [10:05:30] 06serviceops, 10Deployments, 06Release-Engineering-Team, 13Patch-For-Review: sync-testservers-k8s takes 4 minutes when deploying a mediawiki-config change - https://phabricator.wikimedia.org/T374907#10262033 (10akosiaris) Just look at the latest data, indeed no discernible gains from the change and it's ca... [10:12:43] 06serviceops, 10Deployments, 06Release-Engineering-Team, 13Patch-For-Review: sync-testservers-k8s takes 4 minutes when deploying a mediawiki-config change - https://phabricator.wikimedia.org/T374907#10262040 (10akosiaris) 05Open→03Resolved @hashar, I think we 've exhausted most of the possible opti... [10:16:40] 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Allow running one-off scripts manually - https://phabricator.wikimedia.org/T341553#10262044 (10Clement_Goubert) >>! In T341553#10260109, @RLazarus wrote: > That's not to say "your fault, should have gone and read the wiki!" but rather -- how can I do better at p... [10:58:49] 06serviceops, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10262138 (10JMeybohm) [11:13:54] 06serviceops, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10262157 (10Clement_Goubert) The vlan migration for eqiad should happen this FY, but the hardware is still in procurement {T368959} [13:27:27] 06serviceops, 06Data-Engineering, 10Event-Platform: Traffic for eventstreams-internal seems to be zero for the past months - https://phabricator.wikimedia.org/T348763#10262406 (10Ottomata) 05Declined→03Open [13:28:53] 06serviceops, 06Data-Engineering, 10Event-Platform: Traffic for eventstreams-internal seems to be zero for the past months - https://phabricator.wikimedia.org/T348763#10262402 (10Ottomata) 05Open→03Declined @BTullis do you think it would be possible to add authentication and a public domain to this servi... [13:45:41] 06serviceops, 06Data-Engineering, 10Event-Platform: Make eventstreams-internal available to WMF staff without an ssh tunnel - https://phabricator.wikimedia.org/T348763#10262504 (10Ottomata) [13:52:52] 06serviceops, 06Data-Engineering, 10Event-Platform: Make eventstreams-internal available to WMF staff without an ssh tunnel - https://phabricator.wikimedia.org/T348763#10262533 (10BTullis) >>! In T348763#10262402, @Ottomata wrote: > @BTullis do you think it would be possible to add authentication and a publi... [14:38:11] 06serviceops, 06Data-Engineering, 10Event-Platform: Make eventstreams-internal available to WMF staff without an ssh tunnel - https://phabricator.wikimedia.org/T348763#10262664 (10CDanis) >>! In T348763#10262531, @BTullis wrote: > * add an authenticating reverse proxy using [[https://gateway.envoyproxy.io/la... [15:27:41] 06serviceops, 10MW-on-K8s, 10Scap, 10Continuous-Integration-Config, and 3 others: Define variant Wikimedia production config in compiled, static files - https://phabricator.wikimedia.org/T223602#10262833 (10jeremyb) @awight @Jdforrester-WMF is this still stalled? where is the best place to track how much i... [15:46:54] 06serviceops, 10MW-on-K8s, 07Datacenter-Switchover, 13Patch-For-Review: Control mw-on-k8s periodic maintenance jobs with an etcd value - https://phabricator.wikimedia.org/T367118#10262879 (10Scott_French) If we are indeed introducing an etcd-based "maintenance enabled" state and wiring that into the confd... [15:55:54] 06serviceops, 06Data-Engineering, 10Event-Platform: Make eventstreams-internal available to WMF staff without an ssh tunnel - https://phabricator.wikimedia.org/T348763#10262910 (10BTullis) >>! In T348763#10262664, @CDanis wrote: > But we're using [[ https://github.com/oauth2-proxy/oauth2-proxy | oauth2-proxy... [17:01:43] 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Allow running one-off scripts manually - https://phabricator.wikimedia.org/T341553#10263088 (10RLazarus) >>! In T341553#10260574, @bd808 wrote: >[...] needing to remember to setup the correct credentials with `kube_env mw-script-deploy codfw` before being able t... [17:11:23] 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Allow running one-off scripts manually - https://phabricator.wikimedia.org/T341553#10263153 (10RLazarus) >>! In T341553#10262044, @Clement_Goubert wrote: > Suggestion: Catch ctrl-c and print out something like > ` > ⚠️ It looks like you may have wanted to stop... [18:20:33] 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Allow running one-off scripts manually - https://phabricator.wikimedia.org/T341553#10263470 (10Urbanecm_WMF) >>! In T341553#10260109, @RLazarus wrote: > Sorry this happened. Unfortunately it's kind of working as intended -- not because it's supposed to be hard t... [18:32:47] 06serviceops, 06MW-Interfaces-Team, 10RESTBase Sunsetting, 13Patch-For-Review: Switchover plan from RESTbase to REST Gateway for rest_v1/page/html and rest_v1/page/title endpoints - https://phabricator.wikimedia.org/T374683#10263499 (10BPirkle) The fix for the items wrapper has ridden the train and is live...