[14:46:44] hi all. I heard there was some discussion (but not written down anywhere?) about https://phabricator.wikimedia.org/T344171 while I was out ? [14:47:30] cdanis: yes there was. In the SIG. Let me fetch you the meeting notes [14:47:34] ahh! [14:47:53] https://www.mediawiki.org/wiki/Kubernetes_SIG/Meetings/2024-08-27 [14:47:56] yes thanks [14:48:01] just got there myself :) [14:48:44] it also lacks an action point for me to look into the calico part for CoreDNS [14:49:04] neither the outside k8s, nor the NodePort approach is one that thrilled anyone in the group [14:49:08] yeah [14:49:13] is this essentially Calico doing a MetalLB imitation? [14:49:40] no, more like having a couple of pods that are always on the same IP [14:49:46] and can be addressed directly [14:49:56] instead of going through the rest of the kubernetes platform [14:50:07] and just delegate from gdnsd to those [14:50:13] got it [14:50:22] whether it will work or not, we 'll see. [14:51:42] so this is using calico as IPAM instead of using calico as BGP advertisements for a clusterIP or something [14:53:00] kinda. It's still IPAM and BGP advertisements in any case. [14:53:15] but what it is is bypassing overall Kubernetes Service resources [14:53:17] right [14:53:24] so no ClusterIP, no NodePort, no DNATs etc [14:53:38] btw you can avoid that even when advertising ClusterIPs [14:53:56] on both calico and metalLB there's a mode where only the pods that have the service in question will advertise the /32 [14:54:36] ? [14:55:03] externalTrafficPolicy: Local https://docs.tigera.io/calico/latest/networking/configuring/advertise-service-ips#advertising-service-ips-quick-glance [14:55:24] not avoid having a Service ofc, but avoid touching too may nodes inside the cluster [14:55:30] or doing a lot of NAT [14:57:26] ah, that setting. That requires that we cut off somehow a section of the cluster (labels?) and delegate to those nodes specifically and make sure we always run 1 or more coredns pods on those nodes. [14:57:40] yeah [14:57:47] we are using already the flip side of it (internalTrafficPolicy) for mcrouter [14:57:52] got it [14:58:03] which immediately requires a daemonset [14:58:12] I think there's a decent argument for a coredns daemonset on the apiservers anyway, but meh :) [14:58:16] but running 200 coredns pods... meh [14:59:17] that being said, delegation is almost certainly the best way to go [14:59:26] yeah [14:59:30] we did discuss also messing with powerdns (the recursors) [14:59:52] but somehow making them aware of this part of the infrastructure, didn't sound very appealing at the end [14:59:59] sure [15:02:48] thanks for catching me upo [15:16:03] yw