[07:08:41] just a heads up, I'm getting rid of wikibase TLS cert & config on the text cluster [07:09:09] otherwise we will have some issues in a few days because we cannot get new TLS certificates for that domain [08:03:53] <_joe_> akosiaris: I'm trying to create a TLS termination envoy image to use on k8s, and I was asking myself what's the best model for that in terms of configuration [08:04:47] for populating the yaml joy that is envoy config? [08:05:04] <_joe_> My idea was we just create /etc/envoy/{clusters.d,listeners.d} as we do in prod, adding the generic tls termination in the image and letting people inject the additional piece of config they might want [08:05:34] <_joe_> so in k8s you just add a configmap with what you want to add [08:06:15] <_joe_> I don't see great alternatives, unless we want to keep a copy of the run-of-the-mill envoy config in *every* chart [08:06:44] hm.. not sure. I 've seen envoy config like once up to now [08:06:51] I 'll have to catch up with the details [08:07:14] <_joe_> akosiaris: the big issue is envoy expects a single yaml with all the config [08:07:25] <_joe_> ofc that's a relative problem if you use xDS [08:07:38] <_joe_> but we're not for now [08:10:01] not even sure what that is :P [08:12:14] btw, a not very involved way of getting envoy working for the k8s services, would be to install it on the k8s hosts with the corresponding config. Not scalable at all and it would require us to go around and mess with the config, plus 2 ports for everything as well [08:12:31] akosiaris: https://gerrit.wikimedia.org/r/c/operations/puppet/+/537325 [08:13:19] but it would be more or less the same approach as for appservers [08:13:33] jynus: so, the RAID beats the local disks in obtain sda? [08:13:51] the funny thing is that I bet that if I create 2 virtual disks [08:14:03] is it always ? or is there some beautiful race condition and if you do it often enough it will make things even more interesting? [08:14:04] it will be sda and sdb [08:14:22] s/often enough/enough times/ [08:14:34] which basically make those recipes useless [08:14:38] text is wikibase free and no new t-shirts have been gifted :( [08:14:44] but I just want to install it, I can remove it later [08:15:07] akosiaris: 2 out of 2 installs at least [08:15:19] but seems conssistent with our experience with dbs [08:15:37] * akosiaris waits for this to be merged and the machine deciding to invert the order again [08:15:43] well [08:15:48] better test it now [08:15:49] anyway, fine by me if it helps move forward [08:16:10] I would suggest to disable automatic reimaging for the backup hosts after installed [08:16:15] like we do for dbs [08:17:18] (our) partman, as faidon has notices many times has many issues, but that is not a battle I want to fight today [08:18:39] also, for more fun, what would it be able to boot after sdb or sdc fail? [08:18:47] *s/what// [08:19:10] depends on the BIOS config I guess ? [08:21:29] <_joe_> akosiaris: yeah my idea is that at first we will add the TLS termination to individual services [09:11:10] So I am not going to research this, but it is possible we are installing cross-dc? [09:11:21] installing oses [09:11:38] I reimaged only after running puppet on install2002 [09:11:48] and it didn't update [09:12:05] but later it did (assuming when puppet run on install1002) [09:12:28] or maybe install2002 is only a backup [09:12:33] ? [09:13:58] both are sending out the DHCP announcement and then the server picks the first response (and thus server) it pick, I have seen eqiad hosts using install2002, 1002 doesn't always "win" by faster response [09:14:31] replying to the reqs I meant ofc [09:14:42] ok, I thought it was local install only [09:14:45] sorry [09:14:54] my mistake [09:15:15] I am cool with it and it makes sense [14:37:13] puppet question: for https://gerrit.wikimedia.org/r/c/operations/puppet/+/536586 I need to read swift::params::account_keys from hiera for both sites, e.g. if I'm in eqiad I want to read the value for codfw too, any easy way to do that as it stands or I need to shuffle things around ? [14:49:16] <_joe_> godog: the latter [14:49:43] <_joe_> unless you can work some hacks with the lookup function, but I advise against doing non-obvious things [14:51:13] agreed, I'm assuming the new datastructure would look sth like eqiad => accounts, codfw => accounts, etc [15:00:06] <_joe_> yeah and be in a common hiera position [15:00:18] <_joe_> it sucks ofc, but better than doing puppet magic [21:09:40] can you just use hieradata/common.yaml ?