[09:30:57] https://www.logicalclocks.com/hopsworks seems really nice [09:31:27] and also they are working to fully support AMD ROCm GPUs (that we are using) [10:51:17] all deployers: scap sync --canary-wait-time option is available (https://phabricator.wikimedia.org/T217924) [12:11:36] hi all do we still use catchpoint (i think it may have got cancled around the time i started)? [12:30:22] yeah we ditched catchpoint [12:32:23] ack thanks raised https://gerrit.wikimedia.org/r/c/operations/puppet/+/606417 [13:09:37] godog: do you have any idea why puppet catalog is unhappy in pontoon? https://phabricator.wikimedia.org/P11587 [13:10:55] huh. for some reason `puppet catalog find --terminus json` works.. [13:12:00] kormat: no sorry, haven't used 'catalog download' [13:12:21] (i'm trying to view the catalog so i can see diffs when i make changes (assuming that makes sense :)) [13:12:27] "there's more than one way to do it" alright [13:13:26] yeah I usually peek on the filesystem for the catalog, limited of course to the agent having ran [13:13:30] "and they all suck" [13:13:49] where on the fs is it visible? [13:14:10] /var/lib/puppet/client_data/catalog [13:15:31] great, thank you :) [13:18:08] np! [15:45:35] jbond42: I'm trying to understand exactly why the key in question (profile::java::hardened_tls from hieradata/common/profile/profile/java.yaml) isn't visible to a VM in the first place. common is certainly present in our search path... [15:45:53] I've mostly avoided/ignored this problem historically so maybe it's a difference that's well-understood? [15:50:02] andrewbogott: the hiera config in production and labs is quite different. specificl labs uses nuyaml and production uses nuyaml3. Im not sure the difference between theses two versions. however i do note that in labs you do not configure any expand paths https://github.com/wikimedia/puppet/blob/production/modules/puppetmaster/files/production.hiera.yaml#L11-L13 [15:50:30] ah, that's probably it [15:50:33] this is the thing that makes nuyaml3 search for a key like profile::foo::bar in common/profile/foo.yaml instead of just in common.yaml [15:50:35] so the next question is… why don't we [15:50:51] i have no idea about that :) [15:52:01] any particular reason when you moved prod to nuyaml3 you didn't also move wmcs? [15:52:38] andrewbogott: im dont think i did that migration [15:52:51] git blame says you did :) [15:53:09] oh maybe this is just a cleanup/refactor [15:53:52] probably the later [15:55:30] here is the the tracking task https://phabricator.wikimedia.org/T188623 [15:56:10] andrewbogott: my gut feeling is that SRE dont know how your custom backends work so where reluctant to change anything [15:56:14] but before my time [15:56:59] also i think your custom backends are all version 1 and im not sure if there are any issues running some hiera v1 backends and some other with hiera v3 [15:57:20] here is the expand_paths story: https://gerrit.wikimedia.org/r/#/q/69e55590c178c585fafe7e691db6da25e93ee248 [16:00:26] andrewbogott: fyi i have this in the pipline https://gerrit.wikimedia.org/r/c/operations/puppet/+/566559. this CR dose nto affect cloud however ultimatly cloud will want to do a simlar thing. I have not looked at the cloud aspect for simlar reason to above. i.e. will need to rewrite your custom backends [16:01:38] T255787 [16:01:38] T255787: Reconcile and/or understand differences between cloud-vps and prod hiera lookups - https://phabricator.wikimedia.org/T255787 [16:02:13] jbond42: I'm generally in favor of that CI test but I want to better understand why we diverge so much from prod in the first place [16:02:33] ack sure [16:02:40] i have subscribed [16:31:48] hi all. I have a quick Puppet question. https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/606459/ is the commit and the issue is https://puppet-compiler.wmflabs.org/compiler1001/23324/malmok.wikimedia.org/change.malmok.wikimedia.org.err [16:32:00] I am trying to reference private data in hieradata and failing at it [16:32:10] can anyone have a quick look and tell me what's wrong here? thanks very much [16:32:33] I first thought that String[1] means a string of one character but I confirmed that it means min and not max, so that's good [16:32:55] then I tried referencing ::passwords from the role and that also failed (which was a bit stupid to do in hindsight but oh well) [16:33:17] (I have added the strings to both puppetmaster private and labs/private) [16:44:33] sukhe: I took a look and was similarly baffled [16:45:00] does it work if you just make it String instead of String[1]? [16:47:46] we already tried :) that means it compiles but can't read the values [16:48:09] i think String[1] means 1 character length, right? [16:48:14] minimum [16:48:19] it was not super obvious in docs [16:48:22] ok [16:48:33] > Data types sometimes take parameters, which make them more specific. (For example, String[8] is the data type of strings with a minimum of eight characters.) [16:48:35] https://puppet.com/docs/puppet/5.5/lang_data_type.html [16:48:36] cdanis: thanks for looking! and yeah I tried that [16:48:37] https://puppet.com/docs/puppet/latest/lang_data_string.html#parameters [16:48:43] says min [16:48:43] right [16:49:24] mutante: yeah, I thought the same but then pcc compiled and I noticed the empty string :) [16:49:57] okay here's a thought: does the puppet compiler use HEAD for labs/private? or does it use the version exported at https://config-master.wikimedia.org/labsprivate-sha1.txt (and thus a puppet-merge is required)? [16:50:15] Fetching new commits from: https://gerrit.wikimedia.org/r/labs/private [16:50:17] No changes to merge. [16:50:19] well, nevermind that then [16:54:24] maybe the "include ::password.." line would have to be in the role? (but that will make style-guide downvote it) [16:54:38] sukhe: i looked at the other example doing this (cassandra) [16:54:43] modules/role/manifests/restbase/base.pp: include ::passwords::cassandra [16:54:52] see that.. it would get downvoted if it was a new change [16:55:02] like you just pasted in the other window earlier [16:55:35] mutante: it gave me this error: https://integration.wikimedia.org/ci/job/operations-puppet-tests-buster-docker/5341/console [16:55:40] 12:24:08 wmf-style: total violations delta 1 [16:55:40] 12:24:08 NEW violations: [16:55:41] 12:24:08 modules/role/manifests/wikidough.pp:11 wmf-style: role 'role::wikidough' includes passwords::dnsdist::wikidough which is neither a role nor a profile [16:56:20] sukhe: yea, that's what i mean. but the existing example that you saw does it anyways [16:56:31] because that was done before the style check existed or it was overridden [16:56:40] aha, sorry misunderstood [16:56:43] hmm [16:57:25] sukhe: AIUI using passwords:: is kind of the 'old way' overall, there's a puppet-private hieradata subtree that I think is preferred nowadays [16:57:48] not that this is really documented anywhere at first, I'm going over my IRC logs from the last time this came up [16:58:46] thanks, I can try that as well. in 521f306f63aa392e4528483b04df9c1dd36861c4, I reference the private data as well and that works. the difference is that I did that in the profile itself and now I am trying to do it in the yaml [17:02:57] ah, the last time it came up: https://phabricator.wikimedia.org/P11592 [17:03:22] secret() is good for grabbing whole files (e.g. certificates, which use it) [17:03:36] private hiera seem to be preferred nowadays for short/password-ish things [17:06:27] sukhe: btw, I do like what you're doing overall, with writing config data in yaml backed by structs that define the types, that's nice [17:13:25] cdanis: thanks! I thought I should keep them as separate types that correspond to dnsdist's functions, that way I can easily match and update them as required [17:13:48] the current problem can be alleviated by making the secret data class parameters but I don't want to do that unless really required :) [17:15:17] I did that in 521f306 though as that was just one parameter but this is different as they belong to the same group