[16:57:06] Hello. Is Toolforge registration currently down? [17:05:12] Prdandsuch: should be working, what error are you seeing? [17:05:42] (url and such would help too) [17:30:05] False-positive. Page threw a 500 internal error upon submission yet creation went through. [17:59:25] if it's giving a 500 there's probably some issue somewhere that should be fixed [18:11:22] cccccbukvgbcjicrlckekijeukidbcndjnhnnbndkdfr [18:20:55] mutante: cat or yubikey? [18:21:46] RhinosF1: always yubikey since it's the nano [18:23:00] the `c`s at the start are a sign of that being an yubikey [20:24:19] On horizon, does hieradata specified in both the puppet prefix and instance configuration merge together or does one take precedence? For instance, "mykey: [1, 2]" on the puppet prefix and "mykey: [1, 2, anotherkey: [3]]" on the instance hieradata [20:35:23] !log anticomposite@tools-bastion-13 tools.stewardbots SULWatcher/manage.sh restart # SULWatchers disconnected [20:35:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [21:13:05] brett: https://wikitech.wikimedia.org/wiki/Puppet/Hiera#The_lookup_hierarchy [21:13:38] oh, maybe that doesnt actually answer it, nevermind [21:13:56] just remembered "the hierachy was somewhere" [21:14:40] Yeah, this one's more about how it's merged together. I'm trying to scrub around the catalogs to see what's set and what's not and compare with the prefix/instance hieradata [21:14:50] (and if it's merged at all) [21:15:03] https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Hiera#Precedence_of_hiera_keys [21:15:31] "first match win" [21:15:55] Is that to say that, in my example, it would be "mykey: [1, 2]" and it wouldn't descend any further down? [21:17:13] sigh, linking to a 404 page on github as the place where we can check the "always up-to-date" version [21:17:38] I am fairly certain that the Puppetserver does hiera lookups in a "first answer wins" manner with a cascade of where to look. So if the first file has an answer things stop there, but if not the 2nd, 3rd, 4th, etc place is examined. [21:18:04] I see [21:19:16] It is possible to perform deep and shallow merges of hiera data, but that happens in explicit business logic in the Pippet manifests [21:19:22] *Puppet [21:19:59] See things like https://gerrit.wikimedia.org/g/operations/puppet/+/2e821327b5e7edd56d744552540fddb2c585ffcc/modules/admin/manifests/init.pp#37 [21:20:38] aha [21:20:53] I think that clarifies things for me. Thank you! [21:24:04] I'm attempting to clean up the duplications we have in our cloud project all around individual instances and the puppet prefix [21:26:16] mutante: I’m guessing the new “always up-to-date” file is one of these: https://codesearch.wmcloud.org/search/?q=Private%20hierarchy [21:26:22] but it’s not clear to me from https://github.com/wikimedia/operations-puppet/commit/20777c245cff0fdae80aac8727312a1d31f3c125 which one’s the best match [21:26:33] this is beyond my puppet skills :S [21:29:38] ack, thanks [21:54:34] brett: that can certainly become a mess. On paper the ideal organization is 1) global settings for the project at the project level, 2) setting for a shared instance role at the prefix level, and 3) instance specific overrides/tweaks at the instance level. In practice it can be difficult to stick with that structure, especially if there are many folks who regularly work on managing instances. [21:54:52] That's exactly what I'm trying to clean it up to do :) [21:55:28] I've succeeded in that organization but there's likely a ton of ancient hieradata values that have been removed years ago. But it's a start [21:55:28] Having things in the ops/puppet.git repo itself also complicates this in some projects like deployment-prep (which is probably a worst of all world example) [21:56:06] the traffic project doesn't seem to be in too bad of shape, our cp instances were just duplicating a lot. Now that it's cleaned up it's in pretty good order [21:56:22] as always, thanks for the help :) [21:57:34] One thing I would like to change about settings stored via horizon is the destructive YAML normalization that happens on save. I would like input order and comments to survive there, but I haven't ever tried to do the work to make that happen. [21:57:59] ah, that would be nice [21:58:10] certainly was unexpected the first time I noticed it eat my comment [22:03:08] I think a.ndrewbogott had a good instinct to apply validation/normalization there, but it would be nice to be able to round trip comments and ordering [22:23:03] yep, most python yaml parsing libraries (and golang for that matter) strip the comments when parsing :/, you need to write your own custom one, or work around it by making sure to store the original [22:23:43] (and pass the string around, instead/alongside of yaml objects) [23:03:38] !status Ok