[08:20:17] BTW, I don't know if this been shared already, but it looks nice: https://opensource.com/article/19/8/introduction-bpftrace [08:21:13] <_joe_> I saw it, and I told myself I need to read and learn :P [08:39:14] godog: thx for reviewing https://gerrit.wikimedia.org/r/c/operations/puppet/+/531334, I've already submitted a new PS [08:41:10] vgutierrez: thanks! yeah I just took another look and I apologize the instructions were unclear [08:41:20] yup.. np [09:57:00] vgutierrez: https://github.com/cloudflare/bpftools and the blogs mentioned in the readme are also worth looking at [14:06:25] I'm interested in getting all puppet code for a certain class at their call site, including all parameters, do you know if there's a simple way to do that? git grep --function-context out of the box sadly isn't doing what I was expecting [14:06:29] e.g. [14:06:31] git grep -h --function-context nrpe::monitor_service . [14:06:55] whereas I'd like to have all calls to nrpe::monitoring_service and their parameters [14:09:12] godog: puppetdb [14:09:25] should be able to answer you [14:09:35] godog: one moment [14:09:55] godog: https://phabricator.wikimedia.org/P8744 can probably help you [14:10:22] just query the resources endpoint [14:10:26] with the appropriate parameters [14:10:44] the appropriate parameters are impossible to craft if you are not volans ;) [14:10:50] oohh nice, thanks volans and cdanis ! [14:10:52] see https://puppet.com/docs/puppetdb/4.4/api/query/v4/resources.html [14:11:08] yeah that sounded a bit like "it is just a matter of programming" ;) [14:11:41] I was being a little naive and thought I could escape puppetdb [14:11:49] cdanis: /pdb/query/v4/resources/ doesn't seem so terrible [14:12:00] volans: the problem is the query strings [14:12:16] last time I spent two minutes reading https://puppet.com/docs/puppetdb/4.4/api/query/v4/ast.html and then I gave up and just fetched everything [14:12:54] no need for other params there [14:13:26] that's true, there is the intermediate "fetch all nrpe::monitoring_service" resources [14:14:38] godog: curl -sG "https://puppetdb1001.eqiad.wmnet/pdb/query/v4/resources/Nrpe::Monitor_service" | jq "." > monitor_service.json [14:14:49] from a cumin host or a puppetmaster or a puppetdb host [14:15:16] 27M [14:15:42] 23378 items [14:16:44] /o\ ok thanks a lot both! I'll dig further into puppetdb [14:16:50] * godog grabs hardhat [14:17:08] godog: what are you looking specifically? [14:17:41] it is much easier to answer "was nrpe::monitor_service ever called with X argument for Y parameter" [14:17:49] jq can do that pretty easily [14:17:56] getting a list of all variations of all parameters used will be harder [14:18:41] godog: the puppetdb output includes file and line, so you can group by them if you need code references [14:19:04] I'm interested in an audit on how we're calling/using alerts from puppet, e.g. nrpe::monitor_service [14:19:29] yeah I think the json output is workable [14:20:17] 117 different files [14:20:45] sorry 116 [14:22:39] the other approach from the code would be with awk I guess, getting from nrpe::monitor_service to the next } [14:23:45] yeah I thought about it too, but it gets hairy pretty fast, } might mean ${foo} and then adding exceptions hoping nothing is missing [14:24:15] right [14:27:29] but yeah json from puppetdb totally works, thanks again [14:28:20] no prob, if you have specific questions on parameters we could do more tweaked queries but for exploration I think the big json + playing with jq gives you more flexibility [17:18:17] anybody up for a quick DNS change review? https://gerrit.wikimedia.org/r/531725 [17:18:20] :) [17:19:44] {done} [18:31:52] someone knows how to use the geoip module in puppet? (I want to add geoip DB files to a server) [18:32:06] I hope It's just an include ::something [18:33:54] I'm gonna guess you just want geoip::data::puppet [18:34:19] that's what authdns seems to use [18:36:53] I see that I can do `include ::geoip` [18:37:15] according to the geoip module [18:37:29] also used in profile::kafkatee::webrequest::ops [18:37:33] looks like that is what kafkatee::webrequest does [18:37:36] yeah [18:39:00] it's amazing if it's that easy [18:39:59] oh! geoip just glues together geoip::data::puppet and geoip::bin [18:40:02] yeah, use that [18:41:51] yeah, but there is only two files in netflow1001:/usr/share/GeoIP/ (where kafkacat is configured too) [18:43:38] while those files should be there as well https://github.com/wikimedia/puppet/blob/b347052863d4d2e87b37d6c2d9f44f833cfd9dc2/modules/puppetmaster/manifests/geoip.pp#L59 [18:44:30] and mentioned in geodns as well https://github.com/wikimedia/puppet/blob/50d493fbd9f012492c0cb4841387d441d108facf/modules/authdns/manifests/init.pp#L71 [18:46:20] anyway, lunch time will look more later [20:39:59] cdanis: actually yeah it seems like I just have to include ::geoip, but I get the lint error "wmf-style: profile 'profile::fastnetmon' includes non-profile class geoip" [20:40:18] not sure if that's a case where I can do `include ::geoip # lint:ignore:wmf_styleguide` [20:42:49] I do not know enough about Puppet to be able to say if instantiating geoip with a dummy name is the right thing to do [20:43:43] I guess you could try it though? geoip {'foo': } [20:44:58] cdanis: if several profiles call the same module, it's instantiated only once, right? [20:45:22] my model is that if they all compile down to the same set of resources, it's the same thing [20:46:16] probably the code gets evaluated multiple times, but since the only output is e.g. a set of the same file resources, it doesn't actually matter? [20:46:27] cdanis: should I do class { '::geoip': } ? [20:48:09] I read https://puppet.com/docs/puppet/5.3/lang_classes.html#include-like-vs-resource-like and now I'm more confused than I was when I started [20:49:53] sorry :) [20:50:11] I guess the other thing you could do [20:50:19] is put the include statement in the base fastnetmon module [20:50:22] instead of in the profile [20:50:29] that might make the linter happier [20:50:37] nah [20:50:52] I tried, and have another similar lint error [20:56:36] cdanis: https://puppet-compiler.wmflabs.org/compiler1001/17989/netflow1001.eqiad.wmnet/change.netflow1001.eqiad.wmnet.err [20:56:42] I guess that says it all [20:56:51] hah [20:57:01] yeah [20:57:03] er, no, my mistake I put them both [20:58:30] this one works [20:58:32] https://puppet-compiler.wmflabs.org/compiler1002/17990/netflow1001.eqiad.wmnet/ [20:59:00] interesting, puppet compiller went faster than the linter [20:59:06] still no +2 [21:00:54] seems like it's working as expected! [21:59:32] cdanis: https://puppet-compiler.wmflabs.org/compiler1001/17992/netflow1001.eqiad.wmnet/change.netflow1001.eqiad.wmnet.err [21:59:37] ah, no we now! [21:59:39] now* [22:00:02] I have the geoip stuff declared in 2 profiles, under the same role [22:00:24] and no bueno, so it needs the include