[06:44:48] <elukey>	 _joe_: atm I have tcpdump with -W and -C, plus ifstat in tail (grepping for high tx bandwidth usage). When I see high values I'll just check the pcap, in this way I don't cause cpu usage for mc1027 :(
[07:08:40] <_joe_>	 elukey: my point is - in the moment of congestion, it's ok to inspect with memkeys
[07:16:19] <elukey>	 _joe_ sure but usually it last few seconds, it is difficult to get it if not sustained
[07:17:05] <_joe_>	 well as long as it's a few seconds and not sustained, it's less of an emergency and that should be fixed by better telemetry from MediaWiki imho
[07:27:30] <elukey>	 I am not saying it is an emergency, just trying to reduce the wtp* servers tko, I agree that we should have a better telemetry
[07:27:50] <elukey>	 but things are not really moving on this front :)
[09:24:12] <elukey>	 so the key that causes some problems should be
[09:24:13] <elukey>	 WANCache:v:nlwiki:preprocess-hash:6b35ce98ba6b117cf3eeb7807471739e:1
[09:24:26] <elukey>	 in slab 154, mc1027 (~200K)
[09:37:36] <elukey>	 created https://phabricator.wikimedia.org/T248962
[15:01:04] <andrewbogott>	 jbond42 (and everyone), when you add a new required param to a profile, please add a default to cloud.yaml.  e.g. https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/584951/
[15:06:01] <jbond42>	 andrewbogott: sorry, i have created https://phabricator.wikimedia.org/T248994 to see if we can check for this in CI
[15:06:18] <andrewbogott>	 that would be great!
[15:16:16] <andrewbogott>	 jbond42: while we're at it… can you have a go at getting puppet to run on jbond-stretch.puppet.eqiad.wmflabs?  I'm trying to get catalogs to compile everywhere so that the puppet/facter version upgrades can take
[15:17:53] <jbond42>	 andrewbogott: one sec i think i can just delete that
[15:18:00] <andrewbogott>	 better yet!
[15:19:23] <jbond42>	 andrewbogott: have deleted all my instances in the puppet project
[15:19:35] <andrewbogott>	 great, thanks!
[15:46:00] <chaomodus>	 o/
[15:47:55] <rlazarus>	 👋
[16:00:44] <hnowlan>	 Is there an easy way/established pattern for giving access to grafana and nothing/very little else for a NDAed user? (A contractor on CPT in this case)
[16:02:24] <volans>	 hnowlan: usually the NDA group, see https://wikitech.wikimedia.org/wiki/LDAP/Groups
[16:02:35] <volans>	 AFAIK there isn't a smaller one at this time
[16:03:34] <volans>	 hnowlan: I assume write access to grafana, as RO is for everyone
[16:04:20] <hnowlan>	 Just read access in this case
[16:04:45] <volans>	 then just a browser :-P
[16:05:36] <hnowlan>	 They don't have an account at all right now, heh.
[16:05:42] <hnowlan>	 But that sounds suitable
[16:05:44] <volans>	 not needed
[16:06:51] <hnowlan>	 oh, sorry, logstash is needed too
[16:07:01] <volans>	 then NDA
[16:07:17] <hnowlan>	 cool, thanks!
[16:07:33] <volans>	 follow the instructions at the top of the page