[14:04:42] <cwhite>	 dr0ptp4kt, inflatador: we built our own debs when upstream did not yet build them.  Now that upstream publishes debs, we use theirs in conjunction with a puppet provider to manage plugins.
[14:08:26] <jinxer-wm>	 FIRING: SystemdUnitFailed: oauth2-proxy.service on prometheus3003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[14:09:41] <inflatador>	 cwhite ACK, thanks for confirming
[14:11:04] <godog>	 that is me btw ^
[14:51:02] <gmodena>	 btullis looks like we still have gobblin metrics for the job we decommissioned (curl  http://prometheus-pushgateway.discovery.wmnet/metrics | grep webrequest_frontend_rc0 | wc -l)
[14:52:25] <gmodena>	 deleting metrics with curl -XDELETE http://prometheus-pushgateway.discovery.wmnet/metrics/job/webrequest_frontend_rc0 did not seem to have an effect
[14:54:51] <gmodena>	 any chance we are hitting a ro endpoint ?
[14:55:40] <gmodena>	 ^ godog 
[15:04:43] <godog>	 gmodena: I'll take a look
[15:05:53] <gmodena>	 godog thanks - and no rush :)
[15:07:28] <godog>	 gmodena: ok! I can confirm what you mentioned, I'll take a closer look this week unless someone beats me to it
[15:12:47] <gmodena>	 godog ack!
[16:07:08] <dr0ptp4kt>	 thanks cwhite. I'm out for a few days here, and will be interested to learn more sometime in the days after return. have a great Monday!
[17:39:25] <mutante>	 If you add a new file under /modules/prometheus/files/usr/local/bin/  in the puppet repo, where would it show up? it seemed to me the answer would be "anywhere" since that is recursively managed and applied in a base class, but I guess not?
[17:41:35] <mutante>	 is there anther place besides https://wikitech.wikimedia.org/wiki/Prometheus#Exporters that explains how to install new exporters?
[17:47:59] <mutante>	 hmm.. maybe I am just missing that my new class isn't included in my role. but then.. should I really use classes in the prometheus module to include in other profiles. I guess the style guide would say that's not good
[17:49:17] <mutante>	 well, seems like everybody else is doing that so I'll do it too
[18:08:41] <jinxer-wm>	 FIRING: SystemdUnitFailed: oauth2-proxy.service on prometheus3003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed