[10:20:51] <topranks>	 godog: looking at the memory usage on netmon1003 it seems to sawtooth up to max and then down again every few days 
[10:21:23] <topranks>	 though it's not needed any intervention until now seemingly 
[10:22:16] <godog>	 yeah filed as https://phabricator.wikimedia.org/T397427
[10:23:51] <godog>	 I do see the sawtooth, also previously atlas-exporter wasn't using 6G of memory
[10:24:00] <godog>	 I'm looking at https://grafana.wikimedia.org/goto/GGVs-4ENR?orgId=1
[10:29:00] <topranks>	 ok yeah, no particular changes to either of those that I know of around the dates they seemed to spike up 
[10:29:19] <topranks>	 atlast-exporter memory seems more reasonable since the reboot, I guess we can keep an eye on it 
[10:30:22] <godog>	 yeah I'm going to lunch shortly, will take another look this afternoon, for sure librenms-syslog is getting a MemoryMax systemd
[20:40:40] <jinxer-wm>	 FIRING: LogstashKafkaConsumerLag: Too many messages in logging-eqiad for group logstash7-eqiad - https://wikitech.wikimedia.org/wiki/Logstash#Kafka_consumer_lag - https://grafana.wikimedia.org/d/000000484/kafka-consumer-lag?var-cluster=logging-eqiad&var-datasource=eqiad%20prometheus/ops - https://alerts.wikimedia.org/?q=alertname%3DLogstashKafkaConsumerLag
[20:45:40] <jinxer-wm>	 RESOLVED: LogstashKafkaConsumerLag: Too many messages in logging-eqiad for group logstash7-eqiad - https://wikitech.wikimedia.org/wiki/Logstash#Kafka_consumer_lag - https://grafana.wikimedia.org/d/000000484/kafka-consumer-lag?var-cluster=logging-eqiad&var-datasource=eqiad%20prometheus/ops - https://alerts.wikimedia.org/?q=alertname%3DLogstashKafkaConsumerLag