[14:57:18] hi folks, I was just noticing jinxer didn't come back yet, not sure what's up though I'd be definitely expecting some more followups from it [14:58:35] godog: Let me bounce it, thank you!! [14:58:43] denisse: sure np! thank you [15:11:39] It should be back, it seems it got caught in a loop after the disconnection. [15:12:07] I'l lfile a task with the error. [15:41:28] RESOLVED: IcingaOverload: Checks are taking long to execute on alert1002:9245 - https://wikitech.wikimedia.org/wiki/Icinga#IcingaOverload - https://grafana.wikimedia.org/d/rsCfQfuZz/icinga - https://alerts.wikimedia.org/?q=alertname%3DIcingaOverload [17:14:25] FIRING: [2x] SystemdUnitFailed: statograph_post.service on alert1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [17:19:25] FIRING: [2x] SystemdUnitFailed: statograph_post.service on alert1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [17:22:12] ^ looking. [17:26:47] It's this metric 'lyfcttm2lhw4', I think it's diving by 0 or something that throws the NaN. [17:44:25] RESOLVED: [2x] SystemdUnitFailed: statograph_post.service on alert1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [17:44:37] It was a weird edge case, I'll send a patch. [18:10:05] thanks denisse <3