' [18:20:38] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5821 bytes in 0.107 second response time [18:31:46] RECOVERY - procs on db2002 is OK: PROCS OK: 103 processes [18:31:53] RECOVERY - users on db2002 is OK: USERS OK - 1 users currently logged in [18:31:54] PROBLEM - apt on db2002 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (11 critical updates). [18:31:57] RECOVERY - ufw service on db2002 is OK: OK: Status of the systemd unit ufw [18:31:59] from /usr/lib/nagios/plugins/check_puppet_run:12:in `

' [18:32:03] RECOVERY - disk / on db2002 is OK: DISK OK - free space: / 29178 MB (93% inode=97%); /boot/efi 126 MB (99% inode=-); [18:32:07] RECOVERY - ssh #page on db2002 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [18:32:09] HTTP CRITICAL - Unable to open TCP socket [18:32:11] RECOVERY - disk on db2002 is OK: DISK OK - free space: / 29178 MB (93% inode=97%); /boot/efi 126 MB (99% inode=-); [18:32:15] RECOVERY - ping6 #page on db2002 is OK: PING OK - Packet loss = 0%, RTA = 0.29 ms [18:32:39] RECOVERY - load on db2002 is OK: OK - load average: 0.13, 0.38, 0.23 [18:32:45] RECOVERY - puppet service on db2002 is OK: OK: Status of the systemd unit run-puppet [18:39:53] RECOVERY - apt on db2002 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:40:13] RECOVERY - apt on tools2002 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:40:39] RECOVERY - apt on bots2002 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:45:13] PROBLEM - Puppet on db2002 is UNKNOWN: UNKNOWN: Failed to check. Reason is: no_summary_file [18:48:36] PROBLEM - Puppet on tools2002 is UNKNOWN: UNKNOWN: Failed to check. Reason is: no_summary_file [18:50:10] MacFan4000: I think that's a check issue, you should be able to see how to fix in the mh puppet repo ^ [18:50:12] PROBLEM - Puppet on bots2002 is UNKNOWN: UNKNOWN: Failed to check. Reason is: no_summary_file [18:50:38] it changed to that because im doing stuff to fix it [18:53:40] Ok [18:54:12] RECOVERY - Puppet on db2002 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [18:54:26] RECOVERY - Puppet on bots2002 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [18:54:32] PROBLEM - Puppet on tools2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[python-pygments] [18:55:49] there we go [18:56:15] and it shouldn't be adding a http check for db2002 yet it is [18:57:28] No it shouldn't [18:57:42] python-pygments I guess should become 3 [18:58:05] not sure why it is though - https://github.com/MirahezeBots/puppet/blob/master/modules/icinga2/templates/zones.d/master/hosts.conf#L195 is set to false [18:58:05] [url] puppet/hosts.conf at master · MirahezeBots/puppet · GitHub | github.com [18:58:45] No idea [19:00:20] from /usr/lib/nagios/plugins/check_puppet_run:154:in `

' [19:03:13] PROBLEM - Puppet on tools2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[python-pygments] [19:05:43] [02puppet] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/677e09b03d9d...0e4bfff97dce [19:05:44] [02puppet] 07MacFan4000 030e4bfff - Update hosts.conf [19:07:28] [02puppet] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/0e4bfff97dce...e0b9cef3b5b8 [19:07:30] [02puppet] 07MacFan4000 03e0b9cef - Update init.pp [19:08:57] from /usr/lib/nagios/plugins/check_puppet_run:154:in `

' [19:09:55] RECOVERY - Puppet on tools2002 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [19:10:51] thats all icinga errors resolved [20:18:27] [02puppet] 07sario528 pushed 031 commit to 03sario528-patch-1 [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/commit/0023430be252 [20:18:28] [02puppet] 07sario528 030023430 - Add channel [20:18:30] [02puppet] 07sario528 created branch 03sario528-patch-1 - 13https://github.com/MirahezeBots/puppet [20:18:48] [02puppet] 07sario528 opened pull request 03#268: Add channel - 13https://github.com/MirahezeBots/puppet/pull/268 [20:32:05] [02puppet] 07sario528 pushed 031 commit to 03sario528-patch-1 [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/0023430be252...bea2877b0486 [20:32:07] [02puppet] 07sario528 closed pull request 03#268: Add channel - 13https://github.com/MirahezeBots/puppet/pull/268 [20:32:08] [02puppet] 07sario528 synchronize pull request 03#268: Add channel - 13https://github.com/MirahezeBots/puppet/pull/268 [20:32:56] [02puppet] 07sario528 deleted branch 03sario528-patch-1 [20:32:58] [02puppet] 07sario528 deleted branch 03sario528-patch-1 - 13https://github.com/MirahezeBots/puppet [23:00:15] $log added bots2002 and tools2002 to web rotation [23:00:23] Saved item "added bots2002 and tools2002 to web rotation" [23:01:35] $log remove tools1 from rotation [23:01:51] Saved item "remove tools1 from rotation" [23:03:26] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1935 bytes in 0.635 second response time [23:03:33] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1931 bytes in 0.093 second response time [23:03:38] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 2189 bytes in 0.101 second response time [23:03:50] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1925 bytes in 0.107 second response time [23:32:38] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5552 bytes in 0.505 second response time [23:32:49] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 5540 bytes in 0.073 second response time [23:33:26] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5555 bytes in 0.223 second response time [23:33:34] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5553 bytes in 0.107 second response time [23:35:25] [02puppet] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/bea2877b0486...0cb01fff1040 [23:35:26] [02puppet] 07MacFan4000 030cb01ff - set loglevel to debug [23:55:25] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1938 bytes in 0.216 second response time [23:55:33] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1936 bytes in 0.101 second response time [23:55:37] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1936 bytes in 0.119 second response time [23:55:49] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1923 bytes in 0.078 second response time [23:56:25] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 9885 bytes in 0.362 second response time [23:56:33] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 9637 bytes in 0.126 second response time [23:56:37] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 9651 bytes in 0.109 second response time [23:56:49] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 9629 bytes in 0.060 second response time [23:57:37] PROBLEM - Phabricator on tools1 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 2095 bytes in 0.074 second response time