[00:00:25] <icinga-wikitide>	 PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN
[00:02:25] <icinga-wikitide>	 PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is CRITICAL: CRITICAL - NGINX Error Rate is 100%
[00:08:10] <botifico>	 [02WikiTideOrg/ssl] 07WikiTideSSL pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/ssl/compare/546db8b9e495...bcbd67025f5b
[00:08:11] <botifico>	 [02WikiTideOrg/ssl] 07WikiTideSSLBot 03bcbd670 - Bot: Update SSL cert for lophocmatngu.wiki
[02:51:19] <icinga-wikitide>	 PROBLEM - prometheus1 PowerDNS Recursor on prometheus1 is CRITICAL: Domain 'wikitide.org' was not found by the server
[02:53:19] <icinga-wikitide>	 RECOVERY - prometheus1 PowerDNS Recursor on prometheus1 is OK: DNS OK: 0.036 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66
[03:02:51] <icinga-wikitide>	 PROBLEM - swiftproxy1 SSH on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:03:03] <icinga-wikitide>	 PROBLEM - swiftproxy1 PowerDNS Recursor on swiftproxy1 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[03:04:52] <icinga-wikitide>	 PROBLEM - mw1 MediaWiki Rendering on mw1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:04:53] <icinga-wikitide>	 PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[03:05:07] <icinga-wikitide>	 PROBLEM - swiftproxy1 Current Load on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[03:05:37] <icinga-wikitide>	 PROBLEM - swiftproxy1 Puppet on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[03:06:09] <icinga-wikitide>	 PROBLEM - swiftproxy1 conntrack_table_size on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[03:06:47] <icinga-wikitide>	 PROBLEM - mw1 MediaWiki Rendering on mw1 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 8191 bytes in 0.157 second response time
[03:07:53] <icinga-wikitide>	 RECOVERY - swiftproxy1 PowerDNS Recursor on swiftproxy1 is OK: DNS OK: 0.386 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66
[03:08:51] <icinga-wikitide>	 PROBLEM - swiftproxy1 NTP time on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:08:52] <icinga-wikitide>	 RECOVERY - swiftproxy1 conntrack_table_size on swiftproxy1 is OK: OK: nf_conntrack is 0 % full
[03:10:28] <icinga-wikitide>	 PROBLEM - cp1 PowerDNS Recursor on cp1 is CRITICAL: Domain 'wikitide.org' was not found by the server
[03:10:33] <icinga-wikitide>	 RECOVERY - swiftproxy1 Puppet on swiftproxy1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[03:10:54] <icinga-wikitide>	 RECOVERY - swiftproxy1 NTP time on swiftproxy1 is OK: NTP OK: Offset -0.002576738596 secs
[03:11:00] <icinga-wikitide>	 RECOVERY - swiftproxy1 SSH on swiftproxy1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0)
[03:14:35] <icinga-wikitide>	 RECOVERY - cp1 PowerDNS Recursor on cp1 is OK: DNS OK: 5.002 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66
[03:28:12] <icinga-wikitide>	 PROBLEM - swiftproxy1 Current Load on swiftproxy1 is WARNING: LOAD WARNING - total load average: 0.37, 1.25, 7.08
[03:30:12] <icinga-wikitide>	 RECOVERY - swiftproxy1 Current Load on swiftproxy1 is OK: LOAD OK - total load average: 0.57, 0.99, 6.27
[03:32:53] <icinga-wikitide>	 RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures
[14:30:52] <icinga-wikitide>	 PROBLEM - mw1 MediaWiki Rendering on mw1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:32:47] <icinga-wikitide>	 PROBLEM - mw1 MediaWiki Rendering on mw1 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 8191 bytes in 0.162 second response time
[14:32:51] <icinga-wikitide>	 PROBLEM - swiftproxy1 SSH on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:34:40] <icinga-wikitide>	 PROBLEM - swiftproxy1 PowerDNS Recursor on swiftproxy1 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[14:35:22] <icinga-wikitide>	 PROBLEM - swiftproxy1 Current Load on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[14:35:39] <icinga-wikitide>	 PROBLEM - swiftproxy1 conntrack_table_size on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[14:35:42] <icinga-wikitide>	 PROBLEM - swiftproxy1 ferm_active on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[14:36:06] <icinga-wikitide>	 PROBLEM - swiftproxy1 NTP time on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:36:22] <icinga-wikitide>	 PROBLEM - swiftproxy1 Puppet on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[14:39:11] <icinga-wikitide>	 PROBLEM - mw2 MediaWiki Rendering on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:41:07] <icinga-wikitide>	 PROBLEM - mw2 MediaWiki Rendering on mw2 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 8191 bytes in 0.175 second response time
[14:44:52] <icinga-wikitide>	 RECOVERY - swiftproxy1 conntrack_table_size on swiftproxy1 is OK: OK: nf_conntrack is 1 % full
[14:44:53] <icinga-wikitide>	 RECOVERY - swiftproxy1 ferm_active on swiftproxy1 is OK: OK ferm input default policy is set
[14:44:56] <icinga-wikitide>	 RECOVERY - swiftproxy1 PowerDNS Recursor on swiftproxy1 is OK: DNS OK: 1.104 second response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66
[14:45:10] <icinga-wikitide>	 RECOVERY - swiftproxy1 SSH on swiftproxy1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0)
[14:45:19] <icinga-wikitide>	 RECOVERY - swiftproxy1 Puppet on swiftproxy1 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures
[14:45:27] <icinga-wikitide>	 RECOVERY - swiftproxy1 NTP time on swiftproxy1 is OK: NTP OK: Offset -0.002763658762 secs
[15:08:37] <icinga-wikitide>	 PROBLEM - swiftproxy1 Current Load on swiftproxy1 is WARNING: LOAD WARNING - total load average: 0.06, 0.57, 7.70
[15:12:24] <icinga-wikitide>	 RECOVERY - swiftproxy1 Current Load on swiftproxy1 is OK: LOAD OK - total load average: 0.18, 0.38, 6.05
[17:03:05] <icinga-wikitide>	 PROBLEM - jobrunner1 Current Load on jobrunner1 is WARNING: LOAD WARNING - total load average: 7.05, 6.70, 5.88
[17:05:03] <icinga-wikitide>	 RECOVERY - jobrunner1 Current Load on jobrunner1 is OK: LOAD OK - total load average: 5.94, 6.45, 5.90
[17:18:43] <icinga-wikitide>	 PROBLEM - jobrunner1 Current Load on jobrunner1 is WARNING: LOAD WARNING - total load average: 7.68, 6.81, 6.30
[17:20:41] <icinga-wikitide>	 RECOVERY - jobrunner1 Current Load on jobrunner1 is OK: LOAD OK - total load average: 5.79, 6.51, 6.26
[17:26:31] <icinga-wikitide>	 PROBLEM - jobrunner1 Current Load on jobrunner1 is WARNING: LOAD WARNING - total load average: 7.10, 6.75, 6.41
[17:28:29] <icinga-wikitide>	 RECOVERY - jobrunner1 Current Load on jobrunner1 is OK: LOAD OK - total load average: 6.10, 6.60, 6.40
[17:41:07] <icinga-wikitide>	 PROBLEM - jobrunner1 Current Load on jobrunner1 is WARNING: LOAD WARNING - total load average: 6.98, 6.96, 6.71
[17:43:04] <icinga-wikitide>	 RECOVERY - jobrunner1 Current Load on jobrunner1 is OK: LOAD OK - total load average: 6.48, 6.80, 6.69
[17:46:57] <icinga-wikitide>	 PROBLEM - jobrunner1 Current Load on jobrunner1 is WARNING: LOAD WARNING - total load average: 6.56, 6.91, 6.77
[18:12:26] <icinga-wikitide>	 RECOVERY - jobrunner1 Current Load on jobrunner1 is OK: LOAD OK - total load average: 5.78, 6.28, 6.71