[00:01:17] PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:03:17] PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [04:03:46] PROBLEM - db1 Backups SQL on db1 is WARNING: FILE_AGE WARNING: /var/log/backup-logs/sql-backup.log is 864208 seconds old and 1977 bytes [15:33:10] PROBLEM - phorge1 phorge-static.wikitide.org HTTPS on phorge1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:37:28] RECOVERY - phorge1 phorge-static.wikitide.org HTTPS on phorge1 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 58195 bytes in 9.852 second response time [15:43:59] PROBLEM - phorge1 phorge-static.wikitide.org HTTPS on phorge1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:48:08] RECOVERY - phorge1 phorge-static.wikitide.org HTTPS on phorge1 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 58195 bytes in 1.707 second response time [17:32:05] PROBLEM - cp5 Varnish Backends on cp5 is CRITICAL: 1 backends are down. mw2 [17:32:20] PROBLEM - swiftproxy1 SSH on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:34:04] RECOVERY - cp5 Varnish Backends on cp5 is OK: All 6 backends are healthy [17:34:54] PROBLEM - swiftproxy1 Current Load on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:35:02] PROBLEM - swiftproxy1 conntrack_table_size on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:36:19] PROBLEM - swiftproxy1 Puppet on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:38:04] PROBLEM - swiftproxy1 PowerDNS Recursor on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:38:15] PROBLEM - swiftproxy1 ferm_active on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:38:28] PROBLEM - swiftproxy1 NTP time on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:41:37] RECOVERY - swiftproxy1 Puppet on swiftproxy1 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [17:42:50] RECOVERY - swiftproxy1 Current Load on swiftproxy1 is OK: LOAD OK - total load average: 1.52, 0.32, 0.10 [17:43:00] RECOVERY - swiftproxy1 conntrack_table_size on swiftproxy1 is OK: OK: nf_conntrack is 0 % full [17:43:02] RECOVERY - swiftproxy1 PowerDNS Recursor on swiftproxy1 is OK: DNS OK: 0.217 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66 [17:43:04] RECOVERY - swiftproxy1 ferm_active on swiftproxy1 is OK: OK ferm input default policy is set [17:43:26] PROBLEM - swiftproxy1 NTP time on swiftproxy1 is WARNING: NTP WARNING: Offset 0.1391014159 secs [17:44:44] RECOVERY - swiftproxy1 SSH on swiftproxy1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [17:45:28] RECOVERY - swiftproxy1 NTP time on swiftproxy1 is OK: NTP OK: Offset 0.001532256603 secs