[00:34:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 196 processes [00:54:42] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 202 processes [00:58:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 187 processes [01:11:43] PROBLEM Total processes is now: CRITICAL on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS CRITICAL: 558 processes [01:16:43] RECOVERY Total processes is now: OK on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS OK: 99 processes [01:43:43] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 178 processes [02:08:43] PROBLEM Total processes is now: CRITICAL on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS CRITICAL: 205 processes [02:24:23] PROBLEM Total processes is now: CRITICAL on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS CRITICAL: 201 processes [02:29:23] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 106 processes [03:13:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 151 processes [03:34:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 197 processes [03:59:43] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 205 processes [04:03:42] PROBLEM Total processes is now: CRITICAL on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS CRITICAL: 205 processes [04:04:22] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 165 processes [04:08:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 196 processes [04:14:23] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 104 processes [04:33:42] PROBLEM Total processes is now: CRITICAL on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS CRITICAL: 211 processes [04:39:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 196 processes [04:54:43] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 223 processes [05:18:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 157 processes [06:00:52] PROBLEM Free ram is now: WARNING on incubator-apache.pmtpa.wmflabs 10.4.0.116 output: Warning: 19% free memory [06:59:42] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 214 processes [07:09:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 193 processes [07:24:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 169 processes [08:04:42] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 217 processes [08:49:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 196 processes [08:54:42] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 217 processes [10:03:23] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 195 processes [10:04:43] PROBLEM Total processes is now: CRITICAL on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS CRITICAL: 220 processes [10:09:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 193 processes [10:13:23] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 128 processes [13:05:23] PROBLEM Free ram is now: CRITICAL on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: Critical: 5% free memory [13:20:43] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 176 processes [13:25:42] RECOVERY Total processes is now: OK on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS OK: 148 processes [13:26:52] PROBLEM Free ram is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: Critical: 5% free memory [13:39:43] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 151 processes [14:54:43] PROBLEM Total processes is now: CRITICAL on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS CRITICAL: 202 processes [15:09:47] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 151 processes [15:29:43] RECOVERY Total processes is now: OK on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS OK: 148 processes [15:47:22] RECOVERY dpkg-check is now: OK on tools-shadow.pmtpa.wmflabs 10.4.1.66 output: All packages OK [15:48:52] RECOVERY Current Load is now: OK on tools-shadow.pmtpa.wmflabs 10.4.1.66 output: OK - load average: 0.29, 0.56, 0.29 [15:49:42] RECOVERY Disk Space is now: OK on tools-shadow.pmtpa.wmflabs 10.4.1.66 output: DISK OK [15:50:13] PROBLEM Free ram is now: UNKNOWN on tools-shadow.pmtpa.wmflabs 10.4.1.66 output: NRPE: Unable to read output [15:51:47] RECOVERY Total processes is now: OK on tools-shadow.pmtpa.wmflabs 10.4.1.66 output: PROCS OK: 79 processes [16:57:43] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 178 processes [16:59:43] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 229 processes [17:05:52] PROBLEM Current Load is now: CRITICAL on oauth-apache01.pmtpa.wmflabs 10.4.1.34 output: Connection refused by host [17:06:32] PROBLEM Disk Space is now: CRITICAL on oauth-apache01.pmtpa.wmflabs 10.4.1.34 output: Connection refused by host [17:06:42] PROBLEM Total processes is now: CRITICAL on oauth-apache01.pmtpa.wmflabs 10.4.1.34 output: Connection refused by host [17:07:12] PROBLEM Free ram is now: CRITICAL on oauth-apache01.pmtpa.wmflabs 10.4.1.34 output: Connection refused by host [17:07:32] PROBLEM dpkg-check is now: CRITICAL on oauth-apache01.pmtpa.wmflabs 10.4.1.34 output: Connection refused by host [17:08:52] PROBLEM Current Load is now: CRITICAL on oauth-sql01.pmtpa.wmflabs 10.4.1.69 output: Connection refused by host [17:08:52] PROBLEM Disk Space is now: CRITICAL on oauth-sql02.pmtpa.wmflabs 10.4.1.93 output: Connection refused by host [17:09:32] PROBLEM Disk Space is now: CRITICAL on oauth-sql01.pmtpa.wmflabs 10.4.1.69 output: Connection refused by host [17:09:32] PROBLEM Free ram is now: CRITICAL on oauth-sql02.pmtpa.wmflabs 10.4.1.93 output: Connection refused by host [17:10:23] PROBLEM Free ram is now: CRITICAL on oauth-sql01.pmtpa.wmflabs 10.4.1.69 output: Connection refused by host [17:11:03] PROBLEM Total processes is now: CRITICAL on oauth-sql02.pmtpa.wmflabs 10.4.1.93 output: Connection refused by host [17:11:43] PROBLEM Total processes is now: CRITICAL on oauth-sql01.pmtpa.wmflabs 10.4.1.69 output: Connection refused by host [17:11:43] PROBLEM dpkg-check is now: CRITICAL on oauth-sql02.pmtpa.wmflabs 10.4.1.93 output: Connection refused by host [17:12:23] PROBLEM dpkg-check is now: CRITICAL on oauth-sql01.pmtpa.wmflabs 10.4.1.69 output: Connection refused by host [17:12:53] PROBLEM Current Load is now: CRITICAL on oauth-sql02.pmtpa.wmflabs 10.4.1.93 output: Connection refused by host [17:22:42] PROBLEM Total processes is now: CRITICAL on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS CRITICAL: 204 processes [17:29:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 193 processes [17:49:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 181 processes [19:32:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 169 processes [20:03:23] PROBLEM Total processes is now: UNKNOWN on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: NRPE: Call to fork() failed [20:04:23] PROBLEM dpkg-check is now: CRITICAL on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: CHECK_NRPE: Error - Could not complete SSL handshake. [20:04:33] PROBLEM Current Load is now: CRITICAL on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: CHECK_NRPE: Error - Could not complete SSL handshake. [20:05:43] PROBLEM Disk Space is now: CRITICAL on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: CHECK_NRPE: Error - Could not complete SSL handshake. [20:07:53] PROBLEM SSH is now: CRITICAL on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: Server answer: [20:08:23] PROBLEM Total processes is now: CRITICAL on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: CHECK_NRPE: Error - Could not complete SSL handshake. [20:12:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 187 processes [20:14:32] RECOVERY Current Load is now: OK on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: OK - load average: 3.00, 1.71, 1.03 [20:15:23] RECOVERY Free ram is now: OK on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: OK: 41% free memory [20:15:43] RECOVERY Disk Space is now: OK on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: DISK OK [20:17:53] RECOVERY SSH is now: OK on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1 (protocol 2.0) [20:19:23] RECOVERY dpkg-check is now: OK on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: All packages OK [20:46:53] RECOVERY Free ram is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: OK: 39% free memory [21:19:42] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 187 processes [21:59:43] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 202 processes [22:27:23] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 151 processes [22:32:23] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 99 processes [22:34:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 196 processes [22:47:42] PROBLEM Total processes is now: WARNING on bots-bnr4.pmtpa.wmflabs 10.4.0.59 output: PROCS WARNING: 182 processes [22:54:44] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 209 processes [23:29:43] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 190 processes