[01:02:43] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 223 processes [01:08:44] PROBLEM Total processes is now: CRITICAL on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS CRITICAL: 371 processes [01:18:44] RECOVERY Total processes is now: OK on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS OK: 102 processes [02:37:44] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 199 processes [03:17:14] PROBLEM Free ram is now: UNKNOWN on aggregator2.pmtpa.wmflabs 10.4.0.193 output: NRPE: Call to fork() failed [03:22:12] PROBLEM Free ram is now: WARNING on aggregator2.pmtpa.wmflabs 10.4.0.193 output: Warning: 10% free memory [04:06:14] PROBLEM Free ram is now: WARNING on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: Warning: 19% free memory [04:07:44] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 205 processes [04:17:44] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 178 processes [04:51:14] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 154 processes [04:56:14] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 146 processes [05:41:22] PROBLEM Free ram is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: Critical: 5% free memory [05:46:13] PROBLEM Free ram is now: WARNING on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: Warning: 13% free memory [06:01:22] PROBLEM Free ram is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: Critical: 4% free memory [06:05:24] PROBLEM dpkg-check is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:07:44] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 202 processes [06:09:54] PROBLEM Current Load is now: WARNING on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: WARNING - load average: 1.33, 17.23, 12.14 [06:10:22] RECOVERY dpkg-check is now: OK on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: All packages OK [06:25:04] PROBLEM Free ram is now: WARNING on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: Warning: 19% free memory [06:29:55] RECOVERY Current Load is now: OK on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: OK - load average: 0.01, 0.83, 3.85 [06:44:52] PROBLEM SSH is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: CRITICAL - Socket timeout after 10 seconds [06:45:33] PROBLEM Disk Space is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: CHECK_NRPE: Socket timeout after 10 seconds. [06:48:03] PROBLEM Current Load is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: CHECK_NRPE: Socket timeout after 10 seconds. [06:48:23] PROBLEM dpkg-check is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:49:03] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [06:54:43] RECOVERY SSH is now: OK on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1 (protocol 2.0) [06:54:43] RECOVERY Total processes is now: OK on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: PROCS OK: 103 processes [06:55:24] RECOVERY Disk Space is now: OK on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: DISK OK [06:57:44] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 184 processes [07:02:44] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 202 processes [07:22:12] PROBLEM Free ram is now: CRITICAL on aggregator2.pmtpa.wmflabs 10.4.0.193 output: NRPE: Call to popen() failed [07:27:13] PROBLEM Free ram is now: UNKNOWN on aggregator2.pmtpa.wmflabs 10.4.0.193 output: NRPE: Call to fork() failed [07:32:13] PROBLEM Free ram is now: WARNING on aggregator2.pmtpa.wmflabs 10.4.0.193 output: Warning: 10% free memory [08:42:44] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 190 processes [08:54:05] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [08:57:49] PROBLEM Total processes is now: WARNING on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: PROCS WARNING: 151 processes [09:12:42] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 214 processes [09:27:44] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 199 processes [09:30:13] PROBLEM Total processes is now: WARNING on incubator-apache.pmtpa.wmflabs 10.4.0.116 output: PROCS WARNING: 158 processes [09:32:43] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 202 processes [10:10:22] PROBLEM Total processes is now: CRITICAL on incubator-apache.pmtpa.wmflabs 10.4.0.116 output: PROCS CRITICAL: 207 processes [11:04:09] PROBLEM Total processes is now: CRITICAL on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: PROCS CRITICAL: 204 processes [11:37:44] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 172 processes [12:12:42] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 202 processes [14:11:53] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [14:12:43] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 214 processes [15:12:43] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 175 processes [15:25:15] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 155 processes [15:30:22] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 129 processes [15:57:16] PROBLEM Free ram is now: CRITICAL on aggregator2.pmtpa.wmflabs 10.4.0.193 output: NRPE: Unable to read output [16:02:43] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 202 processes [16:07:14] PROBLEM Free ram is now: WARNING on aggregator2.pmtpa.wmflabs 10.4.0.193 output: Warning: 10% free memory [16:12:44] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 190 processes [16:13:14] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 162 processes [17:41:52] RECOVERY dpkg-check is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: All packages OK [17:42:12] RECOVERY Total processes is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: PROCS OK: 108 processes [17:43:42] RECOVERY Disk Space is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: DISK OK [17:43:52] RECOVERY SSH is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1 (protocol 2.0) [17:44:22] RECOVERY Free ram is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: OK: 94% free memory [17:44:42] RECOVERY Current Load is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: OK - load average: 1.01, 1.18, 1.35 [19:38:13] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 179 processes [19:48:13] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 124 processes [20:06:13] PROBLEM Total processes is now: CRITICAL on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS CRITICAL: 235 processes [20:27:13] PROBLEM Free ram is now: UNKNOWN on aggregator2.pmtpa.wmflabs 10.4.0.193 output: Unknown [20:32:12] PROBLEM Free ram is now: WARNING on aggregator2.pmtpa.wmflabs 10.4.0.193 output: Warning: 10% free memory [21:07:23] PROBLEM Free ram is now: WARNING on dumps-bot2.pmtpa.wmflabs 10.4.0.60 output: Warning: 19% free memory [21:22:42] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 202 processes [21:27:43] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 181 processes [22:02:42] PROBLEM Total processes is now: CRITICAL on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS CRITICAL: 208 processes [22:42:42] PROBLEM Total processes is now: WARNING on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS WARNING: 181 processes [23:06:13] PROBLEM Free ram is now: WARNING on wikidata-dev-9.pmtpa.wmflabs 10.4.1.41 output: Warning: 18% free memory