[01:10:40] PROBLEM Disk Space is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [01:15:40] RECOVERY Disk Space is now: OK on mobile-testing i-00000271 output: DISK OK [01:18:40] PROBLEM Total Processes is now: CRITICAL on aggregator-test1 i-000002bf output: PROCS CRITICAL: 206 processes [01:23:40] PROBLEM Total Processes is now: WARNING on aggregator-test1 i-000002bf output: PROCS WARNING: 198 processes [01:49:30] PROBLEM Free ram is now: CRITICAL on bots-sql2 i-000000af output: CHECK_NRPE: Socket timeout after 10 seconds. [01:54:20] PROBLEM Free ram is now: WARNING on bots-sql2 i-000000af output: Warning: 19% free memory [02:39:22] RECOVERY Free ram is now: OK on bots-sql2 i-000000af output: OK: 23% free memory [03:07:02] RECOVERY Puppet freshness is now: OK on dumps-incr i-000002bb output: puppet ran at Sat Jun 23 03:06:45 UTC 2012 [03:16:42] PROBLEM Free ram is now: WARNING on incubator-bot1 i-00000251 output: Warning: 19% free memory [03:31:42] PROBLEM Free ram is now: WARNING on nova-daas-1 i-000000e7 output: Warning: 15% free memory [03:50:10] PROBLEM Free ram is now: WARNING on utils-abogott i-00000131 output: Warning: 17% free memory [03:54:40] PROBLEM Free ram is now: WARNING on test-oneiric i-00000187 output: Warning: 17% free memory [03:56:40] PROBLEM Free ram is now: CRITICAL on nova-daas-1 i-000000e7 output: Critical: 4% free memory [04:01:40] PROBLEM Free ram is now: WARNING on orgcharts-dev i-0000018f output: Warning: 14% free memory [04:01:40] RECOVERY Free ram is now: OK on nova-daas-1 i-000000e7 output: OK: 94% free memory [04:10:10] PROBLEM Free ram is now: CRITICAL on utils-abogott i-00000131 output: Critical: 4% free memory [04:14:40] PROBLEM Free ram is now: CRITICAL on test-oneiric i-00000187 output: Critical: 4% free memory [04:15:10] RECOVERY Free ram is now: OK on utils-abogott i-00000131 output: OK: 97% free memory [04:16:40] PROBLEM Free ram is now: CRITICAL on orgcharts-dev i-0000018f output: Critical: 5% free memory [04:19:40] RECOVERY Free ram is now: OK on test-oneiric i-00000187 output: OK: 97% free memory [04:22:20] PROBLEM Free ram is now: WARNING on bots-sql2 i-000000af output: Warning: 19% free memory [04:26:37] RECOVERY Free ram is now: OK on orgcharts-dev i-0000018f output: OK: 95% free memory [05:51:23] PROBLEM Free ram is now: CRITICAL on etherpad-lite i-000002de output: CHECK_NRPE: Socket timeout after 10 seconds. [05:56:13] PROBLEM Free ram is now: UNKNOWN on etherpad-lite i-000002de output: NRPE: Unable to read output [06:27:33] PROBLEM Free ram is now: CRITICAL on configtest-main i-000002dd output: CHECK_NRPE: Socket timeout after 10 seconds. [06:28:37] PROBLEM Total Processes is now: CRITICAL on aggregator-test1 i-000002bf output: PROCS CRITICAL: 203 processes [06:32:07] PROBLEM Free ram is now: UNKNOWN on configtest-main i-000002dd output: NRPE: Unable to read output [06:38:07] RECOVERY Disk Space is now: OK on nagios 127.0.0.1 output: DISK OK [06:44:13] PROBLEM Total Processes is now: WARNING on aggregator-test1 i-000002bf output: PROCS WARNING: 199 processes [07:08:59] PROBLEM Total Processes is now: CRITICAL on aggregator-test1 i-000002bf output: PROCS CRITICAL: 206 processes [07:55:24] PROBLEM dpkg-check is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [07:55:24] PROBLEM Free ram is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [08:00:24] RECOVERY dpkg-check is now: OK on mobile-testing i-00000271 output: All packages OK [08:32:14] PROBLEM Puppet freshness is now: CRITICAL on wikistats-01 i-00000042 output: Puppet has not run in last 20 hours [08:49:07] PROBLEM Total Processes is now: WARNING on aggregator-test1 i-000002bf output: PROCS WARNING: 198 processes [09:06:27] RECOVERY dpkg-check is now: OK on aggregator2 i-000002c0 output: All packages OK [09:09:06] RECOVERY Disk Space is now: OK on aggregator2 i-000002c0 output: DISK OK [09:09:06] RECOVERY Current Users is now: OK on aggregator2 i-000002c0 output: USERS OK - 0 users currently logged in [09:09:06] RECOVERY Current Load is now: OK on aggregator2 i-000002c0 output: OK - load average: 1.37, 1.38, 1.31 [09:09:17] PROBLEM Free ram is now: WARNING on aggregator2 i-000002c0 output: Warning: 6% free memory [09:10:57] RECOVERY Total Processes is now: OK on aggregator2 i-000002c0 output: PROCS OK: 188 processes [09:11:08] RECOVERY SSH is now: OK on aggregator2 i-000002c0 output: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1 (protocol 2.0) [09:39:17] PROBLEM dpkg-check is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [09:44:17] RECOVERY dpkg-check is now: OK on mobile-testing i-00000271 output: All packages OK [09:54:17] PROBLEM Free ram is now: CRITICAL on aggregator2 i-000002c0 output: Critical: 5% free memory [10:14:17] PROBLEM Free ram is now: UNKNOWN on aggregator2 i-000002c0 output: Unknown [10:19:07] PROBLEM Total Processes is now: CRITICAL on aggregator2 i-000002c0 output: CHECK_NRPE: Error - Could not complete SSL handshake. [10:19:27] PROBLEM dpkg-check is now: UNKNOWN on aggregator2 i-000002c0 output: NRPE: Call to fork() failed [10:21:07] PROBLEM Disk Space is now: WARNING on nagios 127.0.0.1 output: DISK WARNING - free space: /home/dzahn 3615 MB (20% inode=75%): [10:21:57] PROBLEM Current Users is now: CRITICAL on aggregator2 i-000002c0 output: CHECK_NRPE: Error - Could not complete SSL handshake. [10:21:57] PROBLEM Current Load is now: CRITICAL on aggregator2 i-000002c0 output: CHECK_NRPE: Error - Could not complete SSL handshake. [10:22:57] PROBLEM Disk Space is now: CRITICAL on aggregator2 i-000002c0 output: CHECK_NRPE: Error - Could not complete SSL handshake. [10:24:07] PROBLEM SSH is now: CRITICAL on aggregator2 i-000002c0 output: Server answer: [10:24:27] PROBLEM dpkg-check is now: CRITICAL on aggregator2 i-000002c0 output: CHECK_NRPE: Error - Could not complete SSL handshake. [11:54:17] PROBLEM Free ram is now: CRITICAL on aggregator2 i-000002c0 output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:16:17] PROBLEM Current Load is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [12:21:17] RECOVERY Current Load is now: OK on mobile-testing i-00000271 output: OK - load average: 0.23, 0.27, 0.24 [12:59:23] PROBLEM dpkg-check is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [13:02:53] !log deployment-prep petrb: deploying 37852 [13:02:55] Logged the message, Master [15:41:04] PROBLEM Current Users is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [15:50:54] RECOVERY Current Users is now: OK on mobile-testing i-00000271 output: USERS OK - 0 users currently logged in [18:18:25] PROBLEM dpkg-check is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [18:28:15] RECOVERY dpkg-check is now: OK on mobile-testing i-00000271 output: All packages OK [18:33:15] PROBLEM Puppet freshness is now: CRITICAL on wikistats-01 i-00000042 output: Puppet has not run in last 20 hours [18:48:49] PROBLEM Free ram is now: WARNING on bots-3 i-000000e5 output: Warning: 19% free memory [20:02:29] PROBLEM Current Load is now: CRITICAL on mobile-testing i-00000271 output: CHECK_NRPE: Socket timeout after 10 seconds. [20:12:19] RECOVERY Current Load is now: OK on mobile-testing i-00000271 output: OK - load average: 0.16, 0.25, 0.22 [20:47:09] PROBLEM Current Load is now: WARNING on bots-cb i-0000009e output: WARNING - load average: 1.83, 14.50, 9.84 [21:02:09] RECOVERY Current Load is now: OK on bots-cb i-0000009e output: OK - load average: 0.30, 1.03, 3.91 [21:33:49] RECOVERY Free ram is now: OK on bots-3 i-000000e5 output: OK: 20% free memory [22:29:39] PROBLEM Free ram is now: WARNING on aggregator-test1 i-000002bf output: Warning: 19% free memory [23:02:59] RECOVERY Free ram is now: OK on bots-sql2 i-000000af output: OK: 21% free memory [23:29:19] PROBLEM Total Processes is now: CRITICAL on aggregator-test1 i-000002bf output: PROCS CRITICAL: 205 processes [23:34:19] PROBLEM Total Processes is now: WARNING on aggregator-test1 i-000002bf output: PROCS WARNING: 198 processes [23:36:49] PROBLEM Free ram is now: WARNING on bots-3 i-000000e5 output: Warning: 19% free memory