[04:13:18] 2014/02/09 04:13 CRIT z-dat-s1-b /sql DISK CRITICAL - free space: /sql 58622 MB (5% inode=99%): [04:18:18] 2014/02/09 04:18 WARN z-dat-s1-b /sql DISK WARNING - free space: /sql 91154 MB (9% inode=99%): [05:02:20] 2014/02/09 05:02 OK z-dat-s1-b /sql DISK OK - free space: /sql 125788 MB (12% inode=99%): [06:05:24] 2014/02/09 05:59 WARN z-dat-s1-b /sql DISK WARNING - free space: /sql 106421 MB (10% inode=99%): [08:59:34] 2014/02/09 08:52 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [09:01:34] 2014/02/09 09:00 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.004 second response time [10:08:39] 2014/02/09 10:01 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [10:11:38] 2014/02/09 10:10 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.005 second response time [12:21:46] 2014/02/09 12:20 ?? z-dat-s7-a /sql CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:46] 2014/02/09 12:21 ?? hyacinth / CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:46] 2014/02/09 12:20 ?? hyacinth /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:46] 2014/02/09 12:20 ?? hyacinth Environment IPMI CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:46] 2014/02/09 12:20 ?? hyacinth Load avg. CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:46] 2014/02/09 12:20 ?? z-dat-s3-a / CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:47] 2014/02/09 12:20 ?? z-dat-s3-a /sql CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:47] 2014/02/09 12:20 ?? z-dat-s3-a /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:48] 2014/02/09 12:20 ?? z-dat-s3-a Load avg. CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:48] 2014/02/09 12:20 CRIT z-dat-s3-a SSH CRITICAL - Socket timeout after 10 seconds [12:26:49] 2014/02/09 12:20 ?? z-dat-s4-a / CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:49] 2014/02/09 12:20 ?? z-dat-s4-a /sql CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:26:50] 2014/02/09 12:20 ?? z-dat-s4-a /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:27:47] 2014/02/09 12:20 ?? hyacinth RAID CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:27:47] 2014/02/09 12:20 CRIT hyacinth SMTP CRITICAL - Socket timeout after 10 seconds [12:27:47] 2014/02/09 12:20 CRIT hyacinth SSH CRITICAL - Socket timeout after 10 seconds [12:27:47] 2014/02/09 12:20 CRIT z-dat-s3-a MySQL Can't connect to MySQL server on 'z-dat-s3-a' (110) [12:27:47] 2014/02/09 12:20 CRIT z-dat-s3-a MySQL slave Can't connect to MySQL server on 'z-dat-s3-a' (110) [12:27:48] 2014/02/09 12:21 CRIT z-dat-s3-a SMTP CRITICAL - Socket timeout after 10 seconds [12:27:48] 2014/02/09 12:21 CRIT z-dat-s4-a SMTP CRITICAL - Socket timeout after 10 seconds [12:27:49] 2014/02/09 12:20 ?? z-dat-s6-a /sql CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:27:49] 2014/02/09 12:20 ?? z-dat-s6-a /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:27:50] 2014/02/09 12:20 ?? z-dat-s6-a Load avg. CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:27:50] 2014/02/09 12:20 CRIT z-dat-s6-a SSH CRITICAL - Socket timeout after 10 seconds [12:27:51] 2014/02/09 12:20 ?? z-dat-s7-a / CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:28:48] 2014/02/09 12:20 CRIT z-dat-s4-a MySQL (Service Check Timed Out) [12:28:48] 2014/02/09 12:21 CRIT z-dat-s6-a MySQL Can't connect to MySQL server on 'z-dat-s6-a' (110) [12:28:48] 2014/02/09 12:21 CRIT z-dat-s6-a MySQL slave Can't connect to MySQL server on 'z-dat-s6-a' (110) [12:28:48] 2014/02/09 12:21 CRIT z-dat-s7-a MySQL Can't connect to MySQL server on 'z-dat-s7-a' (110) [12:35:48] 2014/02/09 12:29 CRIT hyacinth NTP CRITICAL - Socket timeout after 10 seconds [12:35:48] 2014/02/09 12:34 WARN z-dat-s5-b MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2394 [12:36:48] 2014/02/09 12:35 OK z-dat-s5-b MySQL slave Uptime: 1391830 Threads: 3 Questions: 464331176 Slow queries: 867 Opens: 717984 Flush tables: 2 Open tables: 255 Queries per second avg: 333.611 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1438 [12:41:49] 2014/02/09 12:41 CRIT hyacinth / Connection refused or timed out [12:41:49] 2014/02/09 12:41 CRIT hyacinth /tmp Connection refused or timed out [12:41:49] 2014/02/09 12:41 CRIT hyacinth Environment IPMI Connection refused or timed out [12:41:49] 2014/02/09 12:41 CRIT hyacinth Load avg. Connection refused or timed out [12:41:49] 2014/02/09 12:41 CRIT hyacinth PING CRITICAL - Host Unreachable (hyacinth) [12:41:50] 2014/02/09 12:40 CRIT hyacinth RAID Connection refused or timed out [12:41:50] 2014/02/09 12:41 CRIT z-dat-s3-a / Connection refused or timed out [12:41:51] 2014/02/09 12:41 CRIT z-dat-s3-a /sql Connection refused or timed out [12:41:51] 2014/02/09 12:41 CRIT z-dat-s3-a /tmp Connection refused or timed out [12:41:52] 2014/02/09 12:40 CRIT z-dat-s3-a Load avg. Connection refused or timed out [12:41:52] 2014/02/09 12:40 CRIT z-dat-s3-a PING CRITICAL - Host Unreachable (z-dat-s3-a) [12:41:53] 2014/02/09 12:41 CRIT z-dat-s4-a / Connection refused or timed out [12:42:50] 2014/02/09 12:41 CRIT z-dat-s4-a PING CRITICAL - Host Unreachable (z-dat-s4-a) [12:43:50] 2014/02/09 12:43 OK hyacinth PING PING OK - Packet loss = 0%, RTA = 0.11 ms [12:46:50] 2014/02/09 12:46 CRIT z-dat-s3-a CRITICAL - Host Unreachable (z-dat-s3-a) [12:46:50] 2014/02/09 12:46 CRIT z-dat-s4-a CRITICAL - Host Unreachable (z-dat-s4-a) [12:46:50] 2014/02/09 12:46 CRIT z-dat-s7-a CRITICAL - Host Unreachable (z-dat-s7-a) [12:47:50] 2014/02/09 12:47 OK hyacinth / DISK OK - free space: / 8591 MB (28% inode=86%): [12:47:50] 2014/02/09 12:47 OK hyacinth /tmp DISK OK - free space: /tmp 62546 MB (100% inode=99%): [12:47:50] 2014/02/09 12:47 OK hyacinth Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [12:47:50] 2014/02/09 12:47 OK hyacinth Load avg. OK - load average: 0.27, 0.40, 0.23 [12:47:50] 2014/02/09 12:47 OK hyacinth NTP NTP OK: Offset -0.027629 secs [12:47:51] 2014/02/09 12:47 OK hyacinth RAID OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [12:48:50] 2014/02/09 12:48 OK z-dat-s3-a PING OK - Packet loss = 0%, RTA = 0.27 ms [12:48:50] 2014/02/09 12:48 OK z-dat-s4-a PING OK - Packet loss = 0%, RTA = 0.23 ms [12:48:50] 2014/02/09 12:48 OK z-dat-s7-a PING OK - Packet loss = 0%, RTA = 0.34 ms [12:48:50] 2014/02/09 12:47 OK hyacinth SMTP SMTP OK - 0.123 sec. response time [12:48:50] 2014/02/09 12:47 OK hyacinth SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [12:48:51] 2014/02/09 12:48 OK z-dat-s3-a / DISK OK - free space: / 8574 MB (28% inode=86%): [12:48:51] 2014/02/09 12:48 OK z-dat-s3-a /sql DISK OK - free space: /sql 126316 MB (13% inode=98%): [12:48:52] 2014/02/09 12:48 OK z-dat-s3-a /tmp DISK OK - free space: /tmp 48970 MB (99% inode=99%): [12:48:52] 2014/02/09 12:48 OK z-dat-s3-a Load avg. OK - load average: 1.36, 0.65, 0.33 [12:48:53] 2014/02/09 12:48 OK z-dat-s3-a SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [12:48:53] 2014/02/09 12:48 OK z-dat-s4-a / DISK OK - free space: / 8574 MB (28% inode=86%): [12:48:54] 2014/02/09 12:48 OK z-dat-s4-a /sql DISK OK - free space: /sql 178380 MB (25% inode=99%): [12:49:51] 2014/02/09 12:48 OK z-dat-s3-a PING PING OK - Packet loss = 0%, RTA = 0.32 ms [12:49:51] 2014/02/09 12:49 OK z-dat-s3-a SMTP SMTP OK - 0.110 sec. response time [12:49:51] 2014/02/09 12:48 OK z-dat-s4-a PING PING OK - Packet loss = 0%, RTA = 0.25 ms [12:49:51] 2014/02/09 12:49 OK z-dat-s4-a SMTP SMTP OK - 0.129 sec. response time [12:49:51] 2014/02/09 12:48 OK z-dat-s6-a /tmp DISK OK - free space: /tmp 48921 MB (99% inode=99%): [12:49:52] 2014/02/09 12:48 OK z-dat-s6-a Load avg. OK - load average: 1.28, 0.68, 0.34 [12:49:52] 2014/02/09 12:48 WARN z-dat-s7-a /sql DISK WARNING - free space: /sql 64944 MB (9% inode=99%): [12:49:53] 2014/02/09 12:48 OK z-dat-s7-a /tmp DISK OK - free space: /tmp 48901 MB (99% inode=99%): [12:49:53] 2014/02/09 12:49 OK z-dat-s7-a Load avg. OK - load average: 1.19, 0.67, 0.34 [12:50:51] 2014/02/09 12:50 OK z-dat-s6-a MySQL Uptime: 131 Threads: 4 Questions: 2594 Slow queries: 0 Opens: 109 Flush tables: 1 Open tables: 97 Queries per second avg: 19.801 [12:50:51] 2014/02/09 12:50 WARN z-dat-s6-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2990 [12:52:51] 2014/02/09 12:52 OK z-dat-s7-a MySQL Uptime: 249 Threads: 2 Questions: 279 Slow queries: 0 Opens: 72 Flush tables: 1 Open tables: 63 Queries per second avg: 1.120 [12:52:51] 2014/02/09 12:52 WARN z-dat-s7-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3176 [12:56:51] 2014/02/09 12:56 OK z-dat-s6-a MySQL slave Uptime: 500 Threads: 14 Questions: 118495 Slow queries: 21 Opens: 195 Flush tables: 1 Open tables: 178 Queries per second avg: 236.990 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1627 [12:58:51] 2014/02/09 12:58 OK z-dat-s4-a MySQL Uptime: 629 Threads: 1 Questions: 20300 Slow queries: 0 Opens: 49 Flush tables: 1 Open tables: 42 Queries per second avg: 32.273 [13:04:51] 2014/02/09 12:58 CRIT z-dat-s6-a MySQL slave Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [13:07:51] 2014/02/09 13:07 OK z-dat-s7-a MySQL slave Uptime: 1169 Threads: 4 Questions: 188115 Slow queries: 40 Opens: 598 Flush tables: 1 Open tables: 576 Queries per second avg: 160.919 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1787 [13:08:51] 2014/02/09 13:08 OK z-dat-s3-a MySQL Uptime: 1199 Threads: 12 Questions: 1190 Slow queries: 0 Opens: 468 Flush tables: 1 Open tables: 453 Queries per second avg: 0.992 [13:14:51] 2014/02/09 13:08 CRIT z-dat-s7-a MySQL slave Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [14:39:58] 2014/02/09 14:34 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [14:42:58] 2014/02/09 14:39 CRIT ortelius /tmp DISK CRITICAL - free space: /tmp 130 MB (4% inode=99%): [14:42:58] 2014/02/09 14:42 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [14:43:58] 2014/02/09 14:42 ?? ortelius /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [14:44:58] 2014/02/09 14:44 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [14:45:58] 2014/02/09 14:45 CRIT ortelius Sun Grid Engine execd Connection refused by host [14:46:59] 2014/02/09 14:46 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [14:47:59] 2014/02/09 14:46 CRIT ortelius /tmp DISK CRITICAL - free space: /tmp 276 MB (9% inode=99%): [14:50:01] 2014/02/09 14:48 WARN ortelius /tmp DISK WARNING - free space: /tmp 420 MB (14% inode=99%): [14:52:01] 2014/02/09 14:50 OK ortelius /tmp DISK OK - free space: /tmp 2223 MB (46% inode=99%): [14:52:01] 2014/02/09 14:50 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.006 second response time [15:18:03] 2014/02/09 15:11 CRIT ortelius /tmp DISK CRITICAL - free space: /tmp 132 MB (4% inode=99%): [15:20:03] 2014/02/09 15:18 ?? ortelius /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [15:20:03] 2014/02/09 15:12 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [15:21:03] 2014/02/09 15:19 CRIT ortelius /tmp DISK CRITICAL - free space: /tmp 137 MB (5% inode=99%): [15:22:03] 2014/02/09 15:20 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.005 second response time [15:23:04] 2014/02/09 15:21 WARN ortelius /tmp DISK WARNING - free space: /tmp 343 MB (11% inode=99%): [15:25:03] 2014/02/09 15:23 CRIT ortelius /tmp DISK CRITICAL - free space: /tmp 174 MB (6% inode=99%): [15:29:03] 2014/02/09 15:27 WARN ortelius /tmp DISK WARNING - free space: /tmp 367 MB (12% inode=99%): [15:32:04] 2014/02/09 15:30 OK ortelius /tmp DISK OK - free space: /tmp 1113 MB (30% inode=99%): [15:41:03] 2014/02/09 15:40 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [15:43:03] 2014/02/09 15:35 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 6.514 second response time [15:48:04] 2014/02/09 15:46 ?? ortelius /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [15:48:05] 2014/02/09 15:47 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [15:49:05] 2014/02/09 15:47 CRIT ortelius /tmp DISK CRITICAL - free space: /tmp 303 MB (10% inode=99%): [15:49:06] 2014/02/09 15:48 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [15:49:06] 2014/02/09 15:47 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.005 second response time [15:50:06] 2014/02/09 15:48 ?? ortelius /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [15:51:06] 2014/02/09 15:50 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [15:52:06] 2014/02/09 15:50 WARN ortelius /tmp DISK WARNING - free space: /tmp 643 MB (19% inode=99%): [15:55:06] 2014/02/09 15:53 OK ortelius /tmp DISK OK - free space: /tmp 687 MB (21% inode=99%): [16:05:06] 2014/02/09 16:04 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [16:06:06] 2014/02/09 16:05 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [16:08:06] 2014/02/09 16:02 CRIT ortelius /tmp DISK CRITICAL - free space: /tmp 127 MB (4% inode=99%): [16:09:06] 2014/02/09 16:07 OK ortelius /tmp DISK OK - free space: /tmp 3133 MB (98% inode=99%): [17:19:11] 2014/02/09 17:11 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [17:21:11] 2014/02/09 17:20 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.002 second response time [17:31:12] 2014/02/09 17:30 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [17:32:12] 2014/02/09 17:31 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [17:33:12] 2014/02/09 17:32 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [17:35:13] 2014/02/09 17:34 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [17:35:13] 2014/02/09 17:28 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [17:37:13] 2014/02/09 17:36 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [17:38:13] 2014/02/09 17:37 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [17:40:14] 2014/02/09 17:39 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [17:41:15] 2014/02/09 17:40 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [17:41:15] 2014/02/09 17:40 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.008 second response time [17:46:15] 2014/02/09 17:44 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [17:48:15] 2014/02/09 17:47 CRIT ortelius Sun Grid Engine execd Connection refused by host [17:52:16] 2014/02/09 17:47 CRIT ortelius / Connection refused by host [17:52:16] 2014/02/09 17:48 CRIT ortelius Load avg. Connection refused by host [17:53:16] 2014/02/09 17:48 CRIT ortelius Environment IPMI Connection refused by host [17:54:16] 2014/02/09 17:47 CRIT ortelius /tmp Connection refused by host [17:54:16] 2014/02/09 17:48 CRIT ortelius DiskSuite Connection refused by host [18:19:17] 2014/02/09 18:11 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [18:21:17] 2014/02/09 18:20 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.002 second response time [18:48:18] 2014/02/09 18:41 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [18:52:18] 2014/02/09 18:51 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.010 second response time [18:53:18] 2014/02/09 18:52 OK ortelius / DISK OK - free space: / 13090 MB (43% inode=91%): [18:53:18] 2014/02/09 18:52 OK ortelius /tmp DISK OK - free space: /tmp 1758 MB (96% inode=99%): [18:53:19] 2014/02/09 18:53 OK ortelius DiskSuite OK - No disk failures detected [18:53:19] 2014/02/09 18:53 OK ortelius Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [18:53:19] 2014/02/09 18:52 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [18:54:18] 2014/02/09 18:53 OK ortelius Load avg. OK - load average: 1.67, 0.83, 0.62 [19:03:19] 2014/02/09 18:56 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [19:04:19] 2014/02/09 19:03 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [19:06:19] 2014/02/09 19:05 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [19:11:21] 2014/02/09 19:10 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.010 second response time [19:14:21] 2014/02/09 19:13 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [19:15:22] 2014/02/09 19:14 CRIT ortelius Sun Grid Engine execd NRPE: Unable to read output [19:16:22] 2014/02/09 19:15 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [19:22:23] 2014/02/09 19:21 ?? ortelius Sun Grid Engine execd NRPE: Unable to read output [19:23:24] 2014/02/09 19:22 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [19:24:24] 2014/02/09 19:23 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [19:25:24] 2014/02/09 19:24 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [19:25:24] 2014/02/09 19:18 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [19:27:24] 2014/02/09 19:26 CRIT ortelius Sun Grid Engine execd NRPE: Unable to read output [19:28:24] 2014/02/09 19:27 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [19:30:24] 2014/02/09 19:29 CRIT ortelius Sun Grid Engine execd Connection refused by host [19:31:24] 2014/02/09 19:30 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.029 second response time [19:35:24] 2014/02/09 19:28 CRIT ortelius /tmp Connection refused by host [19:35:25] 2014/02/09 19:29 CRIT ortelius DiskSuite Connection refused by host [19:35:25] 2014/02/09 19:29 CRIT ortelius Environment IPMI Connection refused by host [19:36:24] 2014/02/09 19:29 CRIT ortelius / Connection refused by host [19:36:25] 2014/02/09 19:29 CRIT ortelius Load avg. Connection refused by host [19:44:26] 2014/02/09 19:37 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [19:51:26] 2014/02/09 19:50 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.003 second response time [19:58:27] 2014/02/09 19:51 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [20:02:27] 2014/02/09 20:01 OK ortelius / DISK OK - free space: / 13096 MB (43% inode=91%): [20:02:27] 2014/02/09 20:01 OK ortelius /tmp DISK OK - free space: /tmp 933 MB (93% inode=99%): [20:02:27] 2014/02/09 20:02 OK ortelius DiskSuite OK - No disk failures detected [20:02:27] 2014/02/09 20:01 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [20:02:27] 2014/02/09 20:01 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.002 second response time [20:03:27] 2014/02/09 20:02 OK ortelius Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [20:03:27] 2014/02/09 20:02 OK ortelius Load avg. OK - load average: 0.86, 0.62, 0.50 [20:17:28] 2014/02/09 20:16 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [20:18:28] 2014/02/09 20:17 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [20:18:28] 2014/02/09 20:11 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [20:19:29] 2014/02/09 20:18 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [20:20:29] 2014/02/09 20:19 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [20:21:29] 2014/02/09 20:20 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [20:23:29] 2014/02/09 20:22 CRIT ortelius Sun Grid Engine execd Connection refused by host [20:24:29] 2014/02/09 20:23 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [20:26:29] 2014/02/09 20:25 ?? ortelius Sun Grid Engine execd CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [20:28:30] 2014/02/09 20:27 CRIT ortelius Sun Grid Engine execd Connection refused by host [20:30:30] 2014/02/09 20:29 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [20:30:30] 2014/02/09 20:29 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.003 second response time [20:52:30] 2014/02/09 20:51 CRIT ortelius Sun Grid Engine execd Connection refused by host [20:55:31] 2014/02/09 20:54 CRIT ortelius / Connection refused or timed out [20:55:31] 2014/02/09 20:54 CRIT ortelius /tmp Connection refused or timed out [20:55:31] 2014/02/09 20:55 CRIT ortelius DiskSuite Connection refused or timed out [20:55:31] 2014/02/09 20:54 CRIT ortelius Environment IPMI Connection refused or timed out [20:55:31] 2014/02/09 20:54 CRIT ortelius Load avg. Connection refused or timed out [20:55:32] 2014/02/09 20:54 CRIT ortelius NTP CRITICAL - Socket timeout after 10 seconds [20:55:32] 2014/02/09 20:54 CRIT ortelius PING CRITICAL - Host Unreachable (ortelius) [20:55:33] 2014/02/09 20:54 CRIT ortelius SMTP No route to host [20:55:33] 2014/02/09 20:54 CRIT ortelius SSH No route to host [20:55:34] 2014/02/09 20:54 CRIT ortelius toolserver.org HTTP No route to host [20:57:31] 2014/02/09 20:56 OK ortelius / DISK OK - free space: / 13088 MB (43% inode=91%): [20:57:31] 2014/02/09 20:56 OK ortelius /tmp DISK OK - free space: /tmp 26162 MB (99% inode=99%): [20:57:31] 2014/02/09 20:57 OK ortelius DiskSuite OK - No disk failures detected [20:57:31] 2014/02/09 20:57 OK ortelius Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [20:57:31] 2014/02/09 20:57 OK ortelius Load avg. OK - load average: 1.95, 0.90, 0.35 [20:57:32] 2014/02/09 20:57 OK ortelius NTP NTP OK: Offset -0.055793 secs [20:57:32] 2014/02/09 20:56 OK ortelius PING PING OK - Packet loss = 0%, RTA = 0.30 ms [20:57:33] 2014/02/09 20:56 OK ortelius SMTP SMTP OK - 0.256 sec. response time [20:57:33] 2014/02/09 20:56 OK ortelius SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [20:57:34] 2014/02/09 20:56 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [20:57:34] 2014/02/09 20:56 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.013 second response time [22:40:38] 2014/02/09 22:40 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3497 [22:45:39] 2014/02/09 22:45 OK z-dat-s3-a MySQL slave Uptime: 8547 Threads: 22 Questions: 5024331 Slow queries: 135 Opens: 71334 Flush tables: 1 Open tables: 16384 Queries per second avg: 587.847 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1503