[00:01:14] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3135.000000 [00:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:15] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:15] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:02:13] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:02:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:02:14] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:03:14] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1129.000000 [00:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:07:34] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 30322.000000 [00:10:04] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 108191.000000 [00:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 697073.000000 [00:35:53] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2934 [00:37:54] MySQL slave on z-dat-s2-b is OK: Uptime: 17034 Threads: 10 Questions: 7074891 Slow queries: 242 Opens: 157617 Flush tables: 1 Open tables: 256 Queries per second avg: 415.339 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1768 [00:42:24] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [00:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [00:49:34] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [00:49:44] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:50:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [00:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [00:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [00:50:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [00:51:04] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:51:04] RAID on thyme is UNKNOWN: NRPE: Unable to read output [00:51:04] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:05] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:05] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:14] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:23] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:51:33] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:51:44] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 935745.000000 [00:51:54] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:51:55] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [00:51:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 61063 MB (10% inode=99%): [00:51:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [00:52:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:52:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 228325.000000 [00:52:24] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:52:24] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 630857.000000 [00:52:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [00:52:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:52:54] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [00:52:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295824 MB (5% inode=62%): [00:52:55] APT on sage is WARNING: APT WARNING: 3 packages available for upgrade (0 critical updates). [00:53:23] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 716509.000000 [00:57:24] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 26365 [00:59:24] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [01:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:02:14] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:02:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:02:14] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:02:14] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:02:15] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:03:04] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2050.000000 [01:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:07:34] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 23683.000000 [01:10:04] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 111791.000000 [01:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 693523.000000 [01:42:23] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [01:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [01:49:34] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [01:49:44] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:50:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [01:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [01:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [01:50:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [01:51:03] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:51:05] RAID on thyme is UNKNOWN: NRPE: Unable to read output [01:51:05] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:05] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:05] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:13] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:51:44] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 939266.000000 [01:51:54] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 60107 MB (9% inode=99%): [01:51:54] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:51:54] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [01:51:54] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [01:52:05] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:52:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 222492.000000 [01:52:24] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:52:24] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 634457.000000 [01:52:53] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [01:52:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:52:55] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [01:52:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295684 MB (5% inode=62%): [01:52:55] APT on sage is WARNING: APT WARNING: 3 packages available for upgrade (0 critical updates). [01:53:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 719055.000000 [01:55:04] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3622.000000 [01:57:24] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 9732 [01:59:23] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [02:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:02:14] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:02:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:02:14] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:02:14] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:02:14] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:05:04] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3599.000000 [02:07:34] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5730.000000 [02:10:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 115391.000000 [02:12:24] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3346 [02:12:34] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3222.000000 [02:15:24] MySQL slave on z-dat-s5-b is OK: Uptime: 22797 Threads: 5 Questions: 131560659 Slow queries: 192 Opens: 8731 Flush tables: 1 Open tables: 256 Queries per second avg: 5770.963 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1741 [02:15:34] wikidata replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1603.000000 [02:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 688659.000000 [02:37:24] MySQL on z-dat-s1-b is OK: Uptime: 26619 Threads: 5 Questions: 2683 Slow queries: 0 Opens: 73 Flush tables: 1 Open tables: 66 Queries per second avg: 0.100 [02:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [02:49:34] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [02:49:43] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:50:23] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [02:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [02:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [02:50:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [02:51:04] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:51:04] RAID on thyme is UNKNOWN: NRPE: Unable to read output [02:51:04] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:04] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:04] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:14] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:51:33] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:51:43] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 942688.000000 [02:51:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 61009 MB (9% inode=99%): [02:51:55] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [02:51:55] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:51:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:03] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [02:52:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:52:30] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:52:30] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:52:30] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 217719.000000 [02:52:30] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:52:30] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 638057.000000 [02:52:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [02:52:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:52:54] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [02:52:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295542 MB (5% inode=62%): [02:52:54] APT on sage is WARNING: APT WARNING: 3 packages available for upgrade (0 critical updates). [02:53:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 721545.000000 [02:59:24] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 76810 [03:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:02:13] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:02:14] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:02:15] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:02:15] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:02:15] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:05:04] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2634.000000 [03:10:04] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 118991.000000 [03:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 683626.000000 [03:29:03] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1773.000000 [03:34:54] Load avg. on ptolemy is WARNING: WARNING - load average: 14.76, 15.21, 10.87 [03:35:54] Load avg. on ptolemy is OK: OK - load average: 11.79, 14.30, 10.83 [03:37:04] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2169.000000 [03:45:04] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1735.000000 [03:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [03:49:34] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [03:49:43] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:50:23] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [03:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [03:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [03:50:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [03:51:04] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:51:04] RAID on thyme is UNKNOWN: NRPE: Unable to read output [03:51:04] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:04] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:04] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:14] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:51:35] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:51:43] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:55] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 945751.000000 [03:51:55] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [03:51:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 60503 MB (9% inode=99%): [03:51:55] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:51:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:52:03] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [03:52:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:52:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:52:27] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:52:27] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 213014.000000 [03:52:27] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:52:27] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 641657.000000 [03:52:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [03:52:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:52:54] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [03:52:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295366 MB (5% inode=62%): [03:52:54] APT on sage is WARNING: APT WARNING: 3 packages available for upgrade (0 critical updates). [03:53:23] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 724733.000000 [03:56:28] [[Special:Log/newusers]] create 10 * Shubhamashokgandhi * (New user account) [03:59:24] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 65475 [04:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:15] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:15] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:15] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:15] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:10:04] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 122591.000000 [04:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 679109.000000 [04:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [04:49:33] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [04:49:44] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:50:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [04:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [04:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [04:50:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [04:51:04] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:51:05] RAID on thyme is UNKNOWN: NRPE: Unable to read output [04:51:05] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:05] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:05] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:13] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:23] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:51:44] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 948765.000000 [04:51:54] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 60411 MB (9% inode=99%): [04:51:54] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [04:51:54] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:51:54] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [04:52:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:52:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 210482.000000 [04:52:24] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:52:24] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 645257.000000 [04:52:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [04:52:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:52:55] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [04:52:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295179 MB (5% inode=62%): [04:52:55] APT on sage is WARNING: APT WARNING: 3 packages available for upgrade (0 critical updates). [04:53:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 727721.000000 [04:59:24] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 54507 [05:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:15] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:15] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:15] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:15] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:15] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:10:23] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 126211.000000 [05:16:54] Free Memory on damiana is WARNING: WARNING - 6.2% (516484 kB) free! [05:22:54] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (422384 kB) free! [05:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 675031.000000 [05:29:54] Free Memory on damiana is WARNING: WARNING - 5.4% (453796 kB) free! [05:34:50] [[Category:Tools]] ! 10https://wiki.toolserver.org/w/index.php?diff=7909&oldid=7908&rcid=21767 * Jasper Deng * (-16447) (rvt) [05:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [05:49:34] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [05:49:44] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:50:23] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [05:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [05:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [05:50:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:51:04] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:51:04] RAID on thyme is UNKNOWN: NRPE: Unable to read output [05:51:14] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:51:43] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 950952.000000 [05:51:55] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [05:51:55] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:51:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 60268 MB (9% inode=99%): [05:51:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [05:52:04] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:05] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:05] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:05] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 208424.000000 [05:52:24] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:24] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 648857.000000 [05:52:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [05:52:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:52:54] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [05:52:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295012 MB (5% inode=62%): [05:52:54] APT on sage is WARNING: APT WARNING: 3 packages available for upgrade (0 critical updates). [05:53:23] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 730715.000000 [05:59:24] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 47800 [06:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:02:13] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:02:54] Free Memory on damiana is CRITICAL: CRITICAL - 4.5% (374356 kB) free! [06:03:14] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:14] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:14] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:10:24] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 129811.000000 [06:15:54] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [06:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 672085.000000 [06:32:55] Free Memory on damiana is WARNING: WARNING - 5.1% (427024 kB) free! [06:33:54] Free Memory on damiana is CRITICAL: CRITICAL - 4.6% (382364 kB) free! [06:34:55] Free Memory on damiana is WARNING: WARNING - 5.5% (462244 kB) free! [06:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [06:49:34] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [06:50:04] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:50:23] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:50:33] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [06:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [06:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [06:50:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [06:51:04] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:51:05] RAID on thyme is UNKNOWN: NRPE: Unable to read output [06:51:14] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:51:43] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 953750.000000 [06:51:54] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:51:55] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [06:51:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 60077 MB (9% inode=99%): [06:51:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [06:52:05] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:05] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:05] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:05] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:52:15] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 205727.000000 [06:52:24] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:24] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:52:33] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 650869.000000 [06:52:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [06:52:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:52:55] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [06:52:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 294774 MB (5% inode=62%): [06:53:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 733993.000000 [06:54:54] Free Memory on damiana is CRITICAL: CRITICAL - 2.7% (225804 kB) free! [06:59:23] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 40171 [07:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:02:14] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:14] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:14] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:14] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:10:24] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 124483.000000 [07:15:55] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [07:24:23] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 669436.000000 [07:46:34] aliasd on nightshade is CRITICAL: Connection refused [07:48:34] aliasd on nightshade is OK: TCP OK - 0.012 second response time on port 984 [500 Not found.] [07:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [07:49:33] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [07:50:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:50:43] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:50:43] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [07:50:55] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [07:50:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [07:50:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [07:51:03] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:51:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:51:05] RAID on thyme is UNKNOWN: NRPE: Unable to read output [07:51:13] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:51:44] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 955412.000000 [07:51:54] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [07:51:54] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 59877 MB (9% inode=99%): [07:51:54] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:51:54] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [07:52:04] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:52:04] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:52:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 203742.000000 [07:52:24] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:52:24] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 649523.000000 [07:52:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [07:52:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:52:55] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [07:53:05] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:53:05] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:53:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 737004.000000 [07:53:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 294585 MB (5% inode=62%): [07:54:55] Free Memory on damiana is CRITICAL: CRITICAL - 2.8% (236924 kB) free! [07:59:24] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 33735 [08:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:02:14] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:14] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:14] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:14] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:10:23] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 109865.000000 [08:15:54] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [08:19:43] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 668225.000000 [08:29:04] Platonides * Re: [Toolserver-l] Wikidata tables [08:39:04] Magnus Manske * Re: [Toolserver-l] Wikidata tables [08:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [08:49:34] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [08:50:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [08:50:54] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [08:50:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [08:50:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [08:51:04] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:51:04] RAID on thyme is UNKNOWN: NRPE: Unable to read output [08:51:14] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:51:43] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 956554.000000 [08:51:55] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [08:51:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 59551 MB (9% inode=99%): [08:51:55] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:51:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [08:52:05] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:52:05] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:52:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 201783.000000 [08:52:24] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:52:24] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 646006.000000 [08:52:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [08:52:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:52:54] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [08:53:04] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:53:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:53:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 739564.000000 [08:53:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 294070 MB (5% inode=62%): [08:54:04] Kolossos * Re: [Toolserver-l] Wikidata tables [08:54:54] Free Memory on damiana is CRITICAL: CRITICAL - 2.4% (198408 kB) free! [08:59:24] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 27260 [09:01:16] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:02:14] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:15] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:15] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:15] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:15] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:09:04] Kolossos * Re: [Toolserver-l] Wikidata tables [09:10:24] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 88765.000000 [09:15:04] Patricia Pintilie * Re: [Toolserver-l] Wikidata tables [09:15:55] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [09:19:43] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 666621.000000 [09:42:05] Platonides * Re: [Toolserver-l] Wikidata tables [09:42:05] Patricia Pintilie * Re: [Toolserver-l] Wikidata tables [09:42:42] wow, that was a fast reply... [09:43:10] this is looking more and more like spam... [09:49:04] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [09:49:33] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [09:50:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:50:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:50:34] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:50:34] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:50:44] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:50:44] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:50:44] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:50:54] / on thyme is UNKNOWN: NRPE: Unable to read output [09:50:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [09:50:55] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [09:50:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [09:51:03] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:51:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:51:05] RAID on thyme is UNKNOWN: NRPE: Unable to read output [09:51:13] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:51:24] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:51:24] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:51:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:51:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:51:44] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:51:54] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 956863.000000 [09:51:54] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 59311 MB (9% inode=99%): [09:51:54] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:51:55] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [09:51:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:52:04] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [09:52:04] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:52:04] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:52:14] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:52:14] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:52:24] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:52:24] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:52:25] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 198591.000000 [09:52:25] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:52:25] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:52:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 637996.000000 [09:52:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [09:52:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:52:55] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [09:53:04] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:53:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:53:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 742303.000000 [09:53:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 293742 MB (5% inode=62%): [09:54:54] Free Memory on damiana is CRITICAL: CRITICAL - 2.5% (206132 kB) free! [09:59:23] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 22037 [10:01:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:02:14] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:03:14] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:04:14] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:04:14] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:04:15] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:04:15] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:10:34] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 74099.000000 [10:15:55] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [10:19:44] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:24:24] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 664761.000000 [10:26:34] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 20018 MB (2% inode=99%): [10:28:19] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [10:28:30] FC 0/10 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [10:28:30] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [10:28:31] FC 0/18 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [10:28:31] FC 0/20 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [10:28:32] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [10:28:32] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [10:31:25] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [10:43:14] Silke Meyer * [Toolserver-announce] Draft online: Roadmap for the setup of Tool Labs and the migration from Toolserver [10:48:37] can't connect to willow or nightshade, existing connections are frozen [10:49:04] looks like http is down too [10:52:16] sigh [10:52:35] very unstable recently [10:53:21] let's see, tsnag is back [10:54:04] nightshade unfrozen, willow not [11:25:53] [[Special:Log/newusers]] create 10 * Milos98 * (New user account) [11:29:51] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:29:51] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:29:51] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:29:51] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3675.000000 [11:29:51] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 635692.000000 [11:29:52] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 960376.000000 [11:29:52] RAID on thyme is UNKNOWN: NRPE: Unable to read output [11:29:53] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:29:53] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:29:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:29:58] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 199740.000000 [11:29:58] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58944 MB (9% inode=99%): [11:29:58] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:29:58] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:29:59] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:29:59] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:30:00] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [11:30:01] / on thyme is UNKNOWN: NRPE: Unable to read output [11:30:05] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:30:05] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:30:05] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:30:06] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [11:33:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 83113 MB (13% inode=99%): [11:35:19] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:35:19] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [11:35:19] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [11:35:27] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:27] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:27] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:27] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:35:27] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:28] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:48] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:48] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:48] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:48] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:48] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:48] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:35:58] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:36:19] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [11:36:19] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [11:36:19] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 293075 MB (5% inode=62%): [11:36:19] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [11:36:19] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [11:36:27] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [11:39:49] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 668087.000000 [11:41:05] Marlen Caemmerer * [Toolserver-announce] Maintenance: moving data of s1-user / rosemary [11:44:48] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 474733.000000 [11:44:58] /sql on z-dat-s7-a is CRITICAL: DISK CRITICAL - free space: /sql 21807 MB (4% inode=99%): [12:03:04] Marlen Caemmerer * [Toolserver-announce] Maintenance: more disk space for s7 [12:07:04] Marlen Caemmerer * Re: [Toolserver-announce] Maintenance: more disk space for s7 [12:07:27] SSH on yarrow is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:07:47] Sun Grid Engine execd on yarrow is UNKNOWN: Execution timeout exceeded [12:29:17] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [12:29:19] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [12:29:27] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [12:29:27] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:29:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 751012.000000 [12:29:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:29:27] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:29:37] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [12:29:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:29:38] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 52819.000000 [12:29:47] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7275.000000 [12:29:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 627252.000000 [12:29:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:29:48] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:29:57] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:29:57] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:29:57] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:05] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:05] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:36] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:37] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:37] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:37] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:30:37] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:30:47] RAID on thyme is UNKNOWN: NRPE: Unable to read output [12:30:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 960734.000000 [12:30:58] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 195762.000000 [12:30:58] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:58] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:30:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:30:58] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:30:58] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:30:58] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [12:30:59] / on thyme is UNKNOWN: NRPE: Unable to read output [12:31:06] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [12:31:06] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:33:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 78293 MB (12% inode=99%): [12:34:25] @replag [12:34:26] jimmyxu: s1-rr-a: 2h 6m 3s [-]; s1-rr-a-wd: 4d 13h 28m 15s [-]; s1-user-wd: 1w 4d 2h 52m 40s [-]; s2-rr: 2h 6m 5s [-]; s2-user: 2h 6m 5s [-]; s2-user-c: error; s2-user-wd: 1w 6h 11m 6s [-]; s3-user-wd: 5d 21h 17m 35s [-] [12:34:27] jimmyxu: s4-user-wd: 1w 1d 16h 41m 31s [-]; s5-rr-a-wd: 2d 6h 18m 8s [-]; s5-user: 2h 6m 4s [-]; s5-user-c: 14h 24m 14s [-]; s6-user-wd: 1w 16h 43m 9s [-]; s7-user-wd: 5d 10h 28m 9s [-] [12:35:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:35:58] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:35:58] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:35:58] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:35:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:35:58] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:35:58] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:36:18] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:36:18] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [12:36:18] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [12:36:26] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:36:26] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:36:26] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:36:26] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:36:26] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:36:26] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:36:36] Sun Grid Engine execd on yarrow is OK: Host and Queues Ok [12:37:19] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [12:37:19] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [12:37:19] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [12:37:19] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 292744 MB (5% inode=62%): [12:37:19] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [12:37:19] SSH on yarrow is OK: SSH OK - OpenSSH_5.5p1 Debian-6+squeeze3 (protocol 2.0) [12:37:26] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [12:40:46] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 664590.000000 [12:44:47] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 468827.000000 [12:56:04] Marlen Caemmerer * [Toolserver-announce] Maintenance cassia s5-rr [13:00:06] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:29:37] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [13:29:37] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:29:37] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 27087.000000 [13:29:48] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10875.000000 [13:29:48] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 626360.000000 [13:29:48] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:29:48] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:29:57] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:29:57] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:29:58] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:05] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:05] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:19] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [13:30:19] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [13:30:27] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [13:30:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 754036.000000 [13:30:27] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:30:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:28] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:37] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:37] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:38] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:30:38] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:30:38] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:30:46] RAID on thyme is UNKNOWN: NRPE: Unable to read output [13:30:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 961331.000000 [13:30:57] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 192240.000000 [13:30:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:31:05] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [13:31:05] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:31:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:31:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:31:57] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:31:57] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:31:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:31:57] / on thyme is UNKNOWN: NRPE: Unable to read output [13:34:56] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 77950 MB (12% inode=99%): [13:35:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:35:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:35:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:35:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:35:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:35:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:36:18] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:36:18] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [13:36:18] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [13:36:26] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:36:26] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:36:26] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:36:26] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:36:26] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:36:27] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:36:57] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:37:18] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [13:37:18] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [13:37:18] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [13:37:18] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 292455 MB (5% inode=62%): [13:37:18] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [13:37:27] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [13:40:47] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 656422.000000 [13:44:47] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 461699.000000 [14:08:36] s4 replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3066.000000 [14:10:37] s4 replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1452.000000 [14:29:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:29:58] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:30:06] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:27] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [14:30:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 757175.000000 [14:30:27] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:30:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:27] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:36] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [14:30:36] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:30:46] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14535.000000 [14:30:46] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 621651.000000 [14:30:47] RAID on thyme is UNKNOWN: NRPE: Unable to read output [14:30:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 962330.000000 [14:30:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:57] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 189731.000000 [14:30:57] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:58] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:30:58] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:31:05] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [14:31:05] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:31:16] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [14:31:17] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [14:31:37] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:31:37] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:31:37] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:31:37] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:31:37] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:31:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:31:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:31:57] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:31:57] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:31:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:31:57] / on thyme is UNKNOWN: NRPE: Unable to read output [14:34:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 77710 MB (12% inode=99%): [14:35:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:35:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:35:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:35:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:35:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:35:58] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:36:19] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:36:19] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [14:36:20] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [14:36:27] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:36:28] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:36:28] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:36:28] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:36:28] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:36:28] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:37:57] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:38:17] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [14:38:17] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [14:38:17] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [14:38:18] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 291962 MB (5% inode=62%): [14:38:18] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [14:38:27] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [14:40:47] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 647868.000000 [14:45:47] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 457488.000000 [15:29:57] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:30:06] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:30:06] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:30:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:30:27] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [15:30:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 760504.000000 [15:30:27] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:30:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:30:27] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:30:37] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [15:30:37] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:30:46] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18135.000000 [15:30:57] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 187139.000000 [15:30:57] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:05] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [15:31:17] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [15:31:18] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [15:31:37] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:37] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:37] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 611472.000000 [15:31:47] RAID on thyme is UNKNOWN: NRPE: Unable to read output [15:31:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 962092.000000 [15:31:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:57] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:31:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:31:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [15:31:57] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:31:57] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:31:58] / on thyme is UNKNOWN: NRPE: Unable to read output [15:32:05] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:32:36] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:32:36] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:35:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:35:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:35:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:35:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 77361 MB (12% inode=99%): [15:35:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:35:58] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:35:58] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:37:17] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:37:17] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [15:37:17] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [15:37:26] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:37:26] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:37:26] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:37:27] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:37:27] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:37:27] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:37:57] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:38:17] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [15:38:18] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [15:38:18] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [15:38:18] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 291556 MB (5% inode=62%): [15:38:18] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [15:38:27] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [15:40:47] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 638093.000000 [15:45:47] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 453580.000000 [16:07:32] Hi guys. How do I kill a process again. [16:07:41] How do I look up the PID? [16:29:57] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:30:06] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:30:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:30:47] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21735.000000 [16:30:56] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 184777.000000 [16:30:56] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:05] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [16:31:17] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [16:31:18] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [16:31:26] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [16:31:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 764006.000000 [16:31:27] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:31:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:27] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:37] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [16:31:37] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:31:37] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:37] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:37] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 607427.000000 [16:31:47] RAID on thyme is UNKNOWN: NRPE: Unable to read output [16:31:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 962043.000000 [16:31:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:57] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:31:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:31:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [16:31:57] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:31:57] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:31:58] / on thyme is UNKNOWN: NRPE: Unable to read output [16:32:05] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:32:18] FLOOD [16:32:36] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:32:36] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:35:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:35:58] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:35:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 77091 MB (12% inode=99%): [16:35:58] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:35:58] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:35:59] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:35:59] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:37:17] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:37:17] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [16:37:17] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [16:37:26] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:37:26] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:37:26] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:38:17] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [16:38:17] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [16:38:17] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [16:38:27] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:38:27] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:38:27] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:38:57] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:39:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 291213 MB (5% inode=62%): [16:39:18] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [16:39:26] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [16:41:46] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 630561.000000 [16:45:47] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 451412.000000 [16:59:56] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [17:30:46] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25335.000000 [17:30:56] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 182707.000000 [17:30:57] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:30:57] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:05] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [17:31:06] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:18] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [17:31:27] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [17:31:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 767354.000000 [17:31:27] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:31:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:27] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:36] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [17:31:36] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:31:36] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 600284.000000 [17:31:47] RAID on thyme is UNKNOWN: NRPE: Unable to read output [17:31:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 962124.000000 [17:31:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:57] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:31:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [17:31:58] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:31:58] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:31:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:31:58] / on thyme is UNKNOWN: NRPE: Unable to read output [17:32:05] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:32:17] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [17:32:36] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:32:37] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:32:37] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:32:37] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:36:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:36:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:36:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:36:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:36:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:36:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:36:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 76746 MB (12% inode=99%): [17:37:17] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:37:17] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [17:37:17] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [17:37:26] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:37:26] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:37:27] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:38:17] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [17:38:17] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [17:38:17] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [17:38:27] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:38:27] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:38:27] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:39:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 290837 MB (5% inode=62%): [17:39:18] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [17:39:58] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:40:27] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [17:41:47] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 621466.000000 [17:45:48] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 450284.000000 [17:47:38] s4 replag on z-dat-s5-b is CRITICAL: (Service Check Timed Out) [17:52:46] s4 replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 932.000000 [17:59:56] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:11:07] * Moe_Epsilon waves and disappears [18:23:04] Avocato * Re: [Toolserver-l] Output files [18:31:06] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [18:31:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:06] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:17] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [18:31:27] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [18:31:27] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28976.000000 [18:31:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 770492.000000 [18:31:27] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:31:27] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:27] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:36] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [18:31:36] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:37] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:31:46] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 591763.000000 [18:31:56] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 179795.000000 [18:31:56] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:31:56] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:56] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:56] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:32:05] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:32:16] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [18:32:35] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:32:36] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:32:36] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:32:36] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:32:47] RAID on thyme is UNKNOWN: NRPE: Unable to read output [18:32:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 961917.000000 [18:32:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:32:57] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:32:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:32:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [18:32:57] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:32:57] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:32:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:32:59] / on thyme is UNKNOWN: NRPE: Unable to read output [18:35:22] Hello all [18:37:17] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:37:17] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [18:37:27] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:37:28] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:37:28] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:37:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:37:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:37:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 76440 MB (12% inode=99%): [18:37:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:37:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:37:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:37:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:38:17] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [18:38:17] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [18:38:17] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [18:38:17] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [18:38:27] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:38:27] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:38:27] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:39:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 290426 MB (5% inode=62%): [18:39:18] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [18:40:57] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:41:27] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [18:41:47] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 616123.000000 [18:46:47] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 445834.000000 [18:59:56] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:08:47] Free Memory on turnera is WARNING: WARNING - 5.4% (451420 kB) free! [19:17:47] Free Memory on turnera is OK: OK - 7.2% (604672 kB) free. [19:31:06] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [19:31:06] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:17] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [19:31:36] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [19:31:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:31:47] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 584272.000000 [19:31:56] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:31:56] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:56] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 177154.000000 [19:32:06] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:06] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:18] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:18] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [19:32:25] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [19:32:25] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32636.000000 [19:32:25] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:32:25] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:25] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:26] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 773528.000000 [19:32:46] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:32:46] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:32:46] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:46] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:46] RAID on thyme is UNKNOWN: NRPE: Unable to read output [19:32:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 962668.000000 [19:32:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:33:06] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:33:07] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:33:07] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [19:33:07] / on thyme is UNKNOWN: NRPE: Unable to read output [19:33:07] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:33:07] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:33:07] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:37:18] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:37:18] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [19:37:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 76061 MB (12% inode=99%): [19:37:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:37:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:38:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:38:09] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:38:09] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:38:09] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:38:17] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [19:38:18] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [19:38:18] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [19:38:18] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [19:38:27] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:38:27] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:38:27] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:39:18] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 290013 MB (5% inode=62%): [19:39:19] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [19:39:35] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:39:35] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:39:36] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:41:06] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:42:26] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [19:42:45] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 615783.000000 [19:46:46] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 441460.000000 [19:53:47] Free Memory on turnera is WARNING: WARNING - 6.0% (498692 kB) free! [20:04:47] Free Memory on turnera is OK: OK - 7.1% (590904 kB) free. [20:29:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:31:16] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [20:31:16] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [20:31:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:25] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:31:48] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 577128.000000 [20:31:58] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [20:31:58] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:58] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 173564.000000 [20:32:06] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:32:06] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:27] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:27] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:27] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36240.000000 [20:32:35] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [20:32:36] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:32:37] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:37] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:46] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 776709.000000 [20:32:58] RAID on thyme is UNKNOWN: NRPE: Unable to read output [20:32:58] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 963432.000000 [20:32:58] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:32:58] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:32:58] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:59] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:33:16] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [20:33:17] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:33:17] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:33:18] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:33:18] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [20:33:25] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:33:26] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:33:26] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:34:06] / on thyme is UNKNOWN: NRPE: Unable to read output [20:37:26] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:38:05] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 75805 MB (12% inode=99%): [20:38:07] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:38:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:38:16] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [20:38:26] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [20:38:26] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:38:26] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:38:26] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:38:26] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:38:46] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:38:46] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:38:46] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:39:16] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [20:39:16] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [20:39:16] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [20:39:16] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 289577 MB (5% inode=62%): [20:39:46] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:39:47] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:39:47] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:40:22] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [20:41:16] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:42:36] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [20:42:47] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 616170.000000 [20:46:48] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 439814.000000 [21:03:55] Free Memory on turnera is WARNING: WARNING - 6.3% (524212 kB) free! [21:11:07] Free Memory on turnera is OK: OK - 7.3% (609536 kB) free. [21:31:26] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [21:31:47] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:47] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:48] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 571255.000000 [21:31:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:32:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [21:32:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:06] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 170970.000000 [21:32:15] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [21:32:25] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:32:26] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:36] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39850.000000 [21:32:48] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:48] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 780011.000000 [21:32:48] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:48] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [21:32:48] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:32:56] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:56] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:33:07] RAID on thyme is UNKNOWN: NRPE: Unable to read output [21:33:07] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 964858.000000 [21:33:07] Free Memory on turnera is WARNING: WARNING - 6.6% (556728 kB) free! [21:33:07] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:33:07] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:33:08] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:33:08] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:33:25] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:33:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:33:37] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [21:33:47] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:33:48] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:33:56] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:34:07] Free Memory on turnera is OK: OK - 8.3% (695424 kB) free. [21:34:16] / on thyme is UNKNOWN: NRPE: Unable to read output [21:34:16] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [21:34:16] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:37:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:38:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 75520 MB (12% inode=99%): [21:38:26] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:38:26] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:38:36] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [21:38:47] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [21:38:48] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:38:48] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:38:48] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:38:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:38:57] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:38:57] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:38:57] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:39:16] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [21:39:16] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [21:39:25] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [21:39:36] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 289172 MB (5% inode=62%): [21:39:56] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:39:56] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:39:56] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:40:41] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [21:41:25] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:42:46] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [21:42:57] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 612053.000000 [21:46:56] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 439233.000000 [21:56:15] DaBPunkt, how do I look up a PID and kill it again? [21:59:28] DaBPunkt, how do I look up and kill a specific PID [21:59:30] ? [21:59:41] Cyberpower678: ps -ef|grep *insertsomehingmatching here* [21:59:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:03:12] DaBPunkt, thanks. [22:14:28] Cyberpower678, you can also use killall [22:14:52] Won't that kill everything? [22:15:05] only on solaris [22:26:36] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 18834 MB (1% inode=99%): [22:31:28] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [22:32:08] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 562848.000000 [22:32:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:32:18] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:18] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 168184.000000 [22:32:26] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:27] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [22:32:28] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:28] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [22:32:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:37] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:45] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43459.000000 [22:32:57] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 783338.000000 [22:33:19] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [22:33:20] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:33:20] RAID on thyme is UNKNOWN: NRPE: Unable to read output [22:33:20] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 966081.000000 [22:33:20] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:33:20] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:33:20] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:33:30] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:33:30] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:33:30] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:33:30] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:33:30] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:33:30] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:33:38] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [22:33:46] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:33:47] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:34:18] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:34:18] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:34:19] / on thyme is UNKNOWN: NRPE: Unable to read output [22:34:27] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:34:28] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [22:34:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:37:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:38:15] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 75086 MB (12% inode=99%): [22:38:36] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [22:38:36] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:38:36] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:38:58] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [22:39:19] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:39:19] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [22:39:28] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:39:28] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:39:28] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:39:29] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:39:29] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:39:29] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:39:29] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [22:39:29] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [22:39:38] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 288910 MB (5% inode=62%): [22:40:27] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:40:27] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:40:28] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:40:38] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [22:41:45] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:42:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [22:43:06] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 610229.000000 [22:47:05] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 437724.000000 [22:48:19] Free Memory on turnera is WARNING: WARNING - 6.0% (506320 kB) free! [22:56:19] Free Memory on turnera is OK: OK - 7.2% (606100 kB) free. [22:59:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:19:38] @replag commons [23:19:38] Dispenser: s1-rr-a: 12h 51m 16s [+1.00 s/s]; s1-rr-a-wd: 4d 10h 14m 43s [-0.30 s/s]; s1-user-wd: 1w 4d 4h 41m 2s [+0.17 s/s]; s2-rr: 12h 51m 18s [+1.00 s/s]; s2-user: 12h 51m 18s [+1.00 s/s]; s2-user-c: error; s2-user-wd: 6d 11h 13m 46s [-1.76 s/s]; s3-user: 18s [-] [23:19:39] Dispenser: s3-user-wd: 5d 11h 20m 54s [-0.92 s/s]; s4-user-wd: 1w 2d 2h 19m 10s [+0.90 s/s]; s5-rr-a-wd: 1d 22h 16m 4s [-0.75 s/s]; s5-user: 12h 51m 17s [+1.00 s/s]; s5-user-c: 15m 31s [-1.32 s/s]; s6-user-wd: 1w 42m 50s [-1.49 s/s]; s7-user-wd: 5d 1h 28m 25s [-0.84 s/s] [23:31:36] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [23:32:08] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 557357.000000 [23:32:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:32:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:35] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [23:32:47] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [23:32:47] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:32:47] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:48] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:57] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47070.000000 [23:33:08] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 786697.000000 [23:33:19] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:19] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [23:33:19] RAID on thyme is UNKNOWN: NRPE: Unable to read output [23:33:19] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 166176.000000 [23:33:19] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 967485.000000 [23:33:19] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:33:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:36] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:36] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:46] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:33:46] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:33:47] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [23:33:47] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:47] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:58] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:34:18] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:34:19] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:34:19] / on thyme is UNKNOWN: NRPE: Unable to read output [23:34:19] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:34:26] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:34:36] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [23:34:36] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:34:48] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:38:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:38:19] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 74795 MB (12% inode=99%): [23:38:48] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [23:38:49] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:38:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:39:08] APT on sage is CRITICAL: APT CRITICAL: 7 packages available for upgrade (4 critical updates). [23:39:18] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:39:38] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [23:39:38] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:39:38] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:39:38] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:39:38] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:39:38] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:39:38] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [23:39:39] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [23:39:47] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 287406 MB (5% inode=62%): [23:39:57] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:40:37] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:40:46] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [23:40:46] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:40:46] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:40:57] s4 replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2205.000000 [23:42:19] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:42:57] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [23:43:08] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 604640.000000 [23:45:19] /sql on z-dat-s7-a is CRITICAL: DISK CRITICAL - free space: /sql 21665 MB (4% inode=99%): [23:47:09] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 436712.000000 [23:48:47] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [23:59:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL