[00:00:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:00:06] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:00:06] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:00:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:00:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:00:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:00:07] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:00:07] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:00:08] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:00:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:00:16] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:00:16] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 641324.000000 [00:00:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1088665.000000 [00:00:16] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:00:16] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:00:17] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 345612.000000 [00:01:26] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 54884 MB (8% inode=99%): [00:01:26] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:26] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:01:26] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:01:26] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [00:01:27] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:01:36] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:36] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:36] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:36] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:37] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:01:37] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:01:45] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:01:45] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:02:05] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10805.000000 [00:03:06] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10862.000000 [00:04:15] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10874.000000 [00:06:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [00:06:36] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [00:06:55] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [00:07:35] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [00:07:35] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 256179 MB (4% inode=62%): [00:07:35] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [00:08:06] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [00:19:26] / on thyme is UNKNOWN: NRPE: Unable to read output [00:20:16] RAID on thyme is UNKNOWN: NRPE: Unable to read output [00:23:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [00:30:25] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:59:36] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:36] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:36] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:45] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [00:59:45] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [00:59:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:45] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:55] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [00:59:56] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:56] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:56] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1019787.000000 [00:59:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:59:56] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:59:56] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:01:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:01:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:05] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:01:06] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:01:06] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:01:06] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:01:15] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 644984.000000 [01:01:15] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:15] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1092325.000000 [01:01:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:01:16] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:16] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 349212.000000 [01:01:26] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 54615 MB (8% inode=99%): [01:01:26] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:01:26] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:27] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:01:27] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [01:01:28] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:01:35] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:36] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:36] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:36] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:01:36] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:36] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:37] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:01:45] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:01:45] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:02:06] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14405.000000 [01:03:06] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14461.000000 [01:04:16] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14474.000000 [01:06:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [01:06:36] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [01:06:55] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:06:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [01:07:36] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [01:07:36] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 258904 MB (4% inode=62%): [01:07:36] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [01:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [01:19:26] / on thyme is UNKNOWN: NRPE: Unable to read output [01:20:15] RAID on thyme is UNKNOWN: NRPE: Unable to read output [01:23:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [01:30:36] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:59:35] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:59:45] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [01:59:46] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [01:59:46] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:59:46] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:59:46] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:59:56] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [01:59:56] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:59:56] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:59:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:59:56] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1023386.000000 [01:59:56] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:59:56] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:00:36] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:00:36] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:01:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:01:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:01:06] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:01:07] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:01:07] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:01:15] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 648584.000000 [02:01:16] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:16] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:16] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:01:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1095926.000000 [02:01:17] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:17] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:25] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 352812.000000 [02:01:26] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 53266 MB (8% inode=99%): [02:01:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:01:27] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:27] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:27] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:01:27] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [02:01:27] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:01:37] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:37] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:37] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:01:37] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:37] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:37] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:01:46] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:01:46] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:03:06] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18065.000000 [02:04:05] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18121.000000 [02:05:16] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18134.000000 [02:06:56] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:06:56] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [02:07:35] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [02:07:35] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [02:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [02:08:37] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [02:08:37] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [02:08:37] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 258548 MB (4% inode=62%): [02:20:15] RAID on thyme is UNKNOWN: NRPE: Unable to read output [02:20:25] / on thyme is UNKNOWN: NRPE: Unable to read output [02:23:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [02:59:36] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:59:46] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [02:59:46] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [02:59:46] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:59:46] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:59:56] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:59:56] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [02:59:56] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:59:56] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:59:56] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1026987.000000 [02:59:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:59:56] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:59:57] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:00:25] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:00:36] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:00:36] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:01:06] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:06] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:06] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:06] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:07] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:07] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:16] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 652184.000000 [03:01:16] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:16] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1099526.000000 [03:01:16] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:16] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:16] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:36] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:36] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:36] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:36] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:36] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:01:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:01:46] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:01:46] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:02:25] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 356472.000000 [03:02:26] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 54757 MB (8% inode=99%): [03:02:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:02:26] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:02:27] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:02:27] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:02:27] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:02:27] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:02:35] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:03:06] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21665.000000 [03:04:05] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21721.000000 [03:05:16] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21734.000000 [03:06:56] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:06:56] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [03:07:35] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [03:07:35] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [03:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [03:08:36] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [03:08:37] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [03:08:37] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 258414 MB (4% inode=62%): [03:20:15] RAID on thyme is UNKNOWN: NRPE: Unable to read output [03:20:25] / on thyme is UNKNOWN: NRPE: Unable to read output [03:23:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [03:26:59] * Krinkle added https to bit.ly links [03:59:36] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:59:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:26] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:00:36] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:36] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:45] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [04:00:45] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [04:00:46] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:46] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:54] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [04:00:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:56] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:00:56] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1030647.000000 [04:00:56] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:00:56] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:01:36] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:01:36] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:01:37] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:01:37] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:01:37] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:01:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:02:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:02:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:02:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:05] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:05] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:05] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:02:06] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:02:06] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:02:07] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:02:15] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 655844.000000 [04:02:15] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:16] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:02:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1103186.000000 [04:02:16] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:16] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:36] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:02:46] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:02:47] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:03:06] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25265.000000 [04:03:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 360131.000000 [04:03:26] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 53279 MB (8% inode=99%): [04:03:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:03:26] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:03:26] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:03:26] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:03:26] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [04:04:06] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25321.000000 [04:05:16] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25334.000000 [04:06:55] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:06:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [04:07:25] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:07:35] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [04:07:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [04:08:04] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [04:08:37] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [04:08:37] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [04:08:37] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 258237 MB (4% inode=62%): [04:20:16] RAID on thyme is UNKNOWN: NRPE: Unable to read output [04:21:26] / on thyme is UNKNOWN: NRPE: Unable to read output [04:23:46] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [04:59:36] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:59:56] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:26] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:00:36] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:46] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [05:00:46] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [05:00:46] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:46] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:46] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:56] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [05:00:57] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:57] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:57] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:00:57] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1034248.000000 [05:00:57] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:00:57] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:01:36] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:01:36] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:01:36] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:01:37] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:01:46] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:02:06] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:02:06] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:02:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:16] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:02:16] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:02:16] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:02:16] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 659444.000000 [05:02:16] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:17] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:17] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:02:18] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:18] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:36] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:02:45] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1106812.000000 [05:02:46] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:02:46] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:02:46] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:03:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 363733.000000 [05:03:26] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 52419 MB (8% inode=99%): [05:03:27] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:03:27] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:03:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:03:27] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:03:27] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [05:04:05] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28925.000000 [05:04:05] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28921.000000 [05:05:16] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28934.000000 [05:06:56] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:06:57] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [05:07:35] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [05:08:36] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [05:08:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [05:09:36] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [05:09:37] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [05:09:37] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 258058 MB (4% inode=62%): [05:10:16] Free Memory on turnera is CRITICAL: CRITICAL - 3.5% (294960 kB) free! [05:18:15] Free Memory on turnera is OK: OK - 88.3% (7400044 kB) free. [05:20:34] RAID on thyme is UNKNOWN: NRPE: Unable to read output [05:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [05:24:25] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [05:59:45] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:05] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:05] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:05] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [06:01:05] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [06:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [06:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:45] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:01:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1037908.000000 [06:01:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:02:24] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:02:34] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 663065.000000 [06:02:35] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:02:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:03:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:03:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:03:05] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1110435.000000 [06:03:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:05] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:03:06] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:03:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:03:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:03:35] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:03:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [06:03:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 52195 MB (8% inode=99%): [06:04:05] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32525.000000 [06:04:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:04:14] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:04:15] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 367387.000000 [06:04:24] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:05:05] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32581.000000 [06:05:55] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32575.000000 [06:07:05] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:07:35] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:07:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [06:08:04] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [06:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [06:09:15] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [06:09:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [06:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 257906 MB (4% inode=62%): [06:10:15] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [06:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:16:05] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:16:15] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:16:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:20:34] RAID on thyme is UNKNOWN: NRPE: Unable to read output [06:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [06:24:25] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [06:30:25] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:59:44] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:05] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:05] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:05] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [07:01:06] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [07:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [07:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:45] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:01:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1041471.000000 [07:01:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:02:25] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:02:35] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 666312.000000 [07:02:35] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:02:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:03:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:03:06] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:03:06] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1113417.000000 [07:03:06] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:03:07] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:03:16] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:03:35] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:03:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [07:03:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 52049 MB (8% inode=99%): [07:03:55] aliasd on mayapple is CRITICAL: No data received from host [07:04:05] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 33370.000000 [07:04:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:04:15] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:04:15] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 370276.000000 [07:04:25] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:05:04] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 33694.000000 [07:05:55] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36175.000000 [07:07:05] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:07:35] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:07:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [07:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [07:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [07:09:15] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [07:09:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [07:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 257745 MB (4% inode=62%): [07:10:15] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [07:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:16:05] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:16:15] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:16:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:20:35] RAID on thyme is UNKNOWN: NRPE: Unable to read output [07:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [07:24:24] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [07:28:55] aliasd on nightshade is OK: TCP OK - 0.022 second response time on port 984 [500 Not found.] [07:30:25] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:35:55] aliasd on nightshade is CRITICAL: Connection refused [07:37:55] aliasd on nightshade is OK: TCP OK - 0.004 second response time on port 984 [500 Not found.] [07:59:45] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:05] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:05] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:05] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [08:01:05] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [08:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [08:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:45] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:01:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1044740.000000 [08:01:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:02:25] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:02:35] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 662355.000000 [08:02:35] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:02:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:03:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:03:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:03:05] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1111571.000000 [08:03:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:05] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:03:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:03:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:03:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:03:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [08:03:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 51799 MB (8% inode=99%): [08:03:56] aliasd on mayapple is CRITICAL: No data received from host [08:04:05] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 13769.000000 [08:04:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:04:15] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:04:15] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 369009.000000 [08:04:25] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:05:05] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14255.000000 [08:05:55] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39775.000000 [08:07:35] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:07:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [08:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [08:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [08:09:14] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [08:09:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [08:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 257526 MB (4% inode=62%): [08:10:15] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [08:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:05] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:14] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:15] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:20:35] RAID on thyme is UNKNOWN: NRPE: Unable to read output [08:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [08:24:25] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [08:30:24] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:31:05] s4 replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3124.000000 [08:36:04] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3051.000000 [08:37:05] s4 replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1734.000000 [08:43:04] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1574.000000 [08:59:45] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:05] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:05] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:05] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [09:01:05] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [09:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [09:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:45] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:01:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1048017.000000 [09:01:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:01:55] aliasd on yarrow is CRITICAL: Connection refused [09:02:25] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:02:35] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:02:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:02:35] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 660503.000000 [09:03:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:03:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:03:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:03:05] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1111098.000000 [09:03:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:05] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:05] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:03:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:03:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:03:35] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:03:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [09:03:56] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 51610 MB (8% inode=99%): [09:03:56] aliasd on mayapple is CRITICAL: No data received from host [09:04:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:04:14] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:04:15] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 367502.000000 [09:04:25] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:05:54] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43375.000000 [09:07:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:07:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [09:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [09:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [09:09:14] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [09:09:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [09:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 256634 MB (4% inode=62%): [09:10:20] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [09:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:05] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:14] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:15] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:20:35] RAID on thyme is UNKNOWN: NRPE: Unable to read output [09:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [09:24:25] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [09:30:24] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:59:45] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:05] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:05] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:05] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [10:01:05] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [10:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [10:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:46] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:46] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:01:56] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1051337.000000 [10:01:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:01:56] aliasd on yarrow is CRITICAL: Connection refused [10:02:25] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:02:35] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:02:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:02:35] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 658688.000000 [10:03:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:03:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:03:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:03:05] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1110510.000000 [10:03:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:03:05] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:03:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:03:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:03:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:03:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:03:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [10:03:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 51458 MB (8% inode=99%): [10:03:55] aliasd on mayapple is CRITICAL: No data received from host [10:04:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:04:15] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:04:15] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 366274.000000 [10:04:25] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:05:55] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46975.000000 [10:07:35] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:07:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [10:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [10:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [10:09:12] Hi all, I have trouble signing up in JIRA [10:09:15] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [10:09:33] I registered yesterday my username (cristiancantoro) and received the activation email [10:09:38] but now I see this: http://img841.imageshack.us/img841/8911/jiraaccounterror.png [10:09:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [10:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 256155 MB (4% inode=62%): [10:10:10] and if I attemp password recovery it says that neither my username, nor my email address are found [10:10:15] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [10:10:50] and "Contact system administrators" is unset (https://jira.toolserver.org/secure/ContactAdministrators!default.jspa) which is kinda LoL [10:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:05] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:14] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:19:16] CristianCantoro: you should email ts-admins@wikimedia.org [10:19:52] legoktm: thanks for the suggestion! I'll do that right away :) [10:20:02] np [10:20:35] RAID on thyme is UNKNOWN: NRPE: Unable to read output [10:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [10:24:25] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [10:29:05] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 17086 MB (1% inode=99%): [10:30:24] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:59:45] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:05] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:05] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:05] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [11:01:05] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [11:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [11:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:45] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:01:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1054711.000000 [11:01:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:01:55] aliasd on yarrow is CRITICAL: Connection refused [11:02:25] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:02:35] Sun Grid Engine execd on ortelius is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:02:36] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:02:36] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:02:36] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 658553.000000 [11:03:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:03:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:03:06] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:03:06] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:03:06] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1109998.000000 [11:03:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:03:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:03:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:03:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:03:34] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:03:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [11:03:56] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 51142 MB (8% inode=99%): [11:03:56] aliasd on mayapple is CRITICAL: No data received from host [11:04:15] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:04:16] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:04:16] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 365222.000000 [11:04:25] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:05:55] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50575.000000 [11:07:35] Sun Grid Engine execd on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:07:35] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:07:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [11:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [11:08:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [11:09:14] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [11:09:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [11:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 255446 MB (4% inode=62%): [11:10:15] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [11:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:16:06] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:16:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:16:16] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:20:35] RAID on thyme is UNKNOWN: NRPE: Unable to read output [11:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [11:24:25] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [11:25:45] Free Memory on damiana is CRITICAL: CRITICAL - 4.3% (358872 kB) free! [11:27:45] Free Memory on damiana is WARNING: WARNING - 5.4% (448688 kB) free! [11:30:24] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:30:44] Free Memory on damiana is CRITICAL: CRITICAL - 4.1% (343888 kB) free! [11:59:45] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:06] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:06] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [12:01:06] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [12:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [12:01:45] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:02:24] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:02:34] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:02:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:02:35] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 658699.000000 [12:02:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1058139.000000 [12:02:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:02:55] aliasd on yarrow is CRITICAL: Connection refused [12:03:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:03:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:03:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:03:35] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:03:56] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [12:03:56] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 50948 MB (8% inode=99%): [12:03:56] aliasd on mayapple is CRITICAL: No data received from host [12:04:05] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1108891.000000 [12:04:05] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:04:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:04:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:04:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:04:14] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:04:15] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 364453.000000 [12:04:24] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:05:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:06:00] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54175.000000 [12:07:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:07:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [12:08:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:08:06] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [12:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [12:09:15] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [12:09:44] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [12:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 255152 MB (4% inode=62%): [12:10:15] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [12:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:16:05] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:16:15] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:16:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:20:35] RAID on thyme is UNKNOWN: NRPE: Unable to read output [12:22:05] / on thyme is UNKNOWN: NRPE: Unable to read output [12:22:25] / on willow is WARNING: DISK WARNING - free space: / 20645 MB (19% inode=98%): [12:22:35] /tmp on willow is WARNING: DISK WARNING - free space: / 20645 MB (19% inode=98%): [12:24:25] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [12:30:25] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:30:25] / on willow is CRITICAL: DISK CRITICAL - free space: / 10971 MB (10% inode=98%): [12:30:34] /tmp on willow is CRITICAL: DISK CRITICAL - free space: / 10782 MB (10% inode=97%): [12:30:44] Free Memory on damiana is CRITICAL: CRITICAL - 1.8% (153312 kB) free! [12:49:24] / on willow is WARNING: DISK WARNING - free space: / 21937 MB (20% inode=98%): [12:49:34] /tmp on willow is WARNING: DISK WARNING - free space: / 21937 MB (20% inode=98%): [12:52:25] / on willow is OK: DISK OK - free space: / 28211 MB (26% inode=99%): [12:52:35] /tmp on willow is OK: DISK OK - free space: / 28211 MB (26% inode=99%): [12:59:45] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:00:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:01:05] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:01:25] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [13:01:35] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:01:35] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [13:01:45] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:01:45] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:01:45] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:01:45] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:02:04] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:02:05] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [13:02:05] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:02:24] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:02:25] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:02:35] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:02:35] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 658423.000000 [13:02:36] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:02:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1061357.000000 [13:02:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:02:55] aliasd on yarrow is CRITICAL: Connection refused [13:03:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:03:05] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:03:15] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:03:15] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:03:15] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:03:15] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:03:25] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:03:35] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:03:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:03:56] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 50577 MB (8% inode=99%): [13:03:56] aliasd on mayapple is CRITICAL: No data received from host [13:04:05] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1107105.000000 [13:04:06] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:04:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:04:06] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:04:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:04:15] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:04:15] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 363573.000000 [13:04:25] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:04:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:05:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:05:55] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57775.000000 [13:07:34] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:07:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [13:08:05] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [13:08:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [13:09:14] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [13:09:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [13:09:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 278843 MB (5% inode=62%): [13:10:15] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [13:16:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:16:06] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:16:14] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:16:15] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:22:08] RAID on thyme is UNKNOWN: NRPE: Unable to read output [13:23:42] Sun Grid Engine execd on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:44:37] Looks like we have problems again [13:44:44] :( [13:49:38] getting password prompt on willow, yeah [13:51:13] tool server is not work? [13:52:09] nope :( [14:59:11] wow, I just got an ssh session on yarrow [14:59:13] \o/ [14:59:23] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 369291.000000 [14:59:23] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:59:23] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 50310 MB (8% inode=99%): [14:59:23] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:59:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:59:24] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:24] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:25] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:59:25] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:59:26] Sun Grid Engine execd on mayapple is CRITICAL: CRITICAL: execd not communicating [14:59:26] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5900 [14:59:32] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:32] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:32] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:32] looks like we're back [14:59:33] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:33] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:33] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:59:42] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:00:22] Sun Grid Engine execd on mayapple is OK: Host and Queues Ok [15:00:41] Free Memory on damiana is OK: OK - 92.4% (7739856 kB) free. [15:00:42] All right :) [15:01:12] s5 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2657.000000 [15:02:13] s5 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 855.000000 [15:03:34] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:04:43] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:05:23] aliasd on mayapple is CRITICAL: No data received from host [15:05:35] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [15:05:35] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [15:05:36] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1113052.000000 [15:05:36] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 377541 MB (7% inode=62%): [15:05:36] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:05:36] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [15:05:36] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [15:05:43] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [15:05:43] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [15:05:44] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [15:06:03] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3147.000000 [15:07:13] MySQL slave on daphne is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3483 [15:07:44] s4 replag on daphne is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3286.000000 [15:09:03] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1540.000000 [15:12:13] MySQL slave on daphne is OK: Uptime: 415534 Threads: 59 Questions: 61118802 Slow queries: 86165 Opens: 19183 Flush tables: 1 Open tables: 782 Queries per second avg: 147.84 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1734 [15:12:44] s4 replag on daphne is OK: QUERY OK: SELECT ts_rc_age() returned 1452.000000 [15:17:24] / on thyme is UNKNOWN: NRPE: Unable to read output [15:23:23] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3504 [15:24:04] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3494.000000 [15:35:02] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1788.000000 [15:35:23] MySQL slave on rosemary is OK: Uptime: 420291 Threads: 12 Questions: 298673693 Slow queries: 151218 Opens: 26270 Flush tables: 1 Open tables: 3610 Queries per second avg: 710.635 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1688 [15:58:34] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:34] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:53] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [15:58:53] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1071706.000000 [15:58:53] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:58:54] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:54] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:54] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:54] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [15:58:54] aliasd on yarrow is CRITICAL: Connection refused [15:58:55] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:55] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:58:56] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [15:59:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:59:03] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:59:04] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:04] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 9500.000000 [15:59:04] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:04] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:04] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 9498.000000 [15:59:04] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:59:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:59:05] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:59:24] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:59:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 372892.000000 [15:59:24] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 50018 MB (8% inode=99%): [15:59:25] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:59:25] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:25] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:59:26] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:59:26] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [15:59:33] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:33] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:34] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:34] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:34] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:34] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:59:44] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:03:33] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:04:53] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:05:34] aliasd on mayapple is CRITICAL: No data received from host [16:05:45] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [16:05:45] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [16:05:45] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 375219 MB (7% inode=62%): [16:05:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:05:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [16:05:45] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [16:05:45] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [16:06:04] Marc-André Pelletier * [Toolserver-l] Tool Labs maintenance cancelled [16:06:34] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [16:06:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [16:06:34] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1116716.000000 [16:17:23] / on thyme is UNKNOWN: NRPE: Unable to read output [16:25:05] Alex Brollo * [Toolserver-l] I can't login any more [16:58:34] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:58:34] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:58:54] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [16:58:54] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1075307.000000 [16:58:54] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:58:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [16:58:55] aliasd on yarrow is CRITICAL: Connection refused [16:58:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:58:55] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:58:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:58:56] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:58:56] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:58:57] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:03] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [16:59:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:59:03] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:59:03] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:13] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 71771.000000 [16:59:13] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 672009.000000 [16:59:13] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:59:13] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:13] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:14] RAID on thyme is UNKNOWN: NRPE: Unable to read output [16:59:14] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:15] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:33] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:59:34] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:34] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:34] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:59:44] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:00:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 13160.000000 [17:00:03] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:00:03] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:00:03] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 13158.000000 [17:00:03] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:00:03] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:00:04] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:00:04] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:00:24] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49615 MB (8% inode=99%): [17:00:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 376553.000000 [17:00:24] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:00:24] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:00:24] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:00:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:00:24] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:00:25] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [17:00:34] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:00:34] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:00:34] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:03:34] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:04:53] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:05:44] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [17:05:44] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [17:05:44] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 375000 MB (7% inode=62%): [17:05:44] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:05:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [17:06:23] aliasd on mayapple is CRITICAL: No data received from host [17:06:33] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [17:06:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [17:06:35] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1120316.000000 [17:06:45] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [17:06:45] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [17:18:23] / on thyme is UNKNOWN: NRPE: Unable to read output [17:58:34] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:58:34] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:03] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [17:59:04] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:59:04] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:59:04] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:13] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75371.000000 [17:59:14] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 675609.000000 [17:59:14] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:59:14] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:14] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:14] RAID on thyme is UNKNOWN: NRPE: Unable to read output [17:59:14] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:15] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:35] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:35] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:44] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:59:54] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [17:59:54] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1078967.000000 [17:59:54] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:59:54] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [17:59:54] aliasd on yarrow is CRITICAL: Connection refused [17:59:54] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:54] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:55] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:56] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:59:56] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:00:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 16760.000000 [18:00:34] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:00:34] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:00:34] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:01:03] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:01:04] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:01:04] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 16818.000000 [18:01:04] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:01:04] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:01:04] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:01:04] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:01:24] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49405 MB (8% inode=99%): [18:01:24] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 380213.000000 [18:01:24] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:01:24] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:01:24] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:01:24] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:01:24] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:01:25] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [18:03:34] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:04:53] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:05:44] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [18:05:44] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [18:05:44] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 374683 MB (7% inode=62%): [18:05:44] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [18:05:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:06:35] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [18:06:35] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [18:06:35] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1123916.000000 [18:06:44] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [18:06:44] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [18:07:23] aliasd on mayapple is CRITICAL: No data received from host [18:19:24] / on thyme is UNKNOWN: NRPE: Unable to read output [18:21:04] Tim Landscheidt * Re: [Toolserver-l] I can't login any more [18:29:34] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:31:11] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 381999.000000 [18:31:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49303 MB (8% inode=99%): [18:31:11] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:31:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:31:11] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:12] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:12] /sql on z-dat-s7-a is CRITICAL: DISK CRITICAL - free space: /sql 18320 MB (3% inode=99%): [18:31:12] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [18:31:12] / on thyme is UNKNOWN: NRPE: Unable to read output [18:31:22] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:22] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:22] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:22] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:22] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:23] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:31:31] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:47:37] im hoping to run a python script on toolserver - but it requires a couple of python modules that are not installed on TS (feedparser and requests). what is the best way to cope with this on TS? [18:50:42] awjr: file a JIRA ticket [18:51:25] Betacommand: ok is that standard practice, or should i just install the modules locally (or do both for now)? [18:51:43] JIRA is standard [18:51:57] ok thanks Betacommand [19:00:38] Betacommand: what would be the correct 'project' to file the jira ticket under? [19:03:33] awjr: TS [19:14:32] thanks Betacommand [19:30:21] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [19:30:22] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [19:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:30:23] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 374148 MB (7% inode=62%): [19:30:23] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [19:30:23] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:23] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:23] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:24] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [19:30:31] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [19:30:31] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:32] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [19:30:32] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [19:30:32] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:32] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:42] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [19:30:42] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:42] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:42] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1084413.000000 [19:30:42] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:42] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [19:30:42] aliasd on yarrow is CRITICAL: Connection refused [19:30:42] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:42] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [19:30:51] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:52] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:30:52] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:52] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:52] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:52] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:30:52] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 22208.000000 [19:30:53] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 22205.000000 [19:30:53] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:54] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:54] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:55] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:31:00] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 681117.000000 [19:31:01] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 80879.000000 [19:31:01] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:01] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:02] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:31:02] RAID on thyme is UNKNOWN: NRPE: Unable to read output [19:31:02] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1128983.000000 [19:31:02] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:02] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:12] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 385599.000000 [19:31:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49066 MB (8% inode=99%): [19:31:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:31:12] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:31:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:31:12] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:12] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:13] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:31:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [19:31:14] / on thyme is UNKNOWN: NRPE: Unable to read output [19:31:14] aliasd on mayapple is CRITICAL: No data received from host [19:31:22] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:22] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:22] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:22] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:31:31] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:32:21] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:32:22] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:52:22] Hello all [20:13:36] DaBPunkt: Ist user_email_authenticated public? [20:14:10] hoo: doesnt the abusefilter expose that anyways? [20:14:28] hoo: yes [20:14:57] legoktm: Yes, but only if a filter is hit, of course [20:15:03] DaBPunkt: Thanks :) [20:15:18] np [20:15:52] hoo: or if the user allows mail from other users [20:25:52] s4 replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2882.000000 [20:27:51] s4 replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1444.000000 [20:30:21] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [20:30:21] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [20:30:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:30:22] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 373771 MB (7% inode=62%): [20:30:22] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [20:30:22] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:23] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:23] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:31] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [20:30:32] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [20:30:32] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:32] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [20:30:32] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [20:30:32] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:32] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:41] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [20:30:42] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1087799.000000 [20:30:42] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:42] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:42] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:30:42] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [20:30:42] aliasd on yarrow is CRITICAL: Connection refused [20:30:43] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:43] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [20:30:50] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:30:51] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:30:51] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:51] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:52] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:30:52] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:05] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 681246.000000 [20:31:05] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 84479.000000 [20:31:05] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:05] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:05] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:31:05] RAID on thyme is UNKNOWN: NRPE: Unable to read output [20:31:05] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1130168.000000 [20:31:05] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:05] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:31:12] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 386583.000000 [20:31:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48854 MB (7% inode=99%): [20:31:21] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:31:31] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:31:51] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 9455.000000 [20:31:51] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:31:51] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:31:52] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:31:52] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:32:11] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:32:12] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:12] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:12] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:32:12] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [20:32:12] / on thyme is UNKNOWN: NRPE: Unable to read output [20:32:13] aliasd on mayapple is CRITICAL: No data received from host [20:32:21] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:22] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:22] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:22] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:32:22] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:45:51] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3417.000000 [20:48:51] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1617.000000 [21:30:22] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [21:30:22] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [21:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:30:22] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 372895 MB (7% inode=62%): [21:30:22] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [21:30:23] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:23] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:24] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:31] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [21:30:31] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [21:30:31] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:31] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [21:30:32] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [21:30:32] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:32] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:41] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [21:30:41] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1091066.000000 [21:30:42] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:42] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:51] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:30:52] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:30:52] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:52] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:52] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:30:52] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 678038.000000 [21:31:01] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 88079.000000 [21:31:02] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:02] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:02] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:31:02] RAID on thyme is UNKNOWN: NRPE: Unable to read output [21:31:03] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1129313.000000 [21:31:03] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:03] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:12] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 385692.000000 [21:31:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48450 MB (7% inode=99%): [21:31:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:31:25] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:31] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:31:41] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:31:41] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [21:31:42] aliasd on yarrow is CRITICAL: Connection refused [21:31:42] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:31:42] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [21:31:52] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:31:52] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:31:52] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:31:52] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:32:11] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:32:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:32:12] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:12] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:12] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:32:12] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [21:32:12] / on thyme is UNKNOWN: NRPE: Unable to read output [21:32:13] aliasd on mayapple is CRITICAL: No data received from host [21:32:21] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:21] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:21] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:21] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:32:21] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:17:04] Alex Brollo * Re: [Toolserver-l] I can't login any more [22:29:32] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 17220 MB (1% inode=99%): [22:30:22] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [22:30:22] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [22:30:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:30:23] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 372688 MB (6% inode=62%): [22:30:23] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [22:30:23] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:23] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:31] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [22:30:31] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [22:30:32] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:32] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:32] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:32] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [22:30:33] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [22:30:41] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [22:30:42] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1094357.000000 [22:30:42] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:42] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:51] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:30:52] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:52] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:30:52] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:52] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:30:52] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:31:02] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 675033.000000 [22:31:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 91680.000000 [22:31:22] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:31:22] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:31:31] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:31:43] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:31:43] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [22:31:43] aliasd on yarrow is CRITICAL: Connection refused [22:31:43] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:31:43] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [22:32:01] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:02] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:02] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:02] RAID on thyme is UNKNOWN: NRPE: Unable to read output [22:32:02] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1127158.000000 [22:32:03] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:03] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:11] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 384503.000000 [22:32:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48237 MB (7% inode=99%): [22:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:32:12] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:12] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:12] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:32:12] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [22:32:13] / on thyme is UNKNOWN: NRPE: Unable to read output [22:32:13] aliasd on mayapple is CRITICAL: No data received from host [22:32:21] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:21] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:21] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:21] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:21] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:32:51] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:52] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:52] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:34:04] John * Re: [Toolserver-l] I can't login any more [22:38:52] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:39:12] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:50:04] Platonides * Re: [Toolserver-l] I can't login any more [22:56:05] nacht ts [23:01:11] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:30:23] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [23:30:23] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [23:30:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:30:23] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 371235 MB (6% inode=62%): [23:30:23] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [23:30:24] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:24] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:31] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [23:30:41] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [23:30:41] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1097668.000000 [23:30:41] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:41] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:51] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:30:52] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:52] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:52] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:52] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:30:52] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:31:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 95280.000000 [23:31:02] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 672318.000000 [23:31:22] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:31:22] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:31:31] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:31:31] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [23:31:31] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:31:32] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:31:32] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:31:32] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [23:31:32] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [23:31:41] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:31:42] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [23:32:06] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:32:06] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:06] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:06] RAID on thyme is UNKNOWN: NRPE: Unable to read output [23:32:06] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1125220.000000 [23:32:06] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:06] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:11] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 383862.000000 [23:32:21] aliasd on mayapple is CRITICAL: No data received from host [23:32:22] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:22] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:22] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:23] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:23] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:42] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [23:32:42] aliasd on yarrow is CRITICAL: Connection refused [23:32:42] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:32:52] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:32:52] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:32:52] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:33:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 47982 MB (7% inode=99%): [23:33:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:33:11] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:12] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:33:12] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:33:12] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [23:33:12] / on thyme is UNKNOWN: NRPE: Unable to read output [23:38:52] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:40:12] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds.