[00:02:01] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 35966.000000 [00:02:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 35963 [00:02:11] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 24006.000000 [00:17:23] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100816 MB (16% inode=99%): [00:28:21] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:31:41] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [00:32:30] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:33:41] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [00:35:42] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [00:36:10] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [00:37:01] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [00:37:11] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [00:37:11] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [00:37:21] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:37:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:38:34] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [00:39:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:43:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:44:34] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [00:44:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:02:01] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 34677.000000 [01:02:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 34669 [01:02:11] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 27606.000000 [01:17:21] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100618 MB (16% inode=99%): [01:28:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:31:41] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [01:32:30] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:33:45] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [01:35:40] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [01:36:10] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [01:37:04] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [01:37:11] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [01:37:11] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [01:37:21] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:37:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:38:31] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [01:39:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:43:20] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:44:31] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [01:44:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:02:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32947.000000 [02:02:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 32943 [02:02:11] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 31206.000000 [02:17:21] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100421 MB (16% inode=99%): [02:28:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:31:44] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [02:32:31] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:33:41] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [02:35:41] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [02:36:10] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [02:37:01] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [02:37:13] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [02:37:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [02:37:21] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:37:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:38:31] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [02:39:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:43:20] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:44:31] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [02:44:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:02:05] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 30973.000000 [03:02:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 30972 [03:02:11] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 34806.000000 [03:17:20] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100258 MB (16% inode=99%): [03:28:34] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:31:41] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [03:32:30] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:33:41] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [03:35:40] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [03:36:10] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [03:37:01] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [03:37:15] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [03:37:15] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [03:37:21] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:37:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:38:31] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [03:39:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:43:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:44:35] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [03:44:35] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:54:11] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 64526 MB (6% inode=97%): [03:54:11] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 64526 MB (6% inode=97%): [04:02:26] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 38426.000000 [04:02:54] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 30912.000000 [04:03:03] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 30911 [04:17:23] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100050 MB (16% inode=99%): [04:29:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:31:54] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [04:32:33] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:33:52] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [04:35:52] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [04:36:17] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [04:37:15] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [04:37:15] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [04:37:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:37:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:37:53] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [04:38:33] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [04:39:55] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:43:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:44:33] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [04:44:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:02:37] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 42029.000000 [05:02:53] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 31020.000000 [05:03:05] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 31027 [05:17:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99919 MB (16% inode=99%): [05:29:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:31:55] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [05:31:55] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:32:33] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:33:53] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [05:35:53] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [05:36:15] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [05:37:13] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [05:37:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [05:37:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:37:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:37:56] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [05:38:32] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [05:39:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:43:35] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:44:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:44:33] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [06:01:34] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [06:02:33] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 45630.000000 [06:02:52] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 31297.000000 [06:03:02] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 31305 [06:17:23] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99753 MB (16% inode=99%): [06:29:25] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:31:53] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [06:32:33] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:33:54] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [06:35:53] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [06:36:14] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [06:37:13] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [06:37:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [06:37:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:37:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:37:53] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [06:38:34] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [06:39:52] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:43:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:44:36] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:44:36] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [07:02:33] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 49229.000000 [07:02:53] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 29843.000000 [07:03:03] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 29844 [07:17:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99564 MB (16% inode=99%): [07:29:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:31:52] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [07:32:33] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:33:54] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [07:35:53] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [07:36:13] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [07:37:13] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [07:37:14] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [07:37:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:37:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:37:56] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [07:38:32] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [07:39:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:43:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:44:36] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [07:44:36] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:58:16] /sql on thyme is WARNING: DISK WARNING - free space: /sql 200435 MB (20% inode=99%): [08:03:03] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50900.000000 [08:03:03] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 29886 [08:03:16] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 29887.000000 [08:17:23] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99363 MB (16% inode=99%): [08:29:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:31:53] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [08:32:33] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:33:57] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [08:35:52] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [08:36:12] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [08:37:17] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [08:37:17] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [08:37:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:37:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:38:02] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [08:38:33] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [08:39:54] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:43:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:44:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:44:33] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [09:03:03] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39331.000000 [09:03:03] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 31043 [09:03:16] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 31042.000000 [09:17:23] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99109 MB (16% inode=99%): [09:19:23] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 92032 MB (9% inode=99%): [09:20:04] Environment IPMI on wolfsbane is UNKNOWN: CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [09:21:33] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:29:24] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:31:52] APT on nightshade is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [09:33:55] APT on mayapple is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [09:35:52] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [09:36:15] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [09:37:13] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [09:37:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [09:37:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:37:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:38:33] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [09:38:53] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [09:39:28] @replag [09:39:29] liangent: s1-user: 8h 33m 0s [-0.04 s/s]; s3-rr-a: 32s [-0.01 s/s]; s3-user: 32s [-0.01 s/s]; s7-rr-a: 13s [+0.00 s/s]; s7-user: 13s [+0.00 s/s] [09:39:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:43:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:44:33] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [09:44:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:47:23] /sql on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:47:45] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:47:45] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:48:55] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 31716 MB (7% inode=99%): [09:49:13] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 63600 MB (6% inode=97%): [09:49:13] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 63600 MB (6% inode=97%): [09:53:42] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:54:23] /sql on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:54:43] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:02:39] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [10:02:39] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:02:39] APT on yarrow is CRITICAL: APT CRITICAL: 12 packages available for upgrade (12 critical updates). [10:03:28] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 31037.000000 [10:03:28] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14374.000000 [10:03:49] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 31059 [10:08:18] aliasd on mayapple is CRITICAL: NRPE: Command check_aliasd not defined [10:08:18] APT on mayapple is OK: APT OK: 0 packages available for upgrade (0 critical updates). [10:08:40] APT on nightshade is OK: APT OK: 0 packages available for upgrade (0 critical updates). [10:08:40] aliasd on yarrow is CRITICAL: NRPE: Command check_aliasd not defined [10:08:40] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [10:16:18] aliasd on mayapple is OK: TCP OK - 0.003 second response time on port 984 [500 Not found.] [10:17:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98840 MB (16% inode=99%): [10:21:41] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:30:27] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3243.000000 [10:34:28] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1135.000000 [11:01:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:02:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [11:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [11:02:18] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [11:02:30] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [11:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [11:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:03:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 30694.000000 [11:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 30687 [11:08:37] aliasd on yarrow is CRITICAL: Connection refused [11:09:48] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 32571 MB (7% inode=99%): [11:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98536 MB (16% inode=99%): [11:21:37] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:29:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:01:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:01:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [12:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [12:02:19] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [12:02:30] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [12:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [12:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:03:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 29066.000000 [12:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 29050 [12:08:37] aliasd on yarrow is CRITICAL: Connection refused [12:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98305 MB (16% inode=99%): [12:21:38] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:29:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:01:47] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:01:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [13:02:19] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [13:02:28] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [13:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [13:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:03:17] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [13:03:29] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28021.000000 [13:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 28010 [13:08:38] aliasd on yarrow is CRITICAL: Connection refused [13:12:03] hello all [13:16:45] Hi DaBPunkt ;) [13:17:37] (hmm... jetzt bin ich immer so verdammt freundlich und warte trotzdem weiter... ;) ) [13:17:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98005 MB (16% inode=99%): [13:21:38] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:21:52] jongleur: ich weiß, dass du wartest. Ich bin mir aber noch unsicher, ob ich wirklich 2 weitere jars in tomcat haben möchte – schließlich wird das dadurch nicht kleiner [13:22:38] d.h. im zweifelsfall kann ichs vergessen auf dem Toolserver oder alternativ postgis-zeugs manuell aus string parsen... [13:22:38] schade... [13:22:57] jongleur: wäre es denn möglich das postgres-jar in dein jar aufzunehmen? [13:23:09] nein, eben nicht [13:23:28] das hatte ich eigentlich vor, und hätte ich Skrupellos gemacht [13:23:55] aber dann funktioniert das postgis-zeug nicht; ich habe keine Ahnung, wieso, aber die im Ticket verlinkte Diskussion sagt das auch so [13:24:41] muss irgendwas mit den unterschiedlichen Classloadern zu tun haben, die tomcat für tomcat lib und jar-lib benutzt [13:24:46] ich dachte, dass problem wäre das die jars getrennt sind? [13:25:31] das könnte höchstens auch noch sein, aber da ich nicht beeinflussen kann, dass beide bei mir im archiv liegen (weil tomcat immer die eigene lib bevorzugt), hilft mir das auch nicht weiter, wenn ich das richtig sehe [13:26:30] stimmt... der Beitrag von Karel in der hibernate-spatial-Liste lässt beide Interpretationen zu [13:27:15] können wir natürlich auch ausprobieren, aber dann müsste die postgres-jdbc aus der tomcat-lib raus, oder? [13:27:50] okay, danke ;) weiß ich immerhin, woran es "hakt". Ich probier sonst heute abend mal weiter; muss leider los. [13:28:39] jongleur: ok. Wenn es nicht gehen sollte, nehme ich ebn die jars auf [13:29:09] DaBPunkt: kurze Frage noch dazu: postgres aus der tomcat-lib rauszunehmen ist auch eine Option? [13:29:36] (wenn ich nichts finde, um das auszuschalten, beißt sich das sonst, weil das dann zweimal geladen wird) [13:29:36] sollte gehen, ja [13:29:39] ok. nur nicht, dass das andere noch brauchen [13:29:41] gut, danke [13:29:43] bis demnächst [13:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:01:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:01:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:02:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:02:09] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [14:02:17] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [14:02:27] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [14:02:37] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [14:02:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:03:18] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [14:03:51] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 26884 [14:03:51] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26861.000000 [14:08:38] aliasd on yarrow is CRITICAL: Connection refused [14:09:37] aliasd on yarrow is OK: TCP OK - 0.001 second response time on port 984 [500 Not found.] [14:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 97730 MB (16% inode=99%): [14:21:38] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:02:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [15:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [15:02:29] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [15:02:37] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [15:02:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:03:18] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [15:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 25698 [15:04:28] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25673.000000 [15:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 97510 MB (16% inode=99%): [15:21:39] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:54:48] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 66309 MB (6% inode=97%): [15:54:58] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 66308 MB (6% inode=97%): [16:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:02:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [16:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [16:02:29] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [16:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [16:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:03:17] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [16:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 24704 [16:04:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 24698.000000 [16:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 97287 MB (16% inode=99%): [16:18:08] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:21:40] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:26:00] Hello [16:26:41] When articles coordinates will update to google earth and osm? [16:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:37:38] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [17:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:02:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [17:02:30] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:02:37] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [17:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:03:33] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:03:56] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 23416 [17:04:53] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 23371.000000 [17:17:59] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 96927 MB (15% inode=99%): [17:21:38] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:02:02] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [18:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [18:02:31] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [18:02:37] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [18:02:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:03:18] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [18:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 21309 [18:04:37] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21279.000000 [18:18:00] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 96634 MB (15% inode=99%): [18:21:49] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:30:00] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:02:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:02:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [19:02:30] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [19:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:03:17] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20740 [19:04:37] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 20763.000000 [19:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 96359 MB (15% inode=99%): [19:21:42] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:58:58] /sql on thyme is WARNING: DISK WARNING - free space: /sql 200712 MB (20% inode=99%): [20:01:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:01:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [20:02:18] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [20:02:32] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [20:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [20:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:03:18] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [20:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20438 [20:04:38] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 20465.000000 [20:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 96061 MB (15% inode=99%): [20:21:38] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:01:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:01:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:02:09] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [21:02:18] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:02:18] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [21:02:28] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [21:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [21:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:03:21] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [21:03:50] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20092 [21:04:38] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 20085.000000 [21:17:59] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 95739 MB (15% inode=99%): [21:20:22] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 93504 MB (9% inode=99%): [21:21:38] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:30:00] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:01:51] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:01:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [22:02:20] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:02:20] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [22:02:28] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [22:02:38] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [22:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:03:17] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [22:03:48] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 19662 [22:04:37] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 19668.000000 [22:18:00] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 95474 MB (15% inode=99%): [22:21:40] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:34:09] nacht ts [23:01:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:01:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:01:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:02:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [23:02:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:02:21] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [23:02:28] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [23:02:37] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [23:02:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:03:20] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [23:03:49] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 19208 [23:04:37] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 19195.000000 [23:09:53] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 31056 MB (7% inode=99%): [23:17:58] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 95253 MB (15% inode=99%): [23:21:40] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:29:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output