[00:05:47] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:07:56] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3609.000000 [00:10:56] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3573.000000 [00:15:55] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3605.000000 [00:21:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:35:45] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [00:35:46] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:35:59] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [00:35:59] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [00:36:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:36:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:36:27] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [00:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [00:40:46] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:40:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [01:05:45] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:16:00] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4047.000000 [01:21:45] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:21:55] Load avg. on ortelius is WARNING: WARNING - load average: 17.61, 18.87, 12.59 [01:28:56] Load avg. on ortelius is OK: OK - load average: 11.64, 14.56, 12.97 [01:35:47] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [01:35:47] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:35:55] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [01:35:55] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [01:35:55] Load avg. on ortelius is WARNING: WARNING - load average: 15.62, 20.66, 16.90 [01:36:30] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:36:30] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:36:30] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [01:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [01:40:48] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:40:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:41:46] /sql on thyme is WARNING: DISK WARNING - free space: /sql 201456 MB (20% inode=99%): [01:41:56] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3564.000000 [01:44:09] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [01:55:56] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 56.29, 32.70, 25.98 [02:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:14:55] Load avg. on ortelius is WARNING: WARNING - load average: 11.59, 16.08, 19.85 [02:18:56] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 24.00, 19.32, 20.22 [02:20:58] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1773.000000 [02:20:58] Load avg. on ortelius is WARNING: WARNING - load average: 17.04, 18.31, 19.74 [02:21:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:35:46] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [02:35:46] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:35:56] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [02:35:56] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [02:36:25] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:36:25] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:36:25] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [02:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [02:40:46] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:40:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [02:53:55] Load avg. on ortelius is OK: OK - load average: 3.32, 10.50, 14.38 [02:54:49] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1894 [02:54:56] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1896.000000 [02:55:47] /sql on thyme is OK: DISK OK - free space: /sql 202509 MB (21% inode=99%): [03:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:06:15] Load avg. on willow is WARNING: WARNING - load average: 16.34, 15.85, 14.39 [03:08:15] Load avg. on willow is OK: OK - load average: 12.21, 14.58, 14.10 [03:17:16] Load avg. on willow is WARNING: WARNING - load average: 15.43, 16.35, 15.16 [03:21:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:35:46] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [03:35:46] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:35:55] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [03:35:55] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [03:36:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:36:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:36:27] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [03:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [03:40:46] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:40:55] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [03:54:49] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2725 [03:54:55] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2731.000000 [03:56:57] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:21:45] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:35:48] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [04:35:48] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:35:56] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [04:35:56] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [04:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:36:26] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [04:37:15] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [04:40:50] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:40:55] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [04:46:25] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [04:54:47] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2583 [04:54:55] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2586.000000 [05:05:50] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:17:16] Load avg. on willow is WARNING: WARNING - load average: 14.58, 15.28, 14.32 [05:18:16] Load avg. on willow is OK: OK - load average: 12.58, 14.57, 14.13 [05:21:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:35:46] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [05:35:47] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:35:58] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [05:35:58] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [05:36:28] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:36:28] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:36:28] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [05:37:15] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [05:40:49] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:40:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [05:52:46] MySQL slave on rosemary is OK: Uptime: 467406 Threads: 11 Questions: 222931810 Slow queries: 134345 Opens: 33486 Flush tables: 1 Open tables: 3040 Queries per second avg: 476.955 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1743 [05:52:58] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1730.000000 [06:02:47] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1951 [06:02:56] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1925.000000 [06:05:48] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:07:46] MySQL slave on rosemary is OK: Uptime: 468306 Threads: 12 Questions: 223298373 Slow queries: 134652 Opens: 33532 Flush tables: 1 Open tables: 3051 Queries per second avg: 476.821 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1799 [06:07:59] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1777.000000 [06:21:48] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:31:16] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [06:31:16] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [06:31:26] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [06:35:46] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [06:35:46] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:35:56] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [06:35:56] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [06:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:36:26] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [06:37:17] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [06:40:48] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:40:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:43:40] is the auth at jira broken in a way that users (not only me) are logged out more or less immediately again? [06:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [06:52:25] Free Memory on damiana is WARNING: WARNING - 6.9% (577964 kB) free! [06:54:25] Free Memory on damiana is OK: OK - 7.2% (599412 kB) free. [07:01:57] is there any way to get https://jira.toolserver.org/browse/TS-1550 resolved? it's just copying three jars to the tomcat lib :( [07:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:13:49] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 127206 MB (20% inode=99%): [07:21:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:31:15] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [07:31:15] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [07:31:25] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [07:35:50] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [07:35:50] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:35:56] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [07:35:56] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [07:36:25] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:36:25] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [07:37:18] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [07:40:46] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:40:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [08:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:13:46] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126976 MB (20% inode=99%): [08:21:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:31:16] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [08:31:16] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [08:31:26] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [08:35:45] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [08:35:46] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:35:55] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [08:35:55] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [08:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:36:26] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [08:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [08:40:49] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:40:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [09:05:45] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:13:46] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126727 MB (20% inode=99%): [09:18:56] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 97872 MB (10% inode=99%): [09:21:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:28:46] /sql on thyme is WARNING: DISK WARNING - free space: /sql 201420 MB (20% inode=99%): [09:31:16] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [09:31:16] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [09:31:25] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [09:35:56] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [09:35:56] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:35:56] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [09:35:56] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [09:36:25] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:36:25] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [09:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [09:40:46] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:40:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:44:05] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [10:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:13:46] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126394 MB (20% inode=99%): [10:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:22:29] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 34126 MB (8% inode=99%): [10:23:06] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 66663 MB (6% inode=97%): [10:23:06] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 66663 MB (6% inode=97%): [10:27:36] Free Memory on damiana is CRITICAL: CRITICAL - 3.8% (319564 kB) free! [10:31:16] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [10:31:16] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [10:31:26] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [10:31:36] Free Memory on damiana is WARNING: WARNING - 6.1% (508368 kB) free! [10:34:36] Free Memory on damiana is OK: OK - 7.6% (636608 kB) free. [10:36:07] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [10:36:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:36:07] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [10:36:07] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [10:36:29] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:36:29] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [10:36:36] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [10:41:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:44:07] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [11:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126054 MB (20% inode=99%): [11:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:29:06] /sql on thyme is OK: DISK OK - free space: /sql 202119 MB (21% inode=99%): [11:31:18] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [11:31:18] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [11:31:25] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [11:34:59] siebrand: ping? [11:35:13] what? [11:36:06] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [11:36:25] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [11:36:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:36:42] siebrand: i just wanted to ask if you think that the commonsdelinker fix will be difficult - we plan to have some imagevios deleted so we would rather do it after it's fixed [11:37:06] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:37:06] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [11:37:06] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [11:37:06] to prevent incorrect display of touched articles [11:37:07] I have no idea what "the cd fix" is… [11:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [11:37:24] siebrand: you got it on your commons talk page ;-) [11:37:30] (I also have no intention of making a CD fix. [11:37:46] Feel free to find a volunteer to fix it. [11:38:01] All the code is in subversion; has been for years. [11:38:12] ah, you no longer run it? [11:39:08] I do. [11:39:12] do-ish. [11:39:23] I never made code changes. [11:39:47] otoh i think it's actually change in mw behavior - iianm, it used to trim leading and ending whitespaces on save [11:39:52] I'm busy. Can't continue talking now. [11:39:56] oki [11:41:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [11:55:35] Free Memory on damiana is WARNING: WARNING - 6.3% (524408 kB) free! [12:04:36] Free Memory on damiana is OK: OK - 7.1% (598008 kB) free. [12:05:47] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 125792 MB (20% inode=99%): [12:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:30:37] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:31:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:31:15] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [12:31:26] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [12:31:26] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [12:36:09] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [12:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [12:36:38] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:36:38] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:37:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:37:08] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [12:37:08] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [12:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [12:40:35] Sun Grid Engine execd on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:44:07] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [12:59:10] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:05:48] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 125496 MB (20% inode=99%): [13:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:26:36] Free Memory on damiana is WARNING: WARNING - 6.5% (540628 kB) free! [13:28:35] Free Memory on damiana is CRITICAL: CRITICAL - 4.4% (370144 kB) free! [13:29:36] Free Memory on damiana is WARNING: WARNING - 5.2% (437988 kB) free! [13:31:16] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [13:31:26] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [13:31:26] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [13:36:07] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [13:36:25] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:36:25] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [13:36:36] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:37:06] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:37:06] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [13:37:06] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [13:37:18] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [13:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [13:49:01] hello all [13:49:09] is somebody here who speaks c? [13:59:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:14:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 125202 MB (20% inode=99%): [14:21:11] DaB. * Re: [Toolserver-l] Mail forwarding not working [14:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:31:16] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [14:31:26] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [14:31:26] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [14:36:07] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [14:36:12] @replag [14:36:12] DaBPunkt: s1-user: 6m 12s [+0.03 s/s]; s3-rr-a: 2m 7s [+0.01 s/s]; s3-user: 2m 7s [+0.01 s/s] [14:36:15] APT on mayapple is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:36:26] APT on nightshade is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:36:26] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [14:36:36] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:37:06] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:37:06] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [14:37:06] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [14:37:19] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [14:41:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [14:59:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:05:35] Free Memory on damiana is WARNING: WARNING - 6.6% (555964 kB) free! [15:05:45] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:13:36] Free Memory on damiana is OK: OK - 7.2% (606768 kB) free. [15:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 124905 MB (20% inode=99%): [15:22:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:36:06] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [15:36:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:36:27] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [15:36:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:37:11] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:37:11] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [15:37:11] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [15:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [15:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:44:09] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [15:59:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:05:47] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:14:06] /sql on thyme is WARNING: DISK WARNING - free space: /sql 201452 MB (20% inode=99%): [16:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 124637 MB (20% inode=99%): [16:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:27:37] Free Memory on damiana is WARNING: WARNING - 5.2% (434500 kB) free! [16:28:39] Free Memory on damiana is CRITICAL: CRITICAL - 4.2% (352416 kB) free! [16:29:36] Free Memory on damiana is WARNING: WARNING - 5.7% (473932 kB) free! [16:36:06] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [16:36:25] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:36:25] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [16:36:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:37:06] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:37:06] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [16:37:06] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [16:37:17] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [16:41:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:44:08] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [16:59:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:05:48] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 124343 MB (20% inode=99%): [17:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:25:38] Free Memory on damiana is WARNING: WARNING - 6.6% (550656 kB) free! [17:28:35] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (420536 kB) free! [17:29:36] Free Memory on damiana is WARNING: WARNING - 6.0% (503668 kB) free! [17:36:06] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:36:28] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:36:28] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [17:36:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:37:06] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:37:06] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:37:06] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [17:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:44:10] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [17:59:10] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:05:45] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:09:35] Free Memory on damiana is WARNING: WARNING - 6.4% (536640 kB) free! [18:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 123984 MB (20% inode=99%): [18:18:37] Free Memory on damiana is OK: OK - 7.2% (604660 kB) free. [18:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:31:07] /sql on thyme is OK: DISK OK - free space: /sql 202043 MB (21% inode=99%): [18:34:36] Free Memory on damiana is WARNING: WARNING - 6.7% (557580 kB) free! [18:36:08] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [18:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [18:36:36] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:37:06] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:37:06] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [18:37:06] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [18:37:19] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [18:39:35] Free Memory on damiana is OK: OK - 7.1% (596668 kB) free. [18:41:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [18:59:10] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:05:45] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 123663 MB (20% inode=99%): [19:22:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:36:06] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [19:36:36] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:37:11] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:37:11] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:37:11] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [19:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:44:09] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [19:59:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:05:45] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 123309 MB (20% inode=99%): [20:22:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:36:07] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [20:36:39] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:36:39] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [20:36:39] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:37:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:37:08] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [20:37:08] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [20:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [20:41:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [20:53:36] Free Memory on damiana is WARNING: WARNING - 6.1% (511092 kB) free! [20:59:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:05:45] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:08:36] Free Memory on damiana is OK: OK - 7.1% (592060 kB) free. [21:14:08] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 122958 MB (20% inode=99%): [21:19:10] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 97054 MB (9% inode=99%): [21:22:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:36:07] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [21:36:26] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:36:26] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [21:36:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:37:09] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:37:09] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [21:37:09] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [21:37:15] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [21:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [21:59:10] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 122624 MB (20% inode=99%): [22:22:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:22:26] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 32599 MB (8% inode=99%): [22:23:06] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 67133 MB (6% inode=98%): [22:23:07] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 67133 MB (6% inode=98%): [22:36:06] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [22:36:29] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:36:29] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [22:36:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:37:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:37:07] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [22:37:07] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [22:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [22:41:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:44:06] /sql on thyme is WARNING: DISK WARNING - free space: /sql 201454 MB (20% inode=99%): [22:44:07] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [22:59:10] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:05:46] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:14:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 122411 MB (20% inode=99%): [23:15:41] nacht ts [23:22:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:30:38] Free Memory on damiana is WARNING: WARNING - 6.6% (552320 kB) free! [23:34:35] Free Memory on damiana is OK: OK - 7.2% (600944 kB) free. [23:36:06] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [23:36:30] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:36:30] DiskSuite on damiana is CRITICAL: CRITICAL - submirror d52 of mirror d50 is Needs and submirror d32 of mirror d30 is Needs and submirror d12 of mirror d10 is Needs and submirror d22 of mirror d20 is Needs [23:36:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:37:06] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:37:06] MySQL slave on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [23:37:07] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [23:37:16] s4 replag on z-dat-s4-a is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [23:41:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:44:06] CAM on hemlock is WARNING: WARNING - Storage ts-array5 (2 warnings): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.A:S3: 34:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_BATTERY_NEAR_EXPIRATION.description:S17:Tray.85.Battery.B:S3: 42: [23:52:08] @replag [23:52:09] Merlissimo: s2-user: 26s [+0.00 s/s]; s3-rr-a: 16s [+0.00 s/s]; s3-user: 16s [+0.00 s/s] [23:59:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output