[00:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [00:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:15:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 232267 MB (4% inode=25%): [00:28:08] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 75297 MB (12% inode=99%): [00:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [01:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:14:59] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:15:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 232097 MB (4% inode=25%): [01:28:08] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 75155 MB (12% inode=99%): [01:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [02:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:11:49] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 86556 MB (8% inode=99%): [02:14:59] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:17:38] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:26:30] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 231194 MB (4% inode=25%): [02:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 73213 MB (12% inode=99%): [02:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:55:20] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:05:30] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [03:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:15:00] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:15:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 231786 MB (4% inode=25%): [03:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 74826 MB (12% inode=99%): [03:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [04:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:14:59] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:17:38] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 232095 MB (4% inode=25%): [04:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 74512 MB (12% inode=99%): [04:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:55:18] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:14:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 231748 MB (4% inode=25%): [05:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 74288 MB (12% inode=99%): [05:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [06:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:14:59] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:15:10] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:17:38] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 231573 MB (4% inode=25%): [06:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 74193 MB (12% inode=99%): [06:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [07:06:20] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:14:59] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:15:10] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 231318 MB (4% inode=25%): [07:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 74086 MB (12% inode=99%): [07:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [08:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:15:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 231023 MB (4% inode=25%): [08:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 73957 MB (12% inode=99%): [08:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [09:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:15:59] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:24:49] fisheye.toolserver.org on web.amaranth is WARNING: HTTP WARNING: HTTP/1.1 200 OK - 273 bytes in 16.529 second response time [09:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 230471 MB (4% inode=25%): [09:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 73810 MB (12% inode=99%): [09:34:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [09:36:49] fisheye.toolserver.org on web.amaranth is WARNING: HTTP WARNING: HTTP/1.1 200 OK - 274 bytes in 17.646 second response time [09:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [10:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [10:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 229794 MB (4% inode=25%): [10:27:49] /sql on cassia is WARNING: DISK WARNING - free space: /sql 87097 MB (7% inode=99%): [10:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 73637 MB (12% inode=99%): [10:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [11:05:30] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [11:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:15:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:16:10] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:17:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 229121 MB (4% inode=25%): [11:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 73435 MB (12% inode=99%): [11:37:39] Sun Grid Engine execd on ortelius is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:38:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:43:39] Sun Grid Engine execd on ortelius is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:46:09] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:46:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [12:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [12:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:26:33] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 228490 MB (4% inode=24%): [12:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 73227 MB (12% inode=99%): [12:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [13:05:30] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [13:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 227882 MB (4% inode=24%): [13:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 73026 MB (11% inode=99%): [13:47:38] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:59:09] Free Memory on damiana is WARNING: WARNING - 7.0% (583432 kB) free! [14:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [14:04:09] Free Memory on damiana is OK: OK - 7.1% (596216 kB) free. [14:05:30] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [14:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:11:49] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 86100 MB (8% inode=99%): [14:16:10] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 227300 MB (4% inode=24%): [14:28:10] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 72793 MB (11% inode=99%): [14:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [15:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [15:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 226786 MB (4% inode=24%): [15:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 72577 MB (11% inode=99%): [15:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [16:05:36] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [16:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:16:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 226132 MB (4% inode=24%): [16:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 72377 MB (11% inode=99%): [16:47:38] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:55:18] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [17:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [17:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:16:10] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 225836 MB (4% inode=24%): [17:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 72168 MB (11% inode=99%): [17:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:51:10] Free Memory on damiana is WARNING: WARNING - 6.9% (575244 kB) free! [17:54:00] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [17:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:56:09] Free Memory on damiana is OK: OK - 7.1% (598608 kB) free. [18:02:51] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [18:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B: [18:06:20] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:26:30] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 225364 MB (4% inode=24%): [18:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 71957 MB (11% inode=99%): [18:47:40] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [19:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [19:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 224877 MB (4% inode=24%): [19:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 71737 MB (11% inode=99%): [19:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:55:19] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:02:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [20:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [20:06:19] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:26:30] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 224374 MB (4% inode=24%): [20:28:10] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 71516 MB (11% inode=99%): [20:47:40] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:55:30] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:02:59] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [21:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [21:06:29] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:09:10] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:16:10] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 223662 MB (4% inode=24%): [21:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 71255 MB (11% inode=99%): [21:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:55:29] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:02:59] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [22:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [22:06:29] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:12:49] fisheye.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 272 bytes in 8.992 second response time [22:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 213579 MB (4% inode=23%): [22:27:50] /sql on cassia is WARNING: DISK WARNING - free space: /sql 88136 MB (7% inode=99%): [22:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 71011 MB (11% inode=99%): [22:34:09] Free Memory on damiana is WARNING: WARNING - 6.6% (554448 kB) free! [22:35:09] Free Memory on damiana is OK: OK - 8.3% (695196 kB) free. [22:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:53:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:55:29] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:05:29] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [23:06:29] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:09:09] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:16:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:26:29] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 217573 MB (4% inode=24%): [23:28:09] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 70866 MB (11% inode=99%): [23:47:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:55:29] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7