[00:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 355 MB (1% inode=93%): [00:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 353 MB (1% inode=93%): [00:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:14:59] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 694593.000000 [00:15:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 105723 MB (17% inode=99%): [00:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:24:08] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 342452 MB (6% inode=55%): [00:34:39] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [00:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:45:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:49:08] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 1405 MB (6% inode=99%): [01:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 390 MB (1% inode=93%): [01:00:57] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 389 MB (1% inode=93%): [01:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 696936.000000 [01:15:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 105493 MB (17% inode=99%): [01:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 343183 MB (6% inode=55%): [01:34:39] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [01:39:18] Load avg. on ortelius is WARNING: WARNING - load average: 15.79, 18.21, 14.93 [01:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:45:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:48:18] Load avg. on ortelius is OK: OK - load average: 6.62, 14.21, 14.89 [01:49:08] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 204 MB (1% inode=99%): [01:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 252 MB (0% inode=93%): [02:00:59] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 252 MB (0% inode=93%): [02:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:10:18] Load avg. on ortelius is WARNING: WARNING - load average: 24.86, 24.03, 18.86 [02:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 697959.000000 [02:15:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103556 MB (17% inode=99%): [02:18:59] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 79739 MB (8% inode=99%): [02:20:39] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:24:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 341859 MB (6% inode=55%): [02:26:18] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 23.77, 22.05, 20.25 [02:28:18] Load avg. on ortelius is WARNING: WARNING - load average: 15.74, 20.19, 19.80 [02:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [02:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:45:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:47:17] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 30.37, 23.19, 19.70 [02:49:08] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 1001 MB (4% inode=99%): [02:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 224 MB (0% inode=93%): [03:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 223 MB (0% inode=93%): [03:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:11:58] Load avg. on willow is WARNING: WARNING - load average: 18.66, 16.32, 14.88 [03:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 697645.000000 [03:15:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 105162 MB (17% inode=99%): [03:17:18] Load avg. on ortelius is WARNING: WARNING - load average: 2.41, 10.77, 19.93 [03:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:23:18] Load avg. on ortelius is OK: OK - load average: 4.25, 6.12, 14.71 [03:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339553 MB (6% inode=55%): [03:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:45:38] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:49:07] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 808 MB (4% inode=99%): [03:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 1018 MB (3% inode=93%): [04:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 1015 MB (3% inode=93%): [04:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:11:58] Load avg. on willow is WARNING: WARNING - load average: 21.76, 18.69, 17.43 [04:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 699936.000000 [04:15:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 105022 MB (17% inode=99%): [04:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 340289 MB (6% inode=55%): [04:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc: [04:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:49:08] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 1335 MB (6% inode=99%): [04:54:08] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:00:49] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 976 MB (3% inode=93%): [05:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 981 MB (3% inode=93%): [05:02:37] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:11:58] Load avg. on willow is WARNING: WARNING - load average: 16.23, 15.41, 15.46 [05:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 700409.000000 [05:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104929 MB (17% inode=99%): [05:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 340033 MB (6% inode=55%): [05:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:49:07] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 873 MB (4% inode=99%): [05:58:58] Load avg. on willow is OK: OK - load average: 13.45, 14.35, 14.96 [06:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 964 MB (3% inode=93%): [06:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 971 MB (3% inode=93%): [06:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:06:59] Load avg. on willow is WARNING: WARNING - load average: 14.77, 15.14, 15.18 [06:07:58] Load avg. on willow is OK: OK - load average: 12.96, 14.59, 14.98 [06:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 700975.000000 [06:15:49] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104809 MB (17% inode=99%): [06:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339883 MB (6% inode=55%): [06:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [06:40:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:49:07] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 800 MB (3% inode=99%): [07:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 945 MB (3% inode=93%): [07:00:59] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 944 MB (3% inode=93%): [07:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:14:59] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 701855.000000 [07:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 92763 MB (15% inode=99%): [07:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:23:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339855 MB (6% inode=55%): [07:34:39] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [07:39:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [07:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:49:07] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 1173 MB (5% inode=99%): [08:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 922 MB (3% inode=93%): [08:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 928 MB (3% inode=93%): [08:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:14:48] PostgreSQL on ptolemy is OK: OK - database osm_mapnik (0 sec.) [08:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 701926.000000 [08:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 110467 MB (18% inode=99%): [08:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339795 MB (6% inode=55%): [08:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [08:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:49:08] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 805 MB (3% inode=99%): [08:56:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [08:58:49] PostgreSQL on ptolemy is OK: OK - database osm_mapnik (0 sec.) [09:00:48] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 873 MB (2% inode=93%): [09:00:59] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 872 MB (2% inode=93%): [09:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:05:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [09:14:59] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 702652.000000 [09:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98675 MB (16% inode=99%): [09:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339659 MB (6% inode=55%): [09:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [09:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:45:49] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:49:07] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 859 MB (4% inode=99%): [10:00:49] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 800 MB (2% inode=93%): [10:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 800 MB (2% inode=93%): [10:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:05:49] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [10:14:59] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 703877.000000 [10:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 92809 MB (15% inode=99%): [10:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339340 MB (6% inode=55%): [10:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [10:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:42:48] PostgreSQL on ptolemy is OK: OK - database osm_mapnik (0 sec.) [10:44:29] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:49:08] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 752 MB (3% inode=99%): [10:49:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [11:00:49] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 723 MB (2% inode=93%): [11:00:58] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 730 MB (2% inode=93%): [11:02:37] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:14:59] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 703974.000000 [11:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103400 MB (17% inode=99%): [11:20:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:23:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 334519 MB (6% inode=55%): [11:34:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [11:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:49:08] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 437 MB (2% inode=99%): [11:49:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [12:00:49] /tmp on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 649 MB (2% inode=93%): [12:00:59] / on wolfsbane is CRITICAL: DISK CRITICAL - free space: / 646 MB (2% inode=93%): [12:01:48] /sql on cassia is WARNING: DISK WARNING - free space: /sql 125103 MB (10% inode=99%): [12:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:07:38] Sun Grid Engine execd on ortelius is UNKNOWN: CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:13:49] /sql on cassia is OK: DISK OK - free space: /sql 135776 MB (11% inode=99%): [12:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 704835.000000 [12:15:49] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 95119 MB (15% inode=99%): [12:20:49] /sql on cassia is WARNING: DISK WARNING - free space: /sql 128873 MB (10% inode=99%): [12:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 333829 MB (6% inode=55%): [12:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [12:37:08] /tmp on ortelius is OK: DISK OK - free space: /tmp 18874 MB (89% inode=99%): [12:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:43:48] /tmp on wolfsbane is OK: DISK OK - free space: / 16868 MB (56% inode=93%): [12:43:58] / on wolfsbane is OK: DISK OK - free space: / 16864 MB (56% inode=93%): [12:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:45:49] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:49:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [13:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 707149.000000 [13:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87437 MB (14% inode=99%): [13:23:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 333575 MB (6% inode=55%): [13:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:49:49] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [14:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:14:59] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 709454.000000 [14:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 80358 MB (13% inode=99%): [14:18:59] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 79891 MB (8% inode=99%): [14:23:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 332594 MB (6% inode=55%): [14:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:49:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [15:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:08:39] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 712040.000000 [15:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 75380 MB (12% inode=99%): [15:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 332129 MB (6% inode=55%): [15:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [15:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:49:48] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [15:56:58] Load avg. on willow is WARNING: WARNING - load average: 14.92, 15.49, 14.96 [15:59:58] Load avg. on willow is OK: OK - load average: 13.58, 14.85, 14.82 [16:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:14:58] Load avg. on willow is WARNING: WARNING - load average: 23.00, 23.35, 19.21 [16:14:58] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 714374.000000 [16:15:48] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 72847 MB (11% inode=99%): [16:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 331435 MB (6% inode=55%): [16:26:18] Free Memory on damiana is WARNING: WARNING - 7.0% (589648 kB) free! [16:32:18] Free Memory on damiana is OK: OK - 7.2% (599920 kB) free. [16:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [16:40:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:49:52] PostgreSQL on ptolemy is CRITICAL: CRITICAL - no connection to osm_mapnik (FATAL: remaining connection slots are reserved for non-replication superuser connections [16:51:48] /sql on ptolemy is OK: DISK OK - free space: /sql 165963 MB (27% inode=99%): [16:51:49] PostgreSQL on ptolemy is OK: OK - database osm_mapnik (0 sec.) [17:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:15:08] Load avg. on willow is WARNING: WARNING - load average: 16.48, 17.13, 16.41 [17:15:09] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 716537.000000 [17:23:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [17:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 330995 MB (6% inode=55%): [17:33:18] Free Memory on damiana is WARNING: WARNING - 5.7% (474644 kB) free! [17:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [17:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:43:17] Free Memory on damiana is OK: OK - 7.7% (642288 kB) free. [17:44:29] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:54:09] Load avg. on willow is OK: OK - load average: 13.21, 14.16, 14.89 [17:56:09] Load avg. on ptolemy is WARNING: WARNING - load average: 17.75, 16.92, 11.42 [17:59:08] Load avg. on ptolemy is OK: OK - load average: 12.09, 14.73, 11.54 [18:02:39] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:04:09] Load avg. on willow is WARNING: WARNING - load average: 24.01, 22.70, 18.75 [18:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:15:08] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 718667.000000 [18:24:09] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:24:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 330361 MB (6% inode=55%): [18:31:10] * Coren wavies [18:34:40] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [18:35:45] Does anyone know what time(s) Sike is usually around? [18:38:17] Silke* [18:40:17] Free Memory on damiana is WARNING: WARNING - 6.5% (548304 kB) free! [18:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:41:18] Free Memory on damiana is OK: OK - 8.0% (671248 kB) free. [18:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:53:09] Load avg. on willow is OK: OK - load average: 13.89, 14.55, 14.99 [19:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:15:08] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 718968.000000 [19:17:08] Load avg. on willow is WARNING: WARNING - load average: 14.13, 15.22, 15.17 [19:19:18] Load avg. on ortelius is WARNING: WARNING - load average: 19.09, 21.98, 13.86 [19:20:08] Load avg. on willow is OK: OK - load average: 14.05, 14.69, 14.96 [19:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 329544 MB (6% inode=55%): [19:27:18] Load avg. on ortelius is OK: OK - load average: 8.51, 14.43, 13.72 [19:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [19:40:39] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:53:18] Free Memory on damiana is WARNING: WARNING - 6.6% (555392 kB) free! [19:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:56:18] Free Memory on damiana is OK: OK - 7.4% (616644 kB) free. [20:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:15:08] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 720600.000000 [20:17:08] Load avg. on willow is WARNING: WARNING - load average: 15.73, 15.85, 15.34 [20:24:09] Load avg. on willow is OK: OK - load average: 13.46, 14.49, 14.91 [20:24:18] Free Memory on damiana is WARNING: WARNING - 6.8% (567228 kB) free! [20:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 329066 MB (6% inode=55%): [20:31:18] Free Memory on damiana is OK: OK - 7.2% (602992 kB) free. [20:34:39] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [20:40:18] Free Memory on damiana is WARNING: WARNING - 7.0% (588520 kB) free! [20:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:07:08] Load avg. on willow is WARNING: WARNING - load average: 14.04, 15.30, 14.97 [21:08:08] Load avg. on willow is OK: OK - load average: 13.54, 14.96, 14.88 [21:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:15:08] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 720160.000000 [21:17:09] Load avg. on willow is WARNING: WARNING - load average: 14.33, 15.32, 15.17 [21:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 328956 MB (6% inode=55%): [21:28:17] Free Memory on damiana is CRITICAL: CRITICAL - 4.4% (366164 kB) free! [21:30:18] Free Memory on damiana is WARNING: WARNING - 5.7% (479016 kB) free! [21:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [21:35:18] Free Memory on damiana is OK: OK - 7.7% (646692 kB) free. [21:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:45:49] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:47:08] Load avg. on willow is WARNING: WARNING - load average: 15.21, 15.17, 15.30 [21:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:58:08] Load avg. on willow is OK: OK - load average: 14.02, 14.64, 14.99 [22:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:07:08] Load avg. on willow is WARNING: WARNING - load average: 14.75, 15.40, 15.33 [22:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:15:09] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 720054.000000 [22:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 328086 MB (6% inode=55%): [22:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [22:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:50:08] Load avg. on willow is OK: OK - load average: 13.35, 14.29, 14.89 [22:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:02:38] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:07:08] Load avg. on willow is WARNING: WARNING - load average: 15.02, 15.82, 15.52 [23:08:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:15:08] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 718941.000000 [23:24:18] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 326824 MB (6% inode=55%): [23:30:14] Load avg. on willow is OK: OK - load average: 12.79, 14.28, 14.92 [23:34:38] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [23:40:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:42:08] Load avg. on willow is WARNING: WARNING - load average: 14.99, 15.50, 15.34 [23:44:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:45:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:53:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:59:09] Load avg. on willow is OK: OK - load average: 13.18, 14.34, 14.93