[00:00:07] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [00:06:34] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 95615.000000 [00:06:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:07:58] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [00:08:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 229972 MB (4% inode=67%): [00:11:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:11:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:11:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103792 MB (17% inode=99%): [00:11:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:12:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:19:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 66634.000000 [00:21:57] wikidata replag on z-dat-s6-a is OK: QUERY OK: SELECT ts_rc_age() returned 1753.000000 [00:26:07] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 55454 MB (5% inode=99%): [00:26:37] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 50855 [00:36:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:49:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39682.000000 [00:51:57] APT on mayapple is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [00:55:17] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [00:58:47] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [00:58:47] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [01:00:07] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [01:06:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 97963.000000 [01:06:48] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:07:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [01:08:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 230309 MB (4% inode=67%): [01:11:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:11:20] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:11:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103536 MB (17% inode=99%): [01:11:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:12:28] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:19:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 67346.000000 [01:26:37] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 51661 [01:36:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:49:26] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43283.000000 [01:51:57] APT on mayapple is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [01:55:17] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [01:58:47] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [01:58:47] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [02:00:07] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [02:06:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 100788.000000 [02:06:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:07:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [02:08:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 230372 MB (4% inode=67%): [02:11:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:11:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:11:36] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 101443 MB (16% inode=99%): [02:11:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:12:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:19:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 65579.000000 [02:26:37] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 50377 [02:27:57] Load avg. on ortelius is WARNING: WARNING - load average: 18.55, 19.58, 15.07 [02:34:57] Load avg. on ortelius is OK: OK - load average: 8.43, 14.14, 14.35 [02:36:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:49:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46883.000000 [02:51:57] APT on mayapple is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [02:55:17] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [02:58:47] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [02:58:47] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [03:00:07] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [03:06:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 103355.000000 [03:06:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:07:58] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:08:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 229711 MB (4% inode=67%): [03:11:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:11:18] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:11:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103114 MB (16% inode=99%): [03:11:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:12:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:19:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 62443.000000 [03:26:38] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 48190 [03:36:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:49:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50483.000000 [03:52:57] APT on mayapple is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [03:55:18] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [03:58:46] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [03:58:48] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [04:00:07] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [04:06:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 105422.000000 [04:06:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:07:18] Free Memory on damiana is CRITICAL: CRITICAL - 4.1% (343956 kB) free! [04:07:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [04:08:58] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 229756 MB (4% inode=67%): [04:11:17] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:11:27] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:11:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 102921 MB (16% inode=99%): [04:11:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:12:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:19:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61489.000000 [04:32:52] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 46485 [04:36:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:37:11] Free Memory on damiana is OK: OK - 82.9% (6946752 kB) free. [04:49:51] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54102.000000 [04:53:17] APT on mayapple is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [04:55:56] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [04:56:25] @replag [04:56:25] Betacommand: s1-rr-a-wd: 51s [-]; s1-user: 15h 8m 30s [-]; s1-user-wd: 53s [-]; s2-rr: 12h 29m 38s [-]; s2-user: 12h 29m 38s [-]; s2-user-c: error; s2-user-wd: 17h 5m 27s [-]; s3-user-wd: 9m 47s [-] [04:56:27] Betacommand: s4-user-wd: 1d 5h 43m 16s [-]; s5-user-c: error; s6-user: 12s [-]; s6-user-wd: 28m 11s [-]; s7-user-wd: 51s [-] [04:58:52] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [04:58:52] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [05:00:06] John * [Toolserver-l] database lag issues [05:00:11] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [05:05:11] wikidata replag on z-dat-s6-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1976.000000 [05:06:51] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 107084.000000 [05:07:36] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:08:12] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:09:12] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 229296 MB (4% inode=67%): [05:11:52] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 102753 MB (16% inode=99%): [05:11:52] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:11:52] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:11:58] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:12:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:19:11] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 60678.000000 [05:33:50] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 44003 [05:36:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:43:46] [[Special:Log/newusers]] create 10 * Gadgetmania * (New user account) [05:50:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57763.000000 [05:53:11] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [05:53:19] APT on mayapple is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [05:56:51] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [05:58:51] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [05:58:51] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [06:00:19] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [06:02:11] wikidata replag on z-dat-s6-a is OK: QUERY OK: SELECT ts_rc_age() returned 1736.000000 [06:07:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:07:49] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 107833.000000 [06:09:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [06:10:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 229126 MB (4% inode=67%): [06:11:59] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:12:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:12:51] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 102600 MB (16% inode=99%): [06:12:51] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:12:51] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:19:12] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 59936.000000 [06:31:17] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [06:33:51] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 45236 [06:36:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:50:51] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61363.000000 [06:53:11] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [06:53:18] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [06:56:50] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [06:58:51] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [06:58:57] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:00:18] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:07:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:07:49] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 109045.000000 [07:10:10] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [07:11:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 228905 MB (4% inode=67%): [07:11:58] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:12:51] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 102419 MB (16% inode=99%): [07:12:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:12:51] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:12:51] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:19:11] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 58824.000000 [07:31:17] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:33:51] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 44583 [07:36:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:50:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64962.000000 [07:53:18] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:53:18] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [07:56:57] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:58:58] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:59:51] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [08:00:19] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [08:07:50] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:07:50] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 108338.000000 [08:10:10] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [08:11:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 228628 MB (4% inode=67%): [08:11:57] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:12:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:12:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:13:50] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 102255 MB (16% inode=99%): [08:13:50] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:16:50] Load avg. on nightshade is WARNING: WARNING - load average: 16.41, 17.94, 14.05 [08:20:06] hm [08:20:09] * md_5 can't access phpmyadmin [08:20:10] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 56219.000000 [08:20:15] keeps rejecting my password [08:20:52] I can log in to jira, but not pma [08:30:50] Load avg. on nightshade is CRITICAL: CRITICAL - load average: 31.42, 20.57, 17.28 [08:31:18] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [08:31:49] Load avg. on nightshade is WARNING: WARNING - load average: 20.50, 19.35, 17.06 [08:34:50] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 43265 [08:36:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:37:50] Load avg. on nightshade is OK: OK - load average: 8.59, 12.45, 14.84 [08:50:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68563.000000 [08:53:18] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [08:53:18] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [08:56:58] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [08:59:00] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [08:59:49] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [09:00:17] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [09:05:50] Load avg. on nightshade is WARNING: WARNING - load average: 11.84, 15.13, 14.62 [09:06:50] Load avg. on nightshade is OK: OK - load average: 9.55, 13.89, 14.22 [09:07:58] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:08:49] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 109681.000000 [09:10:10] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [09:11:58] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:12:10] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 228286 MB (4% inode=67%): [09:12:59] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:13:49] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:13:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:14:50] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 101995 MB (16% inode=99%): [09:20:10] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 52893.000000 [09:31:18] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [09:34:50] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 44820 [09:36:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:51:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 72222.000000 [09:53:17] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [09:53:18] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [09:56:57] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [09:58:58] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [09:59:50] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [10:00:18] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [10:07:58] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:08:50] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 111299.000000 [10:10:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [10:11:58] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:12:10] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 227993 MB (4% inode=67%): [10:13:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:14:51] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 101702 MB (16% inode=99%): [10:14:51] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:14:51] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:20:10] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 51306.000000 [10:31:18] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [10:34:50] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 45326 [10:36:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:51:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75823.000000 [10:53:19] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [10:53:19] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [10:56:58] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [10:58:58] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [10:59:50] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [11:00:19] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [11:07:58] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:08:50] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 113277.000000 [11:11:18] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [11:11:58] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:12:10] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 227410 MB (4% inode=67%): [11:13:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:14:50] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 101457 MB (16% inode=99%): [11:15:51] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:15:51] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:20:10] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 48981.000000 [11:31:19] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [11:32:09] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 9.62, 30.07, 19.75 [11:34:10] Load avg. on ortelius is WARNING: WARNING - load average: 9.94, 22.79, 18.24 [11:34:50] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 40271 [11:39:10] Free Memory on turnera is WARNING: WARNING - 5.2% (433016 kB) free! [11:41:10] Free Memory on turnera is CRITICAL: CRITICAL - 4.4% (369116 kB) free! [11:42:10] Load avg. on ortelius is OK: OK - load average: 3.60, 10.53, 14.26 [11:49:10] Free Memory on turnera is WARNING: WARNING - 5.2% (435216 kB) free! [11:51:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79423.000000 [11:54:18] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [11:54:19] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [11:57:00] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [11:59:01] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [11:59:50] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [12:00:29] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [12:06:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 115825.000000 [12:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [12:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:12:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 223701 MB (4% inode=67%): [12:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 101196 MB (16% inode=99%): [12:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47153.000000 [12:27:04] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 53365 MB (5% inode=99%): [12:32:14] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [12:35:34] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 33977 [12:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 83059.000000 [12:54:54] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [12:55:04] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [13:01:05] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [13:01:14] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [13:01:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [13:01:44] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [13:06:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:09:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 118858.000000 [13:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [13:12:45] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:12:55] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 222348 MB (4% inode=67%): [13:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100884 MB (16% inode=99%): [13:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 48199.000000 [13:32:14] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [13:35:34] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 33720 [13:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 86660.000000 [13:54:54] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [13:55:04] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [14:01:04] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [14:01:14] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [14:01:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [14:01:44] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [14:06:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 121502.000000 [14:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [14:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:12:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 220516 MB (4% inode=67%): [14:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100674 MB (16% inode=99%): [14:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46616.000000 [14:21:14] / on ortelius is WARNING: DISK WARNING - free space: / 6238 MB (20% inode=92%): [14:32:14] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [14:35:34] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 27732 [14:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 90262.000000 [14:54:54] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [14:55:04] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [15:01:04] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [15:01:15] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [15:01:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [15:01:44] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [15:06:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 121454.000000 [15:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [15:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:12:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 219942 MB (4% inode=67%): [15:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100392 MB (16% inode=99%): [15:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46614.000000 [15:21:15] / on ortelius is WARNING: DISK WARNING - free space: / 5724 MB (19% inode=92%): [15:32:14] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [15:35:34] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 23664 [15:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 93861.000000 [15:54:54] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [15:55:04] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [16:01:04] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [16:01:14] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [16:01:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [16:01:44] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [16:06:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 119198.000000 [16:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [16:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:12:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 219476 MB (4% inode=67%): [16:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:15:35] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 100150 MB (16% inode=99%): [16:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47378.000000 [16:21:14] / on ortelius is WARNING: DISK WARNING - free space: / 5147 MB (17% inode=92%): [16:32:14] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [16:35:34] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 23830 [16:47:17] Hello all [16:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 97463.000000 [16:54:54] APT on mayapple is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [16:55:04] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [16:59:56] @replag [16:59:57] DaBPunkt: s1-rr-a-wd: 31s [-]; s1-user: 1d 3h 12m 1s [-]; s1-user-c: 1m 27s [-]; s1-user-wd: 3m 51s [-]; s2-rr: 6h 4m 1s [-]; s2-user: 6h 4m 1s [-]; s2-user-c: error; s2-user-wd: 12h 49m 33s [-] [16:59:58] DaBPunkt: s3-user-wd: 30s [-]; s4-user-wd: 1d 8h 43m 39s [-]; s5-user-c: error; s6-user-wd: 30s [-]; s7-user-wd: 30s [-] [17:00:18] strange combination… [17:01:04] APT on nightshade is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [17:01:14] APT on yucca is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [17:01:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [17:01:44] APT on yarrow is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [17:04:54] O_o why do I suddenly get my TS mails on a different address... [17:05:28] do the LDAP records also store a mail address? [17:07:04] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [17:07:20] I received one cron output this morning @arctus.nl, but a query killer report @gmail.com [17:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 117842.000000 [17:09:59] hmyes, my LDAP one is @gmail.com [17:11:55] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [17:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:12:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 218530 MB (4% inode=67%): [17:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99819 MB (16% inode=99%): [17:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:16:48] @replag [17:16:49] Betacommand: s1-rr-a-wd: 24s [-0.01 s/s]; s1-user: 1d 3h 28m 53s [+1.00 s/s]; s1-user-c: 45s [-0.04 s/s]; s1-user-wd: 9m 17s [+0.32 s/s]; s2-rr: 6h 3m 54s [-0.01 s/s]; s2-user: 6h 3m 54s [-0.01 s/s]; s2-user-c: error; s2-user-wd: 12h 49m 9s [-0.02 s/s] [17:16:50] Betacommand: s3-user-wd: 22s [-0.01 s/s]; s4-rr-a: 40s [-]; s4-user: 40s [-]; s4-user-wd: 1d 8h 39m 25s [-0.25 s/s]; s5-user-c: error; s6-user-wd: 28s [-0.00 s/s]; s7-user: 30s [-]; s7-user-wd: 30s [-] [17:17:02] DaBPunkt: any reason for the lag issues? [17:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46326.000000 [17:21:14] / on ortelius is WARNING: DISK WARNING - free space: / 4632 MB (15% inode=92%): [17:23:34] Betacommand: rosemary's enwiki-copy had a dublicated row. [17:32:14] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [17:35:35] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 22476 [17:44:44] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:44:54] APT on mayapple is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:45:04] APT on nightshade is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:45:14] APT on yucca is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:45:15] APT on z-dat-s2-b is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:46:04] APT on z-dat-s5-b is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:52:23] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 99675.000000 [18:01:44] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 99449 [18:03:34] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 1779 MB (9% inode=99%): [18:07:34] /tmp on ortelius is WARNING: DISK WARNING - free space: /tmp 2830 MB (15% inode=99%): [18:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 112005.000000 [18:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [18:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:12:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 217686 MB (4% inode=67%): [18:14:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:15:10] [[Special:Log/newusers]] create 10 * TURIASONA * (New user account) [18:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99603 MB (16% inode=99%): [18:16:03] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:16:13] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:19:35] /tmp on ortelius is CRITICAL: DISK CRITICAL - free space: /tmp 1765 MB (9% inode=99%): [18:21:05] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46674.000000 [18:21:13] / on ortelius is WARNING: DISK WARNING - free space: / 4056 MB (13% inode=92%): [18:35:34] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 21740 [18:36:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 98456.000000 [19:01:45] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 98204 [19:07:35] /tmp on ortelius is OK: DISK OK - free space: /tmp 10631 MB (59% inode=99%): [19:08:45] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 104134.000000 [19:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [19:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:12:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 216967 MB (4% inode=67%): [19:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99277 MB (16% inode=99%): [19:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 45565.000000 [19:21:14] / on ortelius is WARNING: DISK WARNING - free space: / 3493 MB (11% inode=92%): [19:35:34] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 22645 [19:36:59] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:41:14] / on ortelius is CRITICAL: DISK CRITICAL - free space: / 3267 MB (10% inode=92%): [19:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 96444.000000 [20:01:44] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 95838 [20:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:09:24] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 98965.000000 [20:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [20:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:12:54] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 216433 MB (4% inode=67%): [20:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 99040 MB (16% inode=99%): [20:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 44890.000000 [20:34:14] / on ortelius is WARNING: DISK WARNING - free space: / 3335 MB (11% inode=92%): [20:35:33] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 22307 [20:37:00] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:38:14] / on ortelius is OK: DISK OK - free space: / 7099 MB (23% inode=92%): [20:40:07] @replag [20:40:08] DaBPunkt: s1-rr-a-wd: 18s [-0.00 s/s]; s1-user: 1d 2h 16m 35s [-0.36 s/s]; s1-user-wd: 17s [-0.04 s/s]; s2-rr: 6h 10m 1s [+0.03 s/s]; s2-user: 6h 10m 1s [+0.03 s/s]; s2-user-c: error; s2-user-wd: 12h 25m 19s [-0.12 s/s]; s3-user: 10s [-] [20:40:10] DaBPunkt: s3-user-wd: 21s [-0.00 s/s]; s4-user-wd: 1d 2h 55m 5s [-1.69 s/s]; s5-user: 5m 48s [-]; s5-user-c: error; s6-user-wd: 23s [-0.00 s/s]; s7-user-wd: 22s [-0.00 s/s] [20:46:14] Free Memory on damiana is WARNING: WARNING - 5.6% (466164 kB) free! [20:52:24] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 94071.000000 [20:54:15] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (396412 kB) free! [20:58:14] Free Memory on damiana is WARNING: WARNING - 6.8% (568304 kB) free! [20:59:54] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 32.60, 48.82, 29.75 [21:01:44] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93974 [21:07:53] Load avg. on ortelius is WARNING: WARNING - load average: 3.52, 12.97, 19.17 [21:08:44] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:09:23] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 97222.000000 [21:11:13] Free Memory on damiana is OK: OK - 8.1% (674804 kB) free. [21:11:54] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [21:12:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:12:54] Load avg. on ortelius is OK: OK - load average: 4.81, 7.55, 14.99 [21:13:03] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 216134 MB (4% inode=67%): [21:14:24] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:15:34] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98813 MB (16% inode=99%): [21:16:04] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:16:14] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:21:04] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43875.000000 [21:35:53] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [21:52:57] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 95745.000000 [22:01:59] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 95412 [22:07:14] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:09:42] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:09:57] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 98681.000000 [22:11:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [22:12:57] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:13:06] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 215615 MB (4% inode=67%): [22:19:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:19:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98420 MB (16% inode=99%): [22:19:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:19:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:21:16] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 45676.000000 [22:35:57] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17991 [22:57:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 94033.000000 [23:02:57] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93650 [23:09:57] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 99464.000000 [23:10:16] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:12:04] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [23:13:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:13:15] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 214413 MB (4% inode=67%): [23:15:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:16:45] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:16:57] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 98185 MB (16% inode=99%): [23:16:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:21:15] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 45535.000000 [23:35:58] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 12752 [23:36:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:53:57] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 89837.000000