[00:02:53] Free Memory on turnera is OK: OK - 92.0% (7706252 kB) free. [00:12:32] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [00:12:42] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [00:28:12] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 55364 MB (5% inode=99%): [00:28:32] MySQL on z-dat-s5-b is CRITICAL: Cant connect to MySQL server on z-dat-s5-b (146) [00:28:33] MySQL on z-dat-s2-b is CRITICAL: Cant connect to MySQL server on z-dat-s2-b (146) [00:29:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on z-dat-s5-b (146) [00:29:32] MySQL on z-dat-s5-b is OK: Uptime: 125 Threads: 2 Questions: 153 Slow queries: 0 Opens: 46 Flush tables: 1 Open tables: 37 Queries per second avg: 1.224 [00:29:32] MySQL slave on z-dat-s5-b is OK: Uptime: 131 Threads: 1 Questions: 654 Slow queries: 0 Opens: 63 Flush tables: 1 Open tables: 54 Queries per second avg: 4.992 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1762 [00:30:02] wikidata replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1741.000000 [00:31:32] MySQL on z-dat-s2-b is OK: Uptime: 257 Threads: 4 Questions: 98 Slow queries: 0 Opens: 52 Flush tables: 1 Open tables: 43 Queries per second avg: 0.381 [00:32:22] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1928 [00:37:54] nacht ts [00:44:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:44:32] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:44:42] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7873.000000 [00:44:42] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 92610 MB (15% inode=99%): [00:44:52] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 81892.000000 [00:45:02] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [00:45:02] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 199388 MB (3% inode=67%): [00:45:21] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:45:31] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 23902.000000 [00:45:32] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 111996.000000 [00:45:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:45:52] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:45:53] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 23871 [00:45:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:46:22] MySQL slave on z-dat-s2-b is OK: Uptime: 1140 Threads: 6 Questions: 419348 Slow queries: 16 Opens: 3942 Flush tables: 1 Open tables: 256 Queries per second avg: 367.849 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1787 [01:44:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:44:42] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4464.000000 [01:44:42] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 91597 MB (15% inode=99%): [01:44:53] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 84415.000000 [01:45:02] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [01:45:02] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 199202 MB (3% inode=67%): [01:45:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:45:32] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 20510.000000 [01:45:32] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 112873.000000 [01:45:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:45:52] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:45:52] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20466 [01:45:52] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:02:42] wikidata replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3579.000000 [02:14:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:29:42] wikidata replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1771.000000 [02:44:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:44:42] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 92298 MB (15% inode=99%): [02:44:52] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82785.000000 [02:45:02] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [02:45:02] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 196772 MB (3% inode=67%): [02:45:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:45:32] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18176.000000 [02:45:33] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 114762.000000 [02:45:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:45:52] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:45:53] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 18181 [02:45:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:14:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:44:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:44:42] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 92167 MB (15% inode=99%): [03:44:52] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 78255.000000 [03:45:02] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:45:02] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 196373 MB (3% inode=67%): [03:45:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:45:32] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15064.000000 [03:45:32] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 116037.000000 [03:45:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:45:53] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:45:53] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 15045 [03:45:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:53:42] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:54:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:59:42] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:14:13] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:19:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:23:42] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:24:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:26:42] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:27:02] Sun Grid Engine execd on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:27:42] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:44:42] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 91981 MB (15% inode=99%): [04:44:52] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 78256.000000 [04:45:01] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [04:45:02] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 196117 MB (3% inode=67%): [04:45:21] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:45:32] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14448.000000 [04:45:32] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 115581.000000 [04:45:52] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:45:52] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 14446 [04:45:52] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:06:02] Sun Grid Engine execd on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:07:42] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:14:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:28:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:44:42] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 91820 MB (15% inode=99%): [05:44:52] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 74954.000000 [05:45:03] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:45:03] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 195845 MB (3% inode=67%): [05:45:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:45:32] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 12004.000000 [05:45:32] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 115367.000000 [05:45:52] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:45:52] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 11984 [05:45:52] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:56:22] Free Memory on damiana is CRITICAL: CRITICAL - 3.9% (330136 kB) free! [06:07:51] Free Memory on damiana is OK: OK - 92.0% (7710396 kB) free. [06:07:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:28:59] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:44:08] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:44:49] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 91707 MB (15% inode=99%): [06:45:09] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 195071 MB (3% inode=67%): [06:45:49] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:45:50] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 116449.000000 [06:45:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4009.000000 [06:45:50] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75530.000000 [06:46:00] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [06:46:19] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:46:49] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3920 [06:46:50] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:48:52] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3480.000000 [06:48:52] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3444 [06:53:53] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1608.000000 [06:53:53] MySQL slave on rosemary is OK: Uptime: 1247791 Threads: 6 Questions: 1083145585 Slow queries: 645220 Opens: 77771 Flush tables: 1 Open tables: 3614 Queries per second avg: 868.50 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1572 [07:07:53] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:29:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:44:29] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:44:53] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 91542 MB (15% inode=99%): [07:45:30] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 194898 MB (3% inode=67%): [07:45:52] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:45:52] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 115969.000000 [07:45:52] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 77317.000000 [07:46:00] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [07:46:19] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:46:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:02:52] [[Special:Log/newusers]] create 10 * Digital12 * (New user account) [08:03:37] [[User talk:Digital12]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7847&rcid=21645 * Digital12 * (+2629) (Created page with "Electric Press computers are usually becoming essential as a luxury. With each family constantly leading to their very own pool area of photographs, downloaded audio and CDs asso...") [08:08:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:11:31] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:12:10] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:16:31] Sun Grid Engine execd on ortelius is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:17:10] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:22:31] Sun Grid Engine execd on ortelius is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:25:04] [[User talk:Digital12]] ! 10https://wiki.toolserver.org/w/index.php?diff=7848&oldid=7847&rcid=21646 * 130.126.56.169 * (-2629) (spam) [08:25:31] Sun Grid Engine execd on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:26:10] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:35:31] Sun Grid Engine execd on willow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:36:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:41:23] [[Special:Log/newusers]] create 10 * Nikravesha * (New user account) [08:44:30] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:45:04] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 91328 MB (15% inode=99%): [08:45:31] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 194616 MB (3% inode=67%): [08:46:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 117492.000000 [08:46:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 78911.000000 [08:46:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [08:46:39] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:47:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:53:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:27:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:36:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:44:50] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:45:30] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 193960 MB (3% inode=67%): [09:46:04] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 91118 MB (14% inode=99%): [09:46:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:47:03] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 115427.000000 [09:47:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79639.000000 [09:47:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:47:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [09:53:04] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:27:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:36:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:45:31] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 192992 MB (3% inode=67%): [10:46:03] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 90804 MB (14% inode=99%): [10:46:50] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:47:03] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 109081.000000 [10:47:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82793.000000 [10:47:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:47:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [10:53:04] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:59:02] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2174 [11:14:31] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:24:02] MySQL slave on z-dat-s2-b is OK: Uptime: 39400 Threads: 9 Questions: 65151269 Slow queries: 553 Opens: 1357074 Flush tables: 1 Open tables: 256 Queries per second avg: 1653.585 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1678 [11:27:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:36:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:45:31] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 192598 MB (3% inode=67%): [11:46:03] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 90547 MB (14% inode=99%): [11:46:49] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:47:03] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 105939.000000 [11:47:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 84710.000000 [11:47:12] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [11:48:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:53:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:12:05] Silke Meyer * [Toolserver-announce] Toolserver / Tool Labs at the Amsterdam Hackathon [12:14:31] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:28:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:28:31] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 55689 MB (5% inode=99%): [12:36:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:41:05] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2137 [12:42:25] @replag [12:42:25] fale: s1-rr-a-wd: 24s [+0.00 s/s]; s1-user-wd: 25s [-]; s2-rr: 37m 0s [+0.16 s/s]; s2-user: 37m 0s [+0.16 s/s]; s2-user-c: error; s2-user-wd: 1d 17m 38s [+0.60 s/s]; s3-user-wd: 24s [-0.00 s/s]; s4-user-wd: 1d 5h 25m 15s [-0.82 s/s] [12:42:26] fale: s5-user: 21m 58s [-]; s5-user-c: 2w 3d 15h 55m 32s [+1.00 s/s]; s6-user-wd: 24s [-0.00 s/s]; s7-user-wd: 24s [-0.00 s/s] [12:45:31] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 192264 MB (3% inode=67%): [12:46:03] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 90329 MB (14% inode=99%): [12:46:49] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:47:03] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 105857.000000 [12:47:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 87698.000000 [12:47:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [12:49:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:53:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:14:31] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:22:03] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1871 [13:22:11] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1878.000000 [13:29:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:36:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:40:01] MySQL slave on z-dat-s5-b is OK: Uptime: 47555 Threads: 2 Questions: 12931861 Slow queries: 384 Opens: 35830 Flush tables: 1 Open tables: 256 Queries per second avg: 271.934 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1743 [13:40:11] wikidata replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1682.000000 [13:42:02] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2433 [13:45:30] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 191899 MB (3% inode=67%): [13:46:50] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:47:02] MySQL slave on z-dat-s2-b is OK: Uptime: 47980 Threads: 10 Questions: 77224839 Slow queries: 706 Opens: 1648397 Flush tables: 1 Open tables: 256 Queries per second avg: 1609.521 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1672 [13:47:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 90072 MB (14% inode=99%): [13:47:02] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 105413.000000 [13:47:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 89414.000000 [13:47:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:49:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:54:02] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:14:50] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:19:49] Hello all [14:27:22] guten Tag, DaB [14:30:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:36:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:44:34] @replag [14:44:35] DaB|Eingeschneit: s1-rr-a-wd: 27s [+0.00 s/s]; s1-user-c: 31s [-]; s1-user-wd: 3m 28s [+0.02 s/s]; s2-rr: 50s [-0.30 s/s]; s2-user: 50s [-0.30 s/s]; s2-user-c: error; s2-user-wd: 1d 1h 12s [+0.35 s/s]; s3-user: 15s [-0.00 s/s] [14:44:35] DaB|Eingeschneit: s3-user-wd: 24s [-]; s4-user-wd: 1d 5h 45m 7s [+0.16 s/s]; s5-user-c: 2w 3d 17h 57m 41s [+1.00 s/s]; s6-user-wd: 26s [+0.00 s/s]; s7-user-wd: 23s [-0.00 s/s] [14:45:31] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 191485 MB (3% inode=67%): [14:46:50] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:47:03] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 89871 MB (14% inode=99%): [14:47:21] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:48:03] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 107246.000000 [14:48:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 90153.000000 [14:49:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:54:02] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:03:02] Free Memory on turnera is CRITICAL: CRITICAL - 4.0% (332340 kB) free! [15:11:48] Merlissimo: What's your JIRA account? Could you assign https://jira.toolserver.org/browse/TS-1567 to yourself? [15:11:59] scfc_de: i was wondering if you had a better idea than the last line in https://github.com/mzmcbride/database-reports/commit/157aa6141683c4239ef9f05e10061e97f1129b24 [15:13:31] legoktm: "-o ~/var/log", BTW. No, if a report needs access to these databases, that's the way to specify it. [15:13:44] yeah i fixed that afterwards [15:13:50] it just looks ugly though [15:15:32] Well, this is the non-Python bit, so it is allowed to :-). ("-l ~/var/log" leads to "Unable to run job: unknown resource '/home/project/d/b/r/dbreps/var/log'.") [15:18:57] yeah, i fixed that part in the next commit [15:19:44] i do need to optimize the query used in https://github.com/mzmcbride/database-reports/blob/master/wikidatawiki/missing_links.py to use a join rather than a subquery [15:20:26] i'm not too sure about how to go about it though... [15:24:02] Free Memory on turnera is WARNING: WARNING - 6.4% (532452 kB) free! [15:25:20] Basically, you do a "LEFT JOIN" and then test that the right part of the query is NULL. Does JOINs compared to EXISTs really speed things up? [15:26:41] The WHERE in the subquery needs attention, though. Moment. [15:29:15] I'm not sure what it is supposed to do [15:29:55] it looks for ips_item_id which are not on one site? [15:30:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:31:00] changing it can be tricky here [15:31:34] Hi there! We are now officially looking for a third person for toolserver administration: https://www.wikimedia.de/wiki/System_Administrator_%28m/f%29 [15:32:09] Platonides: yes [15:32:09] I think it would be: /* SLOW_OK */ SELECT ips_item_id, COUNT(*) FROM wb_items_per_site LEFT JOIN wb_items_per_site AS data ON (data.ips_item_id=wb_items_per_site.ips_item_id AND data.ips_site_id = ? ) WHERE data.ips_site_id IS NULL GROUP BY ips_item_id ORDER BY COUNT(*) DESC LIMIT 100; [15:32:15] Silke_WMDE, finally! [15:32:47] Yeah, i know, things take time. ;) [15:32:53] spread the word! [15:33:28] Silke_WMDE: Nice to see :-). [15:33:41] Platonides: I'll never get used to that MySQL allows WHERE in JOINs :-). [15:33:52] Platonides: ooh thanks. I'll take a look after class [15:34:26] And I absolutely hate "ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near". PostgreSQL is so user-friendly in this regard. [15:35:04] scfc_de, I don't know what you mean WHEREs placement [15:35:18] AFAIK it is set by SQL standard, not by mysql [15:35:34] I agree that error could be nicer, though :) [15:35:50] Silke_WMDE, you have a space before "This job description..." [15:36:21] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:36:40] Platonides: I don't have access to the website myself. [15:36:47] Platonides: You seem to be right. Is the WHERE condition applied before the JOIN or after? [15:40:02] Free Memory on turnera is CRITICAL: CRITICAL - 3.9% (330584 kB) free! [15:40:45] Silke_WMDE, ouch [15:41:07] ouch what? [15:41:11] I thought that perhaps all wmde members had access there [15:42:44] you should notify Michael Jahn, then [15:42:54] I don't think I know him [15:44:05] Silke Meyer * [Toolserver-l] Toolserver job announcement: System Administrator wanted [15:44:31] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:45:31] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 191017 MB (3% inode=67%): [15:46:50] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:47:03] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 89611 MB (14% inode=99%): [15:47:21] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [15:48:02] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 109639.000000 [15:48:37] Silke_WMDE, should the email have some specific subject? [15:49:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 89891.000000 [15:49:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:50:00] Platonides: How about "Toolserver System Administration"? [15:50:14] * Silke_WMDE hasn't thought about it yet :) [15:51:40] Toolserver System Administration'; UPDATE jobs WHERE name='Toolserver System Administration' SET closed = TRUE; xD [15:52:55] And the poor German readers have to walk through this probably immense web adventure at https://www.germanpersonnel.de/persy/profile/applicantform/startb :-). [15:53:33] Platonides: But WHERE before SET must be a MySQL "feature", or? :-) [15:53:41] it looks like one of those form-after-form pages [15:54:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:54:11] scfc_de, I don't think it would be valid SQL [15:54:21] PostgreSQL rejects it. [15:54:29] probably mysql, too [15:54:46] * Platonides loses the mysql karma he just won a few minutes ago [15:57:41] * Platonides realises that the (m/f) at the end must be male/female [15:57:55] I wondered what those m/w meant [15:58:01] Free Memory on turnera is OK: OK - 89.3% (7482308 kB) free. [15:58:07] "männlich"/"weiblich". [15:58:51] I thought it was the name of some kind of position/title [15:59:12] why should you have to specify that both are valid? [15:59:42] Lawyer's CYA. [16:00:10] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR opening session: SNMPv3 support is unavailable (Required module Digest/SHA1.pm not found). [16:02:07] Free Memory on turnera is OK: OK - 85.0% (7118852 kB) free. [16:07:14] (In German, the male form is the general (covering both sexes) one, but some people don't accept that, and so that they don't sue, "(m/w)" is added.) [16:08:50] it's similar to Spanish in that matter, then [16:09:17] although I haven't seen any similar suffix added to job offers [16:09:57] if an employer wanted something like that, I think it would write "Administrador/a" [16:10:50] That's an alternative in German as well, "System-Administrator/-in". [16:12:16] Ommmmmmmmmmmmmm.... [16:17:57] * Silke_WMDE actually invites ALL genders to apply, not just male and female. [16:20:20] let's hope it will be some form of neutrum – or 1 sex will dominate the other ;) [16:22:52] DaB, only if the number of people is not even [16:23:49] Platonides: it is 1:1 at the moment [16:25:08] (just to be sure: I do not care about the gender) [16:26:54] maybe we can count your mother too, to make it 2:2 :) [16:27:59] Role task: receiving weird and unintelligible phone calls [16:28:38] my mother thinks that paper&pen is superior and going away from them was a mistake [16:30:05] she's right... up to a point [16:30:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:31:08] I wouldn't consider writing a book using pen & paper instead of a computer [16:31:51] otoh, I prefer much more to read them in physical form [16:31:59] same here [16:37:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:44:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:45:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 190594 MB (3% inode=67%): [16:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:47:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 89362 MB (14% inode=99%): [16:47:48] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [16:48:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 111385.000000 [16:49:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 91505.000000 [16:49:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:54:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:57:36] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2078 [17:18:05] DeltaQuad Wikipedia * Re: [Toolserver-l] [Labs-l] Toolserver / Tool Labs at the Amsterdam Hackathon [17:18:22] thanks rebah [17:30:26] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:37:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:37:18] Silke_WMDE: can you point me to the location where we are supposed to list our services on toolserver for the transfer to labs? i've seemed to misplaced the email [17:38:31] Sure, here: http://www.mediawiki.org/wiki/Toolserver/List_of_Tools [17:38:40] thanks :) [17:42:31] https://www.wikimedia.de/wiki/System_Administrator_%28m/f%29 # Yay! [17:44:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [17:45:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 190155 MB (3% inode=67%): [17:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:47:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 89075 MB (14% inode=99%): [17:47:50] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [17:48:28] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 113476.000000 [17:48:43] I hate taxes legislation [17:48:58] well, it's not really hate, just... dislike [17:49:06] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 92299.000000 [17:49:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:50:22] anyway, email sent [17:54:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:56:37] MySQL slave on z-dat-s2-b is OK: Uptime: 62960 Threads: 8 Questions: 106927586 Slow queries: 878 Opens: 1772417 Flush tables: 1 Open tables: 256 Queries per second avg: 1698.341 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1795 [18:04:37] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1834 [18:05:37] MySQL slave on z-dat-s2-b is OK: Uptime: 63500 Threads: 12 Questions: 108009536 Slow queries: 883 Opens: 1776420 Flush tables: 1 Open tables: 256 Queries per second avg: 1700.937 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1755 [18:30:28] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:32:37] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2446 [18:37:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:44:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:45:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 189837 MB (3% inode=67%): [18:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:47:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 88803 MB (14% inode=99%): [18:47:47] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [18:49:08] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 93087.000000 [18:49:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 115468.000000 [18:49:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:54:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:30:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:32:37] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3027 [19:37:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:44:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:46:07] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 189201 MB (3% inode=67%): [19:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:47:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 88484 MB (14% inode=99%): [19:47:47] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [19:49:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 93695.000000 [19:49:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 117539.000000 [19:49:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:54:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:02:37] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3601 [20:30:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:34:37] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3508 [20:37:17] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:42:38] I will stop expresstrain for few minutes for an update [20:44:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:46:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 188690 MB (3% inode=67%): [20:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:47:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 88149 MB (14% inode=99%): [20:47:47] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [20:49:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 94138.000000 [20:49:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 117717.000000 [20:49:48] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:54:28] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:15:18] / on ortelius is WARNING: DISK WARNING - free space: / 6211 MB (20% inode=92%): [21:15:37] MySQL slave on z-dat-s2-b is OK: Uptime: 74900 Threads: 11 Questions: 124123384 Slow queries: 1078 Opens: 1828532 Flush tables: 1 Open tables: 256 Queries per second avg: 1657.188 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1800 [21:19:37] wikidata replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2166.000000 [21:19:57] wikidata replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2184.000000 [21:19:57] wikidata replag on z-dat-s6-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2191.000000 [21:19:57] wikidata replag on z-dat-s7-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2188.000000 [21:20:28] wikidata replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2216.000000 [21:20:37] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2210.000000 [21:21:41] expresstrain runs again. update done [21:25:57] wikidata replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1738.000000 [21:30:28] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:31:57] wikidata replag on z-dat-s7-a is OK: QUERY OK: SELECT ts_rc_age() returned 1755.000000 [21:32:37] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1748.000000 [21:33:27] wikidata replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1779.000000 [21:34:57] wikidata replag on z-dat-s6-a is OK: QUERY OK: SELECT ts_rc_age() returned 1764.000000 [21:38:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:44:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:46:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 188447 MB (3% inode=67%): [21:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:47:38] wikidata replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1797.000000 [21:47:38] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87855 MB (14% inode=99%): [21:47:48] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [21:49:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 96551.000000 [21:49:30] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 119312.000000 [21:49:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:54:16] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:15:17] / on ortelius is WARNING: DISK WARNING - free space: / 5667 MB (18% inode=92%): [22:30:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:37:06] hello all! [22:38:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:44:38] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:46:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 187994 MB (3% inode=67%): [22:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:47:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87532 MB (14% inode=99%): [22:47:47] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [22:49:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 97849.000000 [22:49:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 119444.000000 [22:49:51] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:54:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:15:17] / on ortelius is WARNING: DISK WARNING - free space: / 5210 MB (17% inode=92%): [23:26:17] / on ortelius is OK: DISK OK - free space: / 7232 MB (24% inode=92%): [23:30:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:34:37] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1959 [23:38:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:44:38] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:46:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 184991 MB (3% inode=67%): [23:47:17] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:47:37] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87239 MB (14% inode=99%): [23:47:47] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [23:49:07] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 98758.000000 [23:49:27] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 120544.000000 [23:49:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:54:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output