[00:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 851361.000000 [00:01:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (392272 kB) free! [00:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:02:29] nacht ts [00:08:02] Free Memory on damiana is WARNING: WARNING - 5.5% (459936 kB) free! [00:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 505884 MB (9% inode=66%): [00:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:27:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.5% (375712 kB) free! [00:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [00:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 11435 [00:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [00:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 11454.000000 [00:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [00:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:41:02] Free Memory on damiana is WARNING: WARNING - 5.1% (430384 kB) free! [00:41:07] John * Re: [Toolserver-l] Tool request [00:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87554 MB (14% inode=99%): [00:46:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [00:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 18200 [00:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [00:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18201.000000 [01:01:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 843331.000000 [01:02:13] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:03:03] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 44809 MB (4% inode=99%): [01:03:12] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2167.000000 [01:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 508330 MB (9% inode=67%): [01:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [01:29:43] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1974 [01:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 10127 [01:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [01:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10064.000000 [01:34:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [01:35:42] MySQL slave on z-dat-s2-b is OK: Uptime: 699763 Threads: 11 Questions: 1035548531 Slow queries: 10012 Opens: 6484938 Flush tables: 1 Open tables: 256 Queries per second avg: 1479.856 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1785 [01:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:40:13] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3645.000000 [01:41:02] Free Memory on damiana is WARNING: WARNING - 6.5% (541572 kB) free! [01:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87393 MB (14% inode=99%): [01:42:03] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (417324 kB) free! [01:46:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:46:12] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3584.000000 [01:47:12] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3613.000000 [01:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [01:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17865 [01:53:33] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [01:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17740.000000 [02:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 842956.000000 [02:01:03] Free Memory on damiana is WARNING: WARNING - 5.1% (429912 kB) free! [02:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:07:12] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2477.000000 [02:13:12] wikidata replag on z-dat-s2-b is OK: QUERY OK: SELECT ts_rc_age() returned 1648.000000 [02:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 559084 MB (10% inode=69%): [02:24:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.5% (376500 kB) free! [02:26:13] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:28:08] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [02:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8363 [02:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [02:33:03] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8361.000000 [02:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [02:36:02] Free Memory on damiana is WARNING: WARNING - 5.1% (430832 kB) free! [02:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87238 MB (14% inode=99%): [02:46:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:48:21] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [02:51:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (412940 kB) free! [02:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17557 [02:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [02:56:41] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17472.000000 [03:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 834790.000000 [03:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:05:42] MySQL slave on z-dat-s1-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1924 [03:10:02] Free Memory on damiana is WARNING: WARNING - 5.3% (442996 kB) free! [03:11:02] Free Memory on damiana is OK: OK - 9.0% (751988 kB) free. [03:19:48] @replag [03:19:49] Betacommand: s1-rr-a: 34m 2s [-0.18 s/s]; s1-rr-a-wd: 2m 57s [+0.00 s/s]; s1-user: 4h 45m 29s [-0.06 s/s]; s2-user-c: error; s2-user-wd: 2m 57s [+0.00 s/s]; s3-user: 12m 33s [+0.01 s/s]; s5-user: 2h 19m 12s [+0.09 s/s]; s5-user-c: 1w 2d 15h 18m 33s [-1.35 s/s] [03:19:50] Betacommand: s6-user-wd: 20s [+0.00 s/s] [03:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 558763 MB (10% inode=69%): [03:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [03:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8606 [03:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [03:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8631.000000 [03:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [03:37:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (421280 kB) free! [03:37:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:41:13] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 87050 MB (14% inode=99%): [03:43:57] does anyone know if the jobs queue is replicated? [03:46:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:47:34] why does enwiki still have the cur table? that was killed in WM 1.5 [03:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [03:51:07] John * [Toolserver-l] enwiki_p [03:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16837 [03:53:20] Betacommand: It's not, as I recall. [03:53:26] It changes a lot and there's no reason to replicate it. [03:53:29] May be a blackhole. [03:53:31] Not sure. [03:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [03:53:52] Susan: there was a column that was recently added that is really useful [03:54:05] it tracks the time the job was added to the queue\ [03:54:18] Right. [03:54:27] Well, you can file a ticket in JIRA, I suppose. [03:55:14] I remember asking River about this. [03:55:25] I think it's one of the few tables that isn't replicated because it's so painful to replicate. [03:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17085.000000 [04:00:03] Free Memory on damiana is WARNING: WARNING - 5.3% (441260 kB) free! [04:01:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 830393.000000 [04:01:03] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (420756 kB) free! [04:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:05:42] MySQL slave on z-dat-s1-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2253 [04:15:03] Free Memory on damiana is WARNING: WARNING - 5.1% (428344 kB) free! [04:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 557832 MB (10% inode=69%): [04:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [04:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7623 [04:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [04:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7614.000000 [04:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [04:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 86900 MB (14% inode=99%): [04:46:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [04:52:13] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17358 [04:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [04:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17403.000000 [04:58:02] Free Memory on damiana is WARNING: WARNING - 5.6% (465940 kB) free! [05:01:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 830849.000000 [05:02:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (413240 kB) free! [05:02:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:05:41] MySQL slave on z-dat-s1-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2144 [05:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 558821 MB (10% inode=69%): [05:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:28:03] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [05:32:02] Free Memory on damiana is WARNING: WARNING - 5.2% (434164 kB) free! [05:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7078 [05:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [05:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7095.000000 [05:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [05:35:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.5% (373424 kB) free! [05:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:39:02] Free Memory on damiana is OK: OK - 9.2% (767204 kB) free. [05:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 86750 MB (14% inode=99%): [05:48:21] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17516 [05:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [05:56:41] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17817.000000 [06:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 828359.000000 [06:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:05:42] MySQL slave on z-dat-s1-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2772 [06:10:02] Free Memory on damiana is WARNING: WARNING - 5.3% (446792 kB) free! [06:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:20:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (421592 kB) free! [06:21:02] Free Memory on damiana is WARNING: WARNING - 5.1% (423208 kB) free! [06:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 558713 MB (10% inode=69%): [06:22:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (416964 kB) free! [06:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [06:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7460 [06:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [06:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7470.000000 [06:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [06:37:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 86576 MB (14% inode=99%): [06:48:28] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [06:51:03] Free Memory on damiana is WARNING: WARNING - 5.3% (445612 kB) free! [06:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20320 [06:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [06:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:58:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 20665.000000 [07:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 825650.000000 [07:01:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (409520 kB) free! [07:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:05:42] MySQL slave on z-dat-s1-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2258 [07:14:02] Free Memory on damiana is WARNING: WARNING - 5.2% (436620 kB) free! [07:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:21:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (421320 kB) free! [07:21:41] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 558656 MB (10% inode=69%): [07:22:03] Free Memory on damiana is WARNING: WARNING - 5.1% (426680 kB) free! [07:25:42] MySQL slave on z-dat-s1-b is OK: Uptime: 206716 Threads: 4 Questions: 119360403 Slow queries: 10086 Opens: 3059 Flush tables: 1 Open tables: 256 Queries per second avg: 577.412 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1739 [07:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [07:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7488 [07:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [07:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7495.000000 [07:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [07:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 86293 MB (14% inode=99%): [07:46:02] Free Memory on damiana is WARNING: WARNING - 5.7% (473868 kB) free! [07:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [07:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 22883 [07:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [07:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:57:02] Load avg. on nightshade is WARNING: WARNING - load average: 16.78, 16.45, 13.18 [07:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 23089.000000 [08:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 819729.000000 [08:01:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.5% (380596 kB) free! [08:02:02] Load avg. on nightshade is OK: OK - load average: 14.36, 14.96, 13.39 [08:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 558460 MB (10% inode=69%): [08:22:02] Free Memory on damiana is WARNING: WARNING - 5.4% (449116 kB) free! [08:26:02] Load avg. on nightshade is WARNING: WARNING - load average: 20.57, 18.07, 15.52 [08:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [08:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7194 [08:32:51] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [08:33:03] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7189.000000 [08:34:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [08:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 85974 MB (14% inode=99%): [08:46:02] Load avg. on nightshade is OK: OK - load average: 10.24, 12.77, 14.75 [08:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [08:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 24903 [08:53:02] Free Memory on damiana is CRITICAL: CRITICAL - 3.2% (264164 kB) free! [08:53:31] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [08:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25166.000000 [09:01:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 816935.000000 [09:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:16:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 557907 MB (10% inode=69%): [09:26:13] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [09:32:31] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7020 [09:32:51] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [09:33:03] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6995.000000 [09:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [09:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:39:02] Free Memory on damiana is WARNING: WARNING - 5.1% (423388 kB) free! [09:41:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.8% (401220 kB) free! [09:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 85742 MB (14% inode=99%): [09:48:28] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [09:52:12] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [09:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [09:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28766.000000 [10:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 813891.000000 [10:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:03:02] Free Memory on damiana is WARNING: WARNING - 5.1% (429448 kB) free! [10:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 557591 MB (10% inode=69%): [10:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [10:31:07] Marlen Caemmerer * Re: [Toolserver-l] sql-s1-user-replication will stop [10:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5802 [10:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [10:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5768.000000 [10:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [10:37:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 85477 MB (14% inode=99%): [10:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [10:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 31318 [10:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [10:56:03] Free Memory on damiana is CRITICAL: CRITICAL - 3.6% (298828 kB) free! [10:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:58:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 31474.000000 [11:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 812603.000000 [11:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:21:44] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 557324 MB (10% inode=69%): [11:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [11:31:02] Free Memory on damiana is WARNING: WARNING - 5.3% (441480 kB) free! [11:32:37] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6199 [11:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [11:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6222.000000 [11:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [11:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:38:07] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (390864 kB) free! [11:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 85249 MB (14% inode=99%): [11:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [11:52:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 33463 [11:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [11:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 33675.000000 [12:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 807682.000000 [12:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:03:27] [[User talk:Dab]] ! 10https://wiki.toolserver.org/w/index.php?diff=7870&oldid=7422&rcid=21693 * Muhammad shuaib * (+130) (/* Cluster No, */ new section) [12:04:51] [[User:Muhammad shuaib]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7871&rcid=21694 * Muhammad shuaib * (+199) (Created page with "Hi All, you can contact me [http://ur.wikipedia.org/wiki/%D8%AA%D8%A8%D8%A7%D8%AF%D9%84%DB%82_%D8%AE%DB%8C%D8%A7%D9%84_%D8%B5%D8%A7%D8%B1%D9%81:%D9%85%D8%AD%D9%85%D8%AF_%D8%B4%D8...") [12:05:32] [[User talk:Muhammad shuaib]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7872&rcid=21695 * Muhammad shuaib * (+199) (Created page with "Hi All, you can contact me [http://ur.wikipedia.org/wiki/%D8%AA%D8%A8%D8%A7%D8%AF%D9%84%DB%82_%D8%AE%DB%8C%D8%A7%D9%84_%D8%B5%D8%A7%D8%B1%D9%81:%D9%85%D8%AD%D9%85%D8%AF_%D8%B4%D8...") [12:07:08] [[User talk:Dab]] 10https://wiki.toolserver.org/w/index.php?diff=7873&oldid=7870&rcid=21696 * Betacommand * (+93) (/* Cluster No, */ ) [12:10:18] [[User talk:Dab]] ! 10https://wiki.toolserver.org/w/index.php?diff=7874&oldid=7873&rcid=21697 * Muhammad shuaib * (+72) (/* Cluster No, */ ) [12:16:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 556918 MB (10% inode=69%): [12:26:13] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [12:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6314 [12:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [12:32:53] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2169.000000 [12:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6308.000000 [12:33:02] wikidata replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2204.000000 [12:34:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [12:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:38:02] Free Memory on damiana is CRITICAL: CRITICAL - 3.3% (275268 kB) free! [12:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 85002 MB (13% inode=99%): [12:42:11] [[User talk:Dab]] ! 10https://wiki.toolserver.org/w/index.php?diff=7875&oldid=7874&rcid=21698 * Muhammad shuaib * (+511) (/* Error in submitting task */ ) [12:47:46] [[Main Page/lang]] ! 10https://wiki.toolserver.org/w/index.php?diff=7876&oldid=7858&rcid=21699 * Muhammad shuaib * (+39) () [12:48:23] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [12:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 35439 [12:53:31] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [12:56:30] [[صفحہ اول]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7877&rcid=21700 * Muhammad shuaib * (+1275) (Created page with "{{:Main Page/lang}}
{| class="plainlinks" cellspacing="8" style="font-family:'Tahoma','Lucida Sans',Verdana,sans-serif;" | colspan="2" style="padding...") [12:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:56:53] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3610.000000 [12:57:02] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3643.000000 [12:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 35643.000000 [12:59:40] @replag [12:59:40] multichill: s1-rr-a-wd: 18m 53s [+0.03 s/s]; s1-user: 9h 55m 8s [+0.53 s/s]; s1-user-wd: 20m 57s [+0.00 s/s]; s2-rr: 17m 6s [+0.01 s/s]; s2-user: 17m 6s [+0.01 s/s]; s2-user-c: error; s2-user-wd: 18m 53s [+0.03 s/s]; s3-user: 11s [-0.02 s/s] [12:59:41] multichill: s5-rr-a-wd: 1h 3m 27s [-]; s5-user: 1h 56m 56s [-0.04 s/s]; s5-user-c: 1w 2d 9h 20m 7s [-0.62 s/s] [13:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 811282.000000 [13:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:04:03] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 43964 MB (4% inode=99%): [13:04:04] [[صفحہ اول]] ! 10https://wiki.toolserver.org/w/index.php?diff=7878&oldid=7877&rcid=21701 * Muhammad shuaib * (+1017) () [13:08:08] [[صفحہ اول]] ! 10https://wiki.toolserver.org/w/index.php?diff=7879&oldid=7878&rcid=21702 * Muhammad shuaib * (+635) () [13:08:55] [[صفحہ اول]] ! 10https://wiki.toolserver.org/w/index.php?diff=7880&oldid=7879&rcid=21703 * Muhammad shuaib * (+2) () [13:12:15] [[User:Muhammad shuaib/common.css]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7881&rcid=21704 * Muhammad shuaib * (+235) (Created page with "* {} #content { font-size: 100%; font-family: "Alvi Nastaleeq" !important; } textarea { font-size: 110%; font-family: "Alvi Nastaleeq" !important;} * {font-family: Alvi Nastaleeq...") [13:13:52] [[صفحہ اول]] ! 10https://wiki.toolserver.org/w/index.php?diff=7882&oldid=7880&rcid=21705 * Muhammad shuaib * (+3) () [13:16:59] [[Category:Edit counters]] ! 10https://wiki.toolserver.org/w/index.php?diff=7883&oldid=7685&rcid=21706 * Muhammad shuaib * (+42) () [13:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 556578 MB (10% inode=69%): [13:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [13:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8386 [13:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [13:33:03] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8404.000000 [13:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [13:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:38:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.5% (378492 kB) free! [13:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 84735 MB (13% inode=99%): [13:46:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 37509 [13:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [13:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:56:52] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7210.000000 [13:57:03] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7243.000000 [13:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 37660.000000 [14:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 814882.000000 [14:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:17:03] Free Memory on damiana is WARNING: WARNING - 5.1% (424688 kB) free! [14:18:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (422876 kB) free! [14:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 556104 MB (10% inode=69%): [14:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:28:03] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [14:31:03] Free Memory on damiana is WARNING: WARNING - 5.2% (434824 kB) free! [14:32:33] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 10001 [14:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [14:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10011.000000 [14:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [14:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:41:13] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 84405 MB (13% inode=99%): [14:46:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 39076 [14:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [14:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:56:52] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10809.000000 [14:58:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39284.000000 [14:58:03] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10903.000000 [15:01:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 818482.000000 [15:02:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:05:03] Free Memory on damiana is CRITICAL: CRITICAL - 2.5% (207836 kB) free! [15:10:03] Free Memory on damiana is WARNING: WARNING - 5.1% (429184 kB) free! [15:12:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.4% (369772 kB) free! [15:21:41] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 555735 MB (10% inode=69%): [15:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [15:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7970 [15:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [15:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7971.000000 [15:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [15:37:02] Free Memory on damiana is WARNING: WARNING - 5.1% (423900 kB) free! [15:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:41:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (419100 kB) free! [15:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 84123 MB (13% inode=99%): [15:43:02] Free Memory on damiana is WARNING: WARNING - 5.1% (427688 kB) free! [15:46:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [15:50:52] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3304.000000 [15:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 41188 [15:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [15:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:56:52] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1760.000000 [15:58:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 41432.000000 [15:59:03] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8825.000000 [16:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 821119.000000 [16:02:13] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:04:02] Free Memory on damiana is WARNING: WARNING - 6.1% (510324 kB) free! [16:11:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.8% (406336 kB) free! [16:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 555365 MB (10% inode=69%): [16:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [16:31:02] Free Memory on damiana is WARNING: WARNING - 5.2% (432672 kB) free! [16:32:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (413740 kB) free! [16:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7009 [16:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [16:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6943.000000 [16:34:02] Free Memory on damiana is WARNING: WARNING - 5.2% (432852 kB) free! [16:34:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [16:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 83811 MB (13% inode=99%): [16:46:12] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [16:52:03] wikidata replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3575.000000 [16:52:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 42702 [16:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [16:56:43] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:56:43] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2108 [16:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 42933.000000 [17:01:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 823603.000000 [17:02:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:08:03] wikidata replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1770.000000 [17:13:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.3% (360808 kB) free! [17:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 554902 MB (10% inode=69%): [17:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:28:03] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [17:31:02] Free Memory on damiana is WARNING: WARNING - 5.3% (443204 kB) free! [17:32:07] DaB. * Re: [Toolserver-l] enwiki_p [17:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6230 [17:32:43] MySQL slave on z-dat-s2-b is OK: Uptime: 757183 Threads: 9 Questions: 1117878497 Slow queries: 10855 Opens: 7151399 Flush tables: 1 Open tables: 256 Queries per second avg: 1476.365 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1741 [17:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [17:33:03] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6234.000000 [17:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [17:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 83485 MB (13% inode=99%): [17:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [17:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 43918 [17:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [17:56:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.6% (383564 kB) free! [17:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 44040.000000 [18:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 826139.000000 [18:02:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:08:02] Free Memory on damiana is WARNING: WARNING - 5.1% (430104 kB) free! [18:11:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.3% (363712 kB) free! [18:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 554580 MB (10% inode=69%): [18:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:28:03] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [18:30:02] Free Memory on damiana is WARNING: WARNING - 5.1% (425064 kB) free! [18:32:31] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6446 [18:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [18:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6469.000000 [18:34:02] Free Memory on damiana is OK: OK - 7.2% (605044 kB) free. [18:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [18:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:41:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 83262 MB (13% inode=99%): [18:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [18:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 45759 [18:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [18:56:02] Free Memory on damiana is CRITICAL: CRITICAL - 4.8% (400860 kB) free! [18:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:58:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 45962.000000 [18:59:21] What is the MySQL complement to "CONVERT(? USING utf8)", i. e., given a UTF8 string, what gives the VARBINARY presentation used by Wikimedia? [19:01:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 829408.000000 [19:03:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:21:41] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 554230 MB (10% inode=69%): [19:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [19:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7765 [19:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [19:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7784.000000 [19:34:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:34:57] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [19:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:41:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 82946 MB (13% inode=99%): [19:45:02] Free Memory on damiana is WARNING: WARNING - 5.1% (423684 kB) free! [19:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [19:49:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (422068 kB) free! [19:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 47573 [19:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [19:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47776.000000 [19:59:12] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 57699 MB (10% inode=99%): [20:01:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 832058.000000 [20:01:12] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 58893 MB (11% inode=99%): [20:03:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:16:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 553861 MB (10% inode=69%): [20:23:03] Free Memory on damiana is WARNING: WARNING - 5.2% (438872 kB) free! [20:24:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (412232 kB) free! [20:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:28:03] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [20:32:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6530 [20:32:51] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [20:33:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6473.000000 [20:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [20:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:39:02] Free Memory on damiana is WARNING: WARNING - 5.2% (433300 kB) free! [20:42:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 82661 MB (13% inode=99%): [20:48:23] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [20:52:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 47855 [20:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [20:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47961.000000 [21:01:02] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3577.000000 [21:01:32] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3565 [21:02:03] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 834496.000000 [21:02:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3625 [21:03:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3650.000000 [21:03:07] unitedcabletv * [Toolserver-l] login.toolserver.org&In-Reply [21:03:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:05:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (420940 kB) free! [21:13:32] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3596 [21:14:02] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3563.000000 [21:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 553576 MB (10% inode=69%): [21:23:25] Hi all, anyone know php scripts on a userdir are throwing a 500 all of a sudden? :( [21:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:28:03] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [21:31:03] Free Memory on damiana is WARNING: WARNING - 5.3% (446176 kB) free! [21:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [21:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [21:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:42:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (418564 kB) free! [21:42:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 82369 MB (13% inode=99%): [21:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [21:52:16] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 47532 [21:52:46] Damianz, I suspect LDAP is down [21:53:02] I can't login [21:53:07] Make sense - it's been very intermittent... [21:53:17] Working for maybe 10min at a time then just boom 500 [21:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [21:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:57:07] Platonides * Re: [Toolserver-l] login.toolserver.org&In-Reply [21:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47763.000000 [22:02:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 837135.000000 [22:03:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:13:31] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3289 [22:14:03] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3314.000000 [22:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:21:42] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 553434 MB (10% inode=69%): [22:22:03] wikidata replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1942.000000 [22:22:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3609 [22:23:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3603.000000 [22:23:32] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3579 [22:24:03] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3563.000000 [22:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:27:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3602.000000 [22:27:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3613 [22:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [22:32:02] Free Memory on damiana is WARNING: WARNING - 5.1% (423800 kB) free! [22:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [22:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [22:37:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:40:02] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (417456 kB) free! [22:42:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 82120 MB (13% inode=99%): [22:48:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [22:53:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 46575 [22:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [22:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:58:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46792.000000 [23:02:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3749.000000 [23:02:02] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 839585.000000 [23:02:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3739 [23:03:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:11:32] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3541 [23:12:03] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3550.000000 [23:13:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3610.000000 [23:13:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3644 [23:14:02] Free Memory on damiana is WARNING: WARNING - 5.2% (433004 kB) free! [23:16:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:21:43] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 551759 MB (10% inode=68%): [23:22:03] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (408844 kB) free! [23:22:03] wikidata replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3126.000000 [23:26:02] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3587.000000 [23:26:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:26:32] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3544 [23:27:03] Free Memory on damiana is WARNING: WARNING - 5.1% (424588 kB) free! [23:28:02] APT on sage is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [23:32:52] APT on z-dat-s2-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [23:34:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:34:52] APT on z-dat-s1-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [23:37:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:42:13] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 81791 MB (13% inode=99%): [23:48:23] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [23:49:03] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3604.000000 [23:49:03] Free Memory on damiana is CRITICAL: CRITICAL - 3.8% (314424 kB) free! [23:49:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3612 [23:50:32] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3597 [23:51:02] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3600.000000 [23:51:32] MySQL slave on z-dat-s5-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3604 [23:52:02] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3615.000000 [23:53:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 46894 [23:53:32] APT on z-dat-s5-b is WARNING: APT WARNING: 1 packages available for upgrade (0 critical updates). [23:56:03] Free Memory on damiana is WARNING: WARNING - 5.1% (430784 kB) free! [23:56:42] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:58:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47054.000000