[00:06:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 429574 MB (8% inode=40%): [00:06:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:15:13] /sql on thyme is CRITICAL: DISK CRITICAL - free space: /sql 102725 MB (10% inode=99%): [00:15:43] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 930601.000000 [00:33:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:37:53] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 23209 MB (5% inode=99%): [00:59:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 931896 [00:59:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:06:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 431697 MB (8% inode=40%): [01:06:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:15:44] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 932256.000000 [01:16:53] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 41152 MB (10% inode=99%): [01:32:04] @replag [01:32:05] Dispenser: s1-rr-a: 1m 34s [+0.00 s/s]; s1-user: 1w 3d 19h 3m 56s [+0.51 s/s]; s2-user: 30s [+0.00 s/s]; s3-rr-a: 13s [-0.00 s/s]; s3-user: 13s [-0.00 s/s]; s6-rr-a: 11s [-0.00 s/s]; s6-user: 11s [-0.00 s/s] [01:33:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:59:44] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 933395 [02:00:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:06:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 431667 MB (8% inode=40%): [02:06:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:07:09] <[ceradon]> Helllo [02:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:07:49] <[ceradon]> How do I login to my Toolserver account in linux via the Terminal? [02:08:13] ssh username@willow.toolserver.org [02:08:25] Run that, where "username" is your username. [02:09:29] <[ceradon]> it keeps on requesting a Password [02:09:36] <[ceradon]> and when I give it. [02:09:42] <[ceradon]> It just requests it again. [02:10:05] Your Toolserver account doesn't have a password. [02:10:12] You need to use your private SSH key to get in. [02:10:30] Which you need to copy over to ~/.ssh [02:11:33] <[ceradon]> Blah, Is there anyway I could get a new public key onto the Toolserver? [02:12:52] You can edit ~/.ssh/authorized_keys on your toolserver account [02:13:13] Heh. [02:13:18] Presumably he's not already logged in. [02:13:20] <[ceradon]> I can't access my Toolserver account to edit it. [02:13:30] [ceradon]: Sure, just file a ticket in JIRA. [02:14:52] That works too :P [02:15:43] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 933837.000000 [02:20:06] <[ceradon]> Brooke: One more thing, how do you login to a non-ssh server. [02:20:19] SSH is a protocol. [02:20:50] A server that's running an SSH agent will allow people to connect via SSH. [02:21:02] If a server doesn't have SSH up and running, you can usually use SFTP. [02:21:21] Hmm. [02:21:31] Well, FTP. [02:22:55] regular FTP works on the TS? [02:25:53] I'm not sure he was asking about the Toolserver. [02:26:15] Regular FTP wouldn't work with the Toolserver, no. [02:26:28] (I'm not sure he knew what he was asking.) [02:27:02] "a non-ssh server" [02:27:06] yes, that's a bit confusing [02:34:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:59:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 935037 [03:01:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:06:24] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 429130 MB (8% inode=40%): [03:06:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:07:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:10:53] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 23196 MB (5% inode=99%): [03:15:43] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 935469.000000 [03:28:54] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 40254 MB (9% inode=99%): [03:32:53] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 46800 MB (11% inode=99%): [03:34:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:58:34] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:59:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 936845 [04:01:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 427817 MB (8% inode=40%): [04:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:07:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:08:22] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [04:16:13] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 937216.000000 [04:34:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 938485 [05:01:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:07:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 427779 MB (8% inode=40%): [05:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:07:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:16:14] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 938872.000000 [05:18:53] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 41107 MB (10% inode=99%): [05:34:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:45:23] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 41683 MB (3% inode=98%): [05:54:23] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:55:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:55:23] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:57:53] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 97619 MB (10% inode=98%): [05:57:53] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 30132 MB (7% inode=99%): [05:57:53] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 97618 MB (10% inode=98%): [06:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 940557 [06:01:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:04:53] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 47076 MB (11% inode=99%): [06:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 427570 MB (8% inode=40%): [06:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:07:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:17:14] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 941101.000000 [06:34:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:56:54] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 41163 MB (10% inode=99%): [07:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 942490 [07:01:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 427527 MB (8% inode=40%): [07:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:07:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:17:13] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 942958.000000 [07:23:23] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:23:24] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:24:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:28:34] SMTP on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:34] SMTP on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:34] SMTP on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:34] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:34] SMTP on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:43] SMTP on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:43] SSH on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:44] Load avg. on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:28:44] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:44] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:28:53] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:29:03] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:13] / on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:13] / on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:13] / on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:13] /sql on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:23] / on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:23] /tmp on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:24] /tmp on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:24] /tmp on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:34] Load avg. on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:34] Environment IPMI on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:34] Load avg. on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:34] /tmp on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:34] Load avg. on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:29:34] Load avg. on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:30:13] / on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:30:23] MySQL slave on z-dat-s4-a is CRITICAL: (Service Check Timed Out) [07:30:34] /tmp on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:30:44] MySQL on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [07:30:44] MySQL slave on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [07:30:44] MySQL slave on z-dat-s6-a is CRITICAL: (Service Check Timed Out) [07:30:44] MySQL on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [07:30:53] s4 replag on z-dat-s4-a is CRITICAL: (Service Check Timed Out) [07:30:54] MySQL slave on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [07:30:54] MySQL on z-dat-s4-a is CRITICAL: (Service Check Timed Out) [07:31:23] NTP on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:31:33] MySQL on z-dat-s6-a is CRITICAL: (Service Check Timed Out) [07:34:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:34:53] MySQL slave on z-dat-s7-a is OK: Uptime: 5715566 Threads: 23 Questions: 1649086177 Slow queries: 197908 Opens: 13648897 Flush tables: 1 Open tables: 7114 Queries per second avg: 288.525 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 762 [07:34:54] / on z-dat-s6-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [07:34:54] / on hyacinth is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [07:34:54] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 59529 MB (14% inode=99%): [07:34:54] / on z-dat-s4-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [07:34:54] / on z-dat-s7-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [07:34:54] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 97503 MB (10% inode=98%): [07:34:54] s4 replag on z-dat-s4-a is OK: QUERY OK: SELECT ts_rc_age() returned 748.000000 [07:34:55] MySQL on z-dat-s7-a is OK: Uptime: 5715572 Threads: 14 Questions: 1649086235 Slow queries: 197912 Opens: 13648897 Flush tables: 1 Open tables: 7114 Queries per second avg: 288.525 [07:34:55] / on z-dat-s3-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [07:34:56] /tmp on hyacinth is OK: DISK OK - free space: /tmp 1702 MB (100% inode=99%): [07:34:56] /tmp on z-dat-s7-a is OK: DISK OK - free space: /tmp 1702 MB (99% inode=99%): [07:34:57] MySQL slave on z-dat-s6-a is OK: Uptime: 3758265 Threads: 11 Questions: 834009290 Slow queries: 255005 Opens: 7947626 Flush tables: 1 Open tables: 3137 Queries per second avg: 221.913 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 764 [07:34:57] MySQL on z-dat-s3-a is OK: Uptime: 3806454 Threads: 28 Questions: 4547143687 Slow queries: 253359 Opens: 43618087 Flush tables: 1 Open tables: 16385 Queries per second avg: 1194.587 [07:35:13] MySQL on z-dat-s6-a is OK: Uptime: 3758281 Threads: 9 Questions: 834009942 Slow queries: 255017 Opens: 7947630 Flush tables: 1 Open tables: 3137 Queries per second avg: 221.912 [07:35:13] MySQL slave on z-dat-s4-a is OK: Uptime: 5737220 Threads: 6 Questions: 374157727 Slow queries: 61936 Opens: 136333 Flush tables: 1 Open tables: 1124 Queries per second avg: 65.215 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 729 [07:35:13] NTP on hyacinth is OK: NTP OK: Offset 0.003811 secs [07:35:13] Load avg. on z-dat-s3-a is OK: OK - load average: 1.49, 1.39, 2.00 [07:35:13] Environment IPMI on hyacinth is OK: ok: temperature ok fan ok voltage ok chassis ok [07:35:23] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [07:35:23] SMTP on z-dat-s6-a is OK: SMTP OK - 0.148 sec. response time [07:35:23] SMTP on z-dat-s4-a is OK: SMTP OK - 0.193 sec. response time [07:35:23] SMTP on hyacinth is OK: SMTP OK - 0.120 sec. response time [07:35:23] SMTP on z-dat-s7-a is OK: SMTP OK - 0.483 sec. response time [07:35:33] SMTP on z-dat-s3-a is OK: SMTP OK - 0.097 sec. response time [07:35:33] SSH on hyacinth is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [07:35:33] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [07:35:34] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [07:35:43] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [07:49:53] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 40511 MB (9% inode=99%): [07:55:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:55:23] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:55:23] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:57:53] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 97395 MB (10% inode=98%): [07:57:53] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 97394 MB (10% inode=98%): [07:59:22] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:59:24] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 944124 [08:01:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 427598 MB (8% inode=40%): [08:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:08:44] MySQL slave on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [08:08:54] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:12:23] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2396 [08:17:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 944727.000000 [08:24:23] MySQL slave on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [08:24:48] UH... Could not connect to database host "sql-s7-rr.toolserver.org". - Exception: SQLSTATE[HY000] [2013] Lost connection to MySQL server at 'reading initial communication packet', system error: 0 [08:27:44] SSH on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:27:44] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:27:44] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:28:33] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:28:33] SMTP on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:28:33] SMTP on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:28:34] SMTP on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:28:34] SMTP on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:28:43] SMTP on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:28:53] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:29:03] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:14] / on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:14] / on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:14] / on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:14] /sql on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:14] / on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:23] /tmp on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:24] /tmp on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:24] / on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:24] /tmp on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:34] Environment IPMI on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:34] Load avg. on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:34] /tmp on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:34] Load avg. on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:34] Load avg. on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:34] /tmp on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:34] Load avg. on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:29:43] Load avg. on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:30:33] MySQL slave on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [08:30:53] MySQL on z-dat-s6-a is CRITICAL: (Service Check Timed Out) [08:30:54] MySQL on z-dat-s4-a is CRITICAL: (Service Check Timed Out) [08:31:14] s4 replag on z-dat-s4-a is CRITICAL: (Service Check Timed Out) [08:31:14] MySQL slave on z-dat-s6-a is CRITICAL: (Service Check Timed Out) [08:31:14] MySQL on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [08:31:23] MySQL slave on z-dat-s4-a is CRITICAL: (Service Check Timed Out) [08:31:23] NTP on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:31:34] MySQL on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [08:33:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:33:23] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:33:23] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:34:13] NTP on hyacinth is OK: NTP OK: Offset -0.001928 secs [08:34:53] MySQL on z-dat-s3-a is OK: Uptime: 3810053 Threads: 27 Questions: 4548169826 Slow queries: 255055 Opens: 43627137 Flush tables: 1 Open tables: 16384 Queries per second avg: 1193.728 [08:34:54] /tmp on hyacinth is OK: DISK OK - free space: /tmp 1692 MB (100% inode=99%): [08:34:54] / on hyacinth is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [08:34:54] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 97273 MB (10% inode=98%): [08:34:54] / on z-dat-s6-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [08:34:54] / on z-dat-s4-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [08:34:54] / on z-dat-s3-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [08:34:54] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 29987 MB (7% inode=99%): [08:34:55] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 60171 MB (14% inode=99%): [08:34:55] /tmp on z-dat-s7-a is OK: DISK OK - free space: /tmp 1711 MB (99% inode=99%): [08:34:56] / on z-dat-s7-a is OK: DISK OK - free space: / 8079 MB (26% inode=85%): [08:34:56] s4 replag on z-dat-s4-a is OK: QUERY OK: SELECT ts_rc_age() returned 743.000000 [08:34:57] /tmp on z-dat-s4-a is OK: DISK OK - free space: /tmp 1702 MB (99% inode=99%): [08:34:57] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 97273 MB (10% inode=98%): [08:35:03] MySQL slave on z-dat-s6-a is OK: Uptime: 3761870 Threads: 9 Questions: 834274129 Slow queries: 255751 Opens: 7947648 Flush tables: 1 Open tables: 3137 Queries per second avg: 221.771 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1584 [08:35:03] MySQL slave on z-dat-s4-a is OK: Uptime: 5740810 Threads: 6 Questions: 374293954 Slow queries: 62156 Opens: 136365 Flush tables: 1 Open tables: 1124 Queries per second avg: 65.198 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 742 [08:35:03] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3530 [08:35:03] Load avg. on z-dat-s7-a is OK: OK - load average: 0.10, 0.64, 2.03 [08:35:03] /tmp on z-dat-s6-a is OK: DISK OK - free space: /tmp 1761 MB (99% inode=99%): [08:35:03] MySQL on z-dat-s4-a is OK: Uptime: 5740813 Threads: 6 Questions: 374294043 Slow queries: 62156 Opens: 136365 Flush tables: 1 Open tables: 1124 Queries per second avg: 65.198 [08:35:23] MySQL slave on z-dat-s7-a is OK: Uptime: 5719196 Threads: 5 Questions: 1649591559 Slow queries: 198073 Opens: 13667781 Flush tables: 1 Open tables: 7088 Queries per second avg: 288.430 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 778 [08:35:23] Environment IPMI on hyacinth is OK: ok: temperature ok fan ok voltage ok chassis ok [08:35:23] SMTP on z-dat-s6-a is OK: SMTP OK - 0.023 sec. response time [08:35:23] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [08:35:23] SMTP on z-dat-s7-a is OK: SMTP OK - 0.056 sec. response time [08:35:23] SMTP on z-dat-s4-a is OK: SMTP OK - 0.140 sec. response time [08:35:23] SMTP on hyacinth is OK: SMTP OK - 0.128 sec. response time [08:35:33] SMTP on z-dat-s3-a is OK: SMTP OK - 0.027 sec. response time [08:35:34] SSH on hyacinth is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [08:35:34] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [08:35:34] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [08:35:34] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [08:35:43] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [08:35:53] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 48046 MB (11% inode=99%): [08:37:03] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3636 [08:46:23] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:48:24] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:49:14] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 97299 MB (10% inode=98%): [08:49:14] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 97299 MB (10% inode=98%): [09:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 946142 [09:01:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 426249 MB (8% inode=40%): [09:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:09:54] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:13:53] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 97292 MB (10% inode=98%): [09:13:54] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 97317 MB (10% inode=98%): [09:18:33] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 946748.000000 [09:28:18] @replag [09:28:48] Merlissimo: s1-user: 1w 3d 23h 4m 30s [+0.51 s/s]; s3-rr-a: 1h 46m 41s [+0.22 s/s]; s3-user: error; s6-rr-a: 41s [+0.00 s/s]; s6-user: 41s [+0.00 s/s]; s7-rr-a: 35m 9s [+0.01 s/s]; s7-user: 35m 20s [+0.01 s/s] [09:29:22] @replag [09:29:31] Merlissimo: s1-user: 1w 3d 23h 5m 1s [+0.48 s/s]; s3-rr-a: 1h 46m 37s [-0.06 s/s]; s3-user: 1h 46m 37s [+0.22 s/s]; s6-rr-a: 1m 16s [+0.73 s/s]; s6-user: 1m 16s [+0.73 s/s]; s7-rr-a: 35m 57s [+1.00 s/s]; s7-user: 36m 0s [+1.01 s/s] [09:29:44] MySQL slave on z-dat-s7-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2168 [09:36:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:37:05] when I run pywikipedia on nightshade, I get locale.Error: unsupported locale setting [09:37:11] but on willow everything's fine [09:37:43] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6851 [09:48:43] MySQL slave on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [09:49:36] liangent: create a bug report or write to the mailinglist. [09:49:55] DaBPunkt requested thoese bug reports on his last mail [09:50:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:50:23] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:51:54] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 98186 MB (10% inode=98%): [09:51:54] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 98186 MB (10% inode=98%): [09:52:23] MySQL slave on z-dat-s7-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3527 [09:54:23] MySQL slave on z-dat-s7-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3647 [10:00:44] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 948176 [10:00:54] Free Memory on turnera is WARNING: WARNING - 6.7% (558900 kB) free! [10:01:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:02:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:02:24] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:04:08] Merlissimo: TS-1467 submitted [10:04:53] Free Memory on turnera is OK: OK - 7.1% (597428 kB) free. [10:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 426163 MB (8% inode=40%): [10:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:09:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:18:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 948862.000000 [10:30:54] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 98014 MB (10% inode=98%): [10:36:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:38:03] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 9730 [10:47:03] MySQL slave on z-dat-s7-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3450 [10:49:03] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 97282 MB (10% inode=98%): [10:55:03] MySQL slave on z-dat-s7-a is OK: Uptime: 5727577 Threads: 9 Questions: 1651578355 Slow queries: 199291 Opens: 13703183 Flush tables: 1 Open tables: 7057 Queries per second avg: 288.355 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1584 [10:58:26] @replag [10:58:26] Dispenser: s1-user: 1w 3d 23h 56m 30s [+0.55 s/s]; s3-rr-a: 2h 42m 43s [+0.26 s/s]; s3-user: 2h 42m 43s [+0.26 s/s]; s6-rr-a: 31s [-0.54 s/s]; s6-user: 31s [-0.53 s/s]; s7-rr-a: 12m 9s [-1.90 s/s]; s7-user: 12m 9s [-1.90 s/s] [10:59:03] hello all [11:00:44] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 950256 [11:01:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:06:53] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 41938 MB (10% inode=99%): [11:07:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425629 MB (8% inode=40%): [11:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:09:54] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:19:33] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 951146.000000 [11:36:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:38:03] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6999 [11:41:54] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24145 MB (5% inode=99%): [11:48:23] Hello DaBPunkt, I've made my IRC-bot operation system independent (because of your mail), but I discovered a problem: https://jira.toolserver.org/browse/TS-1469 [11:49:41] Jelte: this is expected [11:50:29] Merlissimo: why? Isn't it strange to schedule a job on the most loaded system? [11:51:37] vim on nightshade doesn't have auto code indention and coloring? [11:53:21] Jelte: i gave solaris a higher priority to reduce number of errors for scripts that are not compatible for linux [11:53:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:53:34] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:53:53] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 95946 MB (9% inode=98%): [11:54:00] but on high load linux is used of course for arch='*' [11:54:03] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 95910 MB (9% inode=98%): [11:56:47] ah, ok. But it is a bit contradicting to DaBPunkt's request to move more longrunning tasks to Linux [12:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 952543 [12:01:44] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:03:53] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 52032 MB (12% inode=99%): [12:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425717 MB (8% inode=40%): [12:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:09:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:15:13] /sql on thyme is CRITICAL: DISK CRITICAL - free space: /sql 101665 MB (10% inode=99%): [12:19:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 953166.000000 [12:20:45] DaBPunkt: FYI, http://status.toolserver.org/ has been down for a few days now [12:23:04] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3485 [12:33:03] MySQL slave on z-dat-s3-a is OK: Uptime: 3824339 Threads: 19 Questions: 4555284385 Slow queries: 262692 Opens: 43659333 Flush tables: 1 Open tables: 16384 Queries per second avg: 1191.129 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1794 [12:36:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:52:53] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 36020 MB (8% inode=99%): [13:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 954419 [13:01:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:01:53] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 23397 MB (5% inode=99%): [13:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425483 MB (8% inode=40%): [13:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:09:54] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:15:24] @replag [13:15:25] liangent: s1-user: 1w 4d 1h 15m 53s [+0.54 s/s]; s2-user: 1m 58s [+0.04 s/s]; s2-user-c: 27s [-0.00 s/s]; s5-user-c: 27s [-0.00 s/s]; s7-rr-a: 20s [-0.09 s/s]; s7-user: 20s [-0.09 s/s] [13:16:56] I need the following in c: String foo=bar.subString(0,bar.indexOf("@")); (it should give me the local-part of an email-adress) [13:20:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 955113.000000 [13:26:53] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 53072 MB (13% inode=99%): [13:37:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:48:36] I have a SGE question: last year, River posted instructions on the mailing list for setting up a .environment file setting environment variables that would be passed to SGE jobs. This seems to work on Solaris servers, but not on Linux. Is there any way to pass environment variable settings to an SGE job running on Linux host? [13:49:57] russblau: fill in a bugreport [13:55:30] DaBPunkt, done (TS-1472) [14:00:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 956384 [14:02:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425437 MB (8% inode=40%): [14:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:09:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:20:33] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 957092.000000 [14:32:13] i'm having a lot of problems downloading files with filezilla, is it everything ok with sftp? [14:37:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:00:44] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 958356 [15:02:43] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:05:54] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 41957 MB (10% inode=99%): [15:07:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425284 MB (8% inode=40%): [15:07:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:09:54] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:20:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 959010.000000 [15:29:03] Free Memory on turnera is WARNING: WARNING - 6.6% (552900 kB) free! [15:32:03] Free Memory on turnera is OK: OK - 7.5% (630972 kB) free. [15:37:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:43:53] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 23891 MB (5% inode=99%): [15:46:18] I can't find a button to reopen a Jira-issue. Does any one know how I can reopen a Jira-issue (reported by myself)? [16:01:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 960164 [16:02:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425244 MB (8% inode=40%): [16:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:09:53] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 50485 MB (12% inode=99%): [16:09:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:13:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:13:33] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:16:53] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 98348 MB (10% inode=98%): [16:17:03] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 98348 MB (10% inode=98%): [16:20:33] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 960709.000000 [16:25:23] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:25:34] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:37:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:44:53] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 98266 MB (10% inode=98%): [16:45:03] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 98262 MB (10% inode=98%): [16:46:48] Danny_B|backup: i'm having problems using the arch=linux. i get the following message running a py script: [16:47:01] /usr/bin/python: error while loading shared libraries: libc.so.6: failed to map segment from shared object: Cannot allocate memory [17:01:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 961824 [17:02:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425143 MB (8% inode=40%): [17:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:09:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:13:57] alchimista: you obviously want to talk to DaBPunkt and not me ;-) [17:15:29] Danny_B|backup: lool, dammit tab :P sorry [17:21:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 962355.000000 [17:37:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:45:23] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 40794 MB (3% inode=98%): [18:01:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 963571 [18:02:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425071 MB (8% inode=40%): [18:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:09:54] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:21:43] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 964305.000000 [18:22:40] i have a problem using cron with qcronsub... it says "Unable to read script file because of error: error opening python: No such file or directory" and the command was qcronsub -l h_rt=0:20:00 -l virtual_free=100M -m n -wd /home/nickanc/pywikipedia "python welcome.py -edit:1 -nlog -sul -quiet -random -limit:200 -lang:vec -break" how to solve this thing? [18:25:17] nickanc, this is only a guess, but try adding "-b y" to your command after "-m n" [18:25:38] I try [18:27:03] nickanc, I think you also need to change python to /usr/bin/python [18:31:01] it seems to work.... thank you [18:31:12] :) russblau [18:37:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:01:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 965738 [19:02:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:07:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 425012 MB (8% inode=40%): [19:07:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:10:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:21:42] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 966498.000000 [19:37:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:01:44] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 967838 [20:02:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 424968 MB (8% inode=40%): [20:07:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:11:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:21:43] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 968513.000000 [20:37:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:58:03] Free Memory on turnera is WARNING: WARNING - 5.8% (487660 kB) free! [21:01:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 969678 [21:02:54] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:04:03] Free Memory on turnera is OK: OK - 7.4% (624148 kB) free. [21:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 424710 MB (8% inode=40%): [21:07:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:11:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:21:43] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 970313.000000 [21:22:11] nacht ts [21:37:55] @replag [21:37:56] cashews: s1-user: 1w 4d 5h 38m 54s [+0.52 s/s]; s2-user: 5m 24s [+0.01 s/s]; s3-rr-a: 30s [+0.00 s/s]; s3-user: 30s [+0.00 s/s] [21:38:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:44:27] @replag [21:44:27] MJ94: s1-rr-a: 12s [-0.00 s/s]; s1-user: 1w 4d 5h 42m 24s [+0.54 s/s]; s2-user: 6m 44s [+0.20 s/s]; s3-rr-a: 46s [+0.04 s/s]; s3-user: 46s [+0.04 s/s] [21:45:03] Free Memory on turnera is WARNING: WARNING - 6.4% (539528 kB) free! [21:45:13] Any idea what's going on? It's unbearably slow. [21:51:03] Free Memory on turnera is OK: OK - 7.3% (612580 kB) free. [22:01:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 971399 [22:03:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 424630 MB (8% inode=40%): [22:08:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:11:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:21:44] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 972065.000000 [22:38:14] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:01:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 973244 [23:03:54] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:07:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 424557 MB (8% inode=40%): [23:08:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:11:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:21:43] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 973885.000000 [23:38:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output