[00:00:39] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:05:40] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.998535/1.95, alarm hl:tmp_free=36491M/100M, alarm hl:np_load_avg=0.869141/2.0, alarm hl:mem_free=315.000000M/350M, alarm hl:available=1/0 [00:05:40] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [00:06:40] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [00:09:59] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [00:11:39] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [00:18:30] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [00:19:50] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28520.000000 [00:19:50] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 28519 [00:24:40] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.615234/1.95, alarm hl:tmp_free=36389M/100M, alarm hl:np_load_avg=0.653320/2.0, alarm hl:mem_free=253.000000M/350M, alarm hl:available=1/0 [00:31:39] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [00:36:00] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:42:50] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 215938 MB (4% inode=26%): [00:52:40] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.638672/1.95, alarm hl:tmp_free=36223M/100M, alarm hl:np_load_avg=0.633301/2.0, alarm hl:mem_free=345.000000M/350M, alarm hl:available=1/0 [00:54:39] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [01:00:50] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:06:40] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [01:10:00] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [01:11:50] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:18:40] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [01:19:50] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 28978 [01:19:50] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28979.000000 [01:25:49] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [01:43:00] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 215403 MB (4% inode=26%): [02:00:51] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:06:50] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [02:07:58] [[Special:Log/newusers]] create 10 * Manhobo * (New user account) [02:10:09] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [02:12:00] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [02:18:40] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [02:20:00] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 28091 [02:20:00] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28091.000000 [02:23:32] :( [02:23:52] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.811035/1.95, alarm hl:tmp_free=35890M/100M, alarm hl:np_load_avg=0.715820/2.0, alarm hl:mem_free=301.000000M/350M, alarm hl:available=1/0 [02:27:30] MySQL slave on z-dat-s7-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1956 [02:28:31] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1901 [02:29:59] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [02:39:40] MySQL slave on z-dat-s6-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1810 [02:41:40] MySQL slave on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [02:43:01] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 211789 MB (3% inode=26%): [02:43:10] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:10] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:10] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:10] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:30] SMTP on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:30] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:43:41] MySQL slave on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [02:43:41] MySQL slave on z-dat-s6-a is CRITICAL: (Service Check Timed Out) [02:43:50] / on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:43:50] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:43:50] / on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:43:50] /tmp on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:43:50] / on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:43:51] Load avg. on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:43:51] / on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:44:01] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:44:01] MySQL slave on z-dat-s6-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2046 [02:44:01] MySQL slave on z-dat-s7-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2900 [02:44:01] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [02:44:01] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [02:44:02] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [02:44:02] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [02:44:09] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2698 [02:44:10] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [02:44:19] / on z-dat-s3-a is OK: DISK OK - free space: / 8284 MB (27% inode=85%): [02:44:20] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 75021 MB (18% inode=99%): [02:44:20] SMTP on z-dat-s6-a is OK: SMTP OK - 0.004 sec. response time [02:44:20] / on z-dat-s4-a is OK: DISK OK - free space: / 8284 MB (27% inode=85%): [02:44:20] /tmp on z-dat-s4-a is OK: DISK OK - free space: /tmp 1179 MB (99% inode=99%): [02:44:21] / on z-dat-s7-a is OK: DISK OK - free space: / 8284 MB (27% inode=85%): [02:44:21] Load avg. on z-dat-s4-a is OK: OK - load average: 0.84, 0.82, 1.00 [02:44:21] / on z-dat-s6-a is OK: DISK OK - free space: / 8284 MB (27% inode=85%): [02:44:30] /sql on z-dat-s3-a is OK: DISK OK - free space: /sql 142003 MB (14% inode=99%): [02:53:50] MySQL slave on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [02:54:11] MySQL slave on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [03:01:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:02:50] MySQL slave on z-dat-s6-a is OK: Uptime: 502822 Threads: 25 Questions: 107036850 Slow queries: 27869 Opens: 798308 Flush tables: 1 Open tables: 2297 Queries per second avg: 212.872 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1787 [03:07:01] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [03:10:09] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [03:12:00] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [03:14:00] MySQL slave on z-dat-s7-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4382 [03:16:22] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4121 [03:18:40] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [03:20:00] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 26350 [03:20:00] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26351.000000 [03:40:13] SMTP on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:40:13] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:40:13] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:40:13] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:40:13] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:40:23] SMF on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:23] /tmp on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:33] SMF on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:33] SMF on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:33] SMF on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:33] SMF on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:33] Environment IPMI on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:34] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:42] SSH on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:41:03] /tmp on z-dat-s7-a is OK: DISK OK - free space: /tmp 1225 MB (99% inode=99%): [03:41:04] SMF on z-dat-s4-a is OK: OK - all services online [03:41:04] SMF on z-dat-s7-a is OK: OK - all services online [03:41:04] SMF on z-dat-s6-a is OK: OK - all services online [03:41:04] SMF on z-dat-s3-a is OK: OK - all services online [03:41:04] SMTP on hyacinth is OK: SMTP OK - 0.074 sec. response time [03:41:04] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [03:41:04] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [03:41:05] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [03:41:05] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [03:41:06] SMF on hyacinth is OK: OK - all services online [03:41:13] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [03:41:13] Environment IPMI on hyacinth is OK: ok: temperature ok fan ok voltage ok chassis ok [03:41:33] SSH on hyacinth is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [03:43:04] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 211673 MB (3% inode=26%): [04:01:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:03:05] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.014649/1.95, alarm hl:tmp_free=35449M/100M, alarm hl:np_load_avg=0.851562/2.0, alarm hl:mem_free=297.000000M/350M, alarm hl:available=1/0 [04:04:04] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [04:04:04] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.177734/1.10, alarm hl:np_load_long=0.674805/1.55, alarm hl:mem_free=9930.000000M/500M, alarm hl:tmp_free=15293M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.177734/1.00, alarm hl:np_load_long=0.674805/1.50, alarm hl:mem_free=9930.000000M/600M, alarm hl:tmp_free=1 [04:05:05] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [04:07:03] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [04:10:03] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.735352/1.95, alarm hl:tmp_free=35424M/100M, alarm hl:np_load_avg=0.848144/2.0, alarm hl:mem_free=220.000000M/350M, alarm hl:available=1/0 [04:10:13] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [04:11:14] MySQL slave on z-dat-s7-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3502 [04:12:03] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [04:15:53] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3535 [04:18:43] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [04:20:03] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 26279 [04:20:13] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26279.000000 [04:21:12] MySQL slave on z-dat-s7-a is OK: Uptime: 87950 Threads: 9 Questions: 18629707 Slow queries: 4806 Opens: 150234 Flush tables: 1 Open tables: 3536 Queries per second avg: 211.821 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1648 [04:35:08] MySQL slave on z-dat-s3-a is OK: Uptime: 2448856 Threads: 35 Questions: 2670467481 Slow queries: 163748 Opens: 25723484 Flush tables: 1 Open tables: 16384 Queries per second avg: 1090.495 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1779 [04:42:24] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:42:25] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:42:25] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:42:45] SMF on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:42:46] SMF on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:42:46] Load avg. on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:42:46] SMF on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:42:46] SMF on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:42:47] SMF on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:42:54] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:43:16] SMF on z-dat-s4-a is OK: OK - all services online [04:43:16] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 211635 MB (3% inode=26%): [04:43:16] Load avg. on z-dat-s6-a is OK: OK - load average: 0.64, 2.05, 3.36 [04:43:16] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [04:43:16] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [04:43:16] SMF on z-dat-s3-a is OK: OK - all services online [04:43:16] SMF on z-dat-s7-a is OK: OK - all services online [04:43:17] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [04:43:17] SMF on hyacinth is OK: OK - all services online [04:43:18] SMF on z-dat-s6-a is OK: OK - all services online [04:43:26] MySQL on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [04:43:26] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [04:43:35] MySQL on z-dat-s3-a is OK: Uptime: 2449367 Threads: 24 Questions: 2670818259 Slow queries: 163818 Opens: 25727965 Flush tables: 1 Open tables: 16383 Queries per second avg: 1090.411 [05:01:16] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:07:15] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [05:10:45] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [05:12:15] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [05:19:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [05:20:16] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 26553 [05:20:16] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26549.000000 [05:28:25] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.689941/1.95, alarm hl:tmp_free=35158M/100M, alarm hl:np_load_avg=0.730957/2.0, alarm hl:mem_free=245.000000M/350M, alarm hl:available=1/0 [05:32:25] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [05:44:14] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 224867 MB (4% inode=27%): [05:53:46] /sql on cassia is WARNING: DISK WARNING - free space: /sql 115000 MB (9% inode=99%): [05:55:04] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:55:35] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:55:35] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:55:35] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:55:54] SMF on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:55:54] SMF on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:55:55] SMF on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:15] Load avg. on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:15] /tmp on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:15] /sql on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:15] / on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:15] / on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:16] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:16] / on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:17] Load avg. on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:17] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:18] / on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:18] /tmp on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:56:24] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [05:56:24] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [05:56:25] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [05:56:25] SMF on z-dat-s7-a is OK: OK - all services online [05:56:25] SMF on z-dat-s3-a is OK: OK - all services online [05:56:25] SMF on z-dat-s6-a is OK: OK - all services online [05:56:34] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [05:56:45] Load avg. on z-dat-s4-a is OK: OK - load average: 1.86, 1.85, 2.35 [05:56:46] /tmp on z-dat-s4-a is OK: DISK OK - free space: /tmp 1007 MB (99% inode=99%): [05:56:46] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 87954 MB (21% inode=99%): [05:56:47] / on z-dat-s3-a is OK: DISK OK - free space: / 8287 MB (27% inode=85%): [05:56:47] / on z-dat-s6-a is OK: DISK OK - free space: / 8287 MB (27% inode=85%): [05:56:47] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 74982 MB (18% inode=99%): [05:56:47] / on z-dat-s4-a is OK: DISK OK - free space: / 8287 MB (27% inode=85%): [05:56:47] / on z-dat-s7-a is OK: DISK OK - free space: / 8287 MB (27% inode=85%): [05:56:47] /sql on z-dat-s3-a is OK: DISK OK - free space: /sql 142268 MB (14% inode=99%): [05:56:48] Load avg. on z-dat-s7-a is OK: OK - load average: 1.89, 1.86, 2.35 [05:56:48] /tmp on z-dat-s6-a is OK: DISK OK - free space: /tmp 971 MB (99% inode=99%): [06:01:36] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:06:25] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.630371/1.95, alarm hl:tmp_free=34742M/100M, alarm hl:np_load_avg=1.311035/2.0, alarm hl:mem_free=208.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.630371/2.3, alarm hl:np_load_long=1.000000/2.5, alarm hl:cpu=99.400000/98, alarm hl:mem_free=208.000000M/200M, al [06:07:34] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [06:10:45] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [06:12:24] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [06:17:25] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [06:19:05] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [06:20:15] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 25919 [06:20:25] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25919.000000 [06:37:24] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.635742/1.95, alarm hl:tmp_free=34477M/100M, alarm hl:np_load_avg=0.752441/2.0, alarm hl:mem_free=324.000000M/350M, alarm hl:available=1/0 [06:38:25] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [06:44:24] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 222108 MB (4% inode=27%): [07:02:25] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.359863/1.95, alarm hl:tmp_free=34380M/100M, alarm hl:np_load_avg=1.032715/2.0, alarm hl:mem_free=315.000000M/350M, alarm hl:available=1/0 [07:02:34] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:03:25] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [07:07:34] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [07:10:55] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [07:12:25] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [07:19:05] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [07:20:15] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 24780 [07:20:35] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 24776.000000 [07:44:25] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 219119 MB (4% inode=26%): [07:48:02] Danny B. * Re: [Toolserver-l] Toolserver user phe account expired [08:02:35] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:06:05] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:07:35] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [08:11:04] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [08:12:35] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [08:19:25] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [08:20:27] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 22482 [08:20:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 22468.000000 [08:25:35] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.830078/1.95, alarm hl:tmp_free=34017M/100M, alarm hl:np_load_avg=0.711426/2.0, alarm hl:mem_free=307.000000M/350M, alarm hl:available=1/0 [08:26:35] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [08:30:36] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [08:44:35] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 216248 MB (4% inode=26%): [09:02:35] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:04:45] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.703125/1.95, alarm hl:tmp_free=33869M/100M, alarm hl:np_load_avg=0.931152/2.0, alarm hl:mem_free=237.000000M/350M, alarm hl:available=1/0 [09:05:45] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [09:07:44] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [09:11:14] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [09:12:45] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [09:19:25] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [09:20:35] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 21019 [09:20:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21011.000000 [09:22:45] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.884766/1.95, alarm hl:tmp_free=33797M/100M, alarm hl:np_load_avg=0.853027/2.0, alarm hl:mem_free=294.000000M/350M, alarm hl:available=1/0 [09:32:48] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [09:44:53] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 213673 MB (4% inode=26%): [09:49:09] [[User:PressRTbox]] !NM 10https://wiki.toolserver.org/w/index.php?oldid=7300&rcid=9765 * PressRTbox * (+329) (Created page with "Choose Recovery Toolbox, Inc., if you are interested in data recovery solutions and Excel recovery methods in particular. This software developer works for years on the market o...") [10:02:53] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:07:53] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [10:11:34] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [10:12:53] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:19:33] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [10:20:43] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 18991 [10:20:53] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18977.000000 [10:45:00] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 210716 MB (3% inode=26%): [11:03:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:08:00] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [11:11:49] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [11:12:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [11:19:00] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.708984/1.95, alarm hl:tmp_free=33342M/100M, alarm hl:np_load_avg=0.627441/2.0, alarm hl:mem_free=298.000000M/350M, alarm hl:available=1/0 [11:20:29] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [11:21:00] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 15724 [11:21:08] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15714.000000 [11:22:59] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [11:29:59] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.101562/1.10, alarm hl:np_load_long=0.753906/1.55, alarm hl:mem_free=10684.000000M/500M, alarm hl:tmp_free=15003M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.101562/1.00, alarm hl:np_load_long=0.753906/1.50, alarm hl:mem_free=10684.000000M/600M, alarm hl:tmp_free [11:31:00] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.852539/1.95, alarm hl:tmp_free=33288M/100M, alarm hl:np_load_avg=0.947266/2.0, alarm hl:mem_free=206.000000M/350M, alarm hl:available=1/0 [11:32:01] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [11:45:09] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 208109 MB (3% inode=25%): [12:03:09] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.723633/1.95, alarm hl:tmp_free=33173M/100M, alarm hl:np_load_avg=0.698731/2.0, alarm hl:mem_free=137.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.723633/2.3, alarm hl:np_load_long=0.616699/2.5, alarm hl:cpu=56.000000/98, alarm hl:mem_free=137.000000M/200M, al [12:03:10] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:04:10] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1840.000000 [12:07:10] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [12:08:09] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [12:11:58] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [12:12:09] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.047851/1.95, alarm hl:tmp_free=33140M/100M, alarm hl:np_load_avg=0.936523/2.0, alarm hl:mem_free=257.000000M/350M, alarm hl:available=1/0 [12:13:09] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:20:39] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [12:21:09] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 12794 [12:21:10] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 12796.000000 [12:34:37] @replag [12:34:39] Jeff|mobile: s1-user: 3h 27m 30s [-0.28 s/s]; s2-user: 12h 7m 35s [+0.32 s/s]; s2-user-c: 49m 28s [+0.02 s/s]; s3-rr-a: 22s [-0.00 s/s]; s3-user: 22s [-0.00 s/s]; s5-user-c: 49m 28s [+0.02 s/s]; s7-rr-a: 10s [-0.00 s/s]; s7-user: 10s [-0.00 s/s] [12:45:09] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 206959 MB (3% inode=25%): [12:49:19] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3627.000000 [12:55:19] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.638672/1.10, alarm hl:np_load_long=1.063476/1.55, alarm hl:mem_free=10344.000000M/500M, alarm hl:tmp_free=14891M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.638672/1.00, alarm hl:np_load_long=1.063476/1.50, alarm hl:mem_free=10344.000000M/600M, alarm hl:tmp_free [13:01:20] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [13:02:20] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.951172/1.95, alarm hl:tmp_free=32957M/100M, alarm hl:np_load_avg=0.840820/2.0, alarm hl:mem_free=319.000000M/350M, alarm hl:available=1/0 [13:03:20] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:08:20] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [13:11:59] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [13:13:19] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.109375/1.10, alarm hl:np_load_long=1.125976/1.55, alarm hl:mem_free=10298.000000M/500M, alarm hl:tmp_free=14896M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.109375/1.00, alarm hl:np_load_long=1.125976/1.50, alarm hl:mem_free=10298.000000M/600M, alarm hl:tmp_free [13:13:20] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [13:14:19] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [13:20:49] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [13:21:09] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 10472 [13:21:19] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10461.000000 [13:22:19] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.431641/1.95, alarm hl:tmp_free=32888M/100M, alarm hl:np_load_avg=0.597168/2.0, alarm hl:mem_free=280.000000M/350M, alarm hl:available=1/0 [13:27:20] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [13:45:19] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 200804 MB (3% inode=25%): [13:49:19] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6080.000000 [14:03:30] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:08:29] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [14:11:58] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [14:13:29] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [14:20:48] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [14:21:08] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8432 [14:21:31] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8420.000000 [14:40:38] siebrand: Yes, some of it is. [14:40:47] siebrand: Though some of it is anonymized. [14:45:18] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 195315 MB (3% inode=24%): [14:49:29] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7863.000000 [14:52:30] Joan: thank you. I'm looking for some specific settings (translationnotifications-lang-%), but I don't see them in metawiki_p. What would be the cause of that? [14:52:57] Might have a whitelist. [14:54:03] http://p.defau.lt/?Yp58F17eEo4V6noPoh9PcA [14:54:17] I imagine the view is configured only to show particular prefs? [14:54:36] I'm checking the other table. [14:55:29] http://p.defau.lt/?xDHlkZdysxPpeOVp22FjdA [14:55:30] Hmmm. [14:55:39] Guess it'd be a question for DaB. or nosy. [14:55:49] Assuming you're sure that the extension stores its prefs in the user_properties table. [14:55:56] siebrand: ^^ [14:56:02] Magnus Manske * Re: [Toolserver-l] Toolserver user phe account expired [14:56:07] Yep. [14:56:32] You can have the data exposed in one of those tables. Depends if it's considered completely private or not. [14:56:40] _anonym masks user ID. [14:56:47] As you probably noticed. ;-) [14:56:53] They are languages users have signed up as translator for. [14:57:06] So probbably not too private. [14:57:08] Personally, I don't really see a privacy issue there. [14:57:24] Doesn't mean there isn't, but I think it's safe. [14:57:27] Yeah, so it's probably just a matter of adjusting the user_properties view to expose those rwos. [14:57:30] rows [14:57:38] So I ask DaB.? [14:57:40] Yes. [14:57:43] k [14:58:05] I did the same queries :) [14:58:12] Technically Reedy or nosy (new TS root) could it, but I think DaB. does all the views work. [14:58:21] Is there a page somewhere for those types of requests? [14:58:35] There's JIRA. [14:58:37] k [15:03:41] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:08:29] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [15:11:59] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [15:14:30] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [15:16:35] Joan: done: https://jira.toolserver.org/browse/TS-1390 [15:16:47] 3(created) [TS-1390] Expose up_value 'translationnotifications-lang-%' in user_properties tables; Toolserver: Databases; Trivial Task <10https://jira.toolserver.org/browse/TS-1390> (Siebrand Mazeland) [15:17:56] Cool. :-) You may need to poke DaB. on IRC if he doesn't notice. [15:18:02] I'm not sure how well JIRA notifies. [15:18:06] Or much he monitors. [15:18:11] how much * [15:18:14] I need to go outside. [15:20:48] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [15:21:19] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5917 [15:22:29] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5883.000000 [15:45:19] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 210042 MB (3% inode=25%): [15:49:29] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 9435.000000 [16:02:50] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.765625/1.95, alarm hl:tmp_free=32242M/100M, alarm hl:np_load_avg=0.993652/2.0, alarm hl:mem_free=216.000000M/350M, alarm hl:available=1/0 [16:03:51] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:08:39] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [16:09:50] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [16:12:19] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [16:12:50] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.930664/1.95, alarm hl:tmp_free=32176M/100M, alarm hl:np_load_avg=0.890137/2.0, alarm hl:mem_free=333.000000M/350M, alarm hl:available=1/0 [16:14:50] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [16:20:59] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [16:21:19] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4325 [16:22:40] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4287.000000 [16:45:24] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 206305 MB (3% inode=25%): [16:49:33] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8124.000000 [17:03:56] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:08:56] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [17:12:34] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [17:14:55] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [17:21:14] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [17:21:24] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3717 [17:22:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3703.000000 [17:26:24] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3586 [17:26:55] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3575.000000 [17:33:05] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.562988/1.95, alarm hl:tmp_free=31553M/100M, alarm hl:np_load_avg=0.502441/2.0, alarm hl:mem_free=322.000000M/350M, alarm hl:available=1/0 [17:33:57] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [17:43:00] s1 replag on thyme is CRITICAL: (Service Check Timed Out) [17:43:08] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3340.000000 [17:45:30] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 203133 MB (3% inode=25%): [17:49:49] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8745.000000 [17:54:08] /sql on cassia is WARNING: DISK WARNING - free space: /sql 118817 MB (10% inode=99%): [18:04:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:09:09] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [18:12:39] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [18:15:09] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [18:20:10] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.526367/1.95, alarm hl:tmp_free=31122M/100M, alarm hl:np_load_avg=0.539551/2.0, alarm hl:mem_free=281.000000M/350M, alarm hl:available=1/0 [18:22:09] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [18:26:29] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2996 [18:32:08] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [18:35:24] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.405762/1.95, alarm hl:tmp_free=31042M/100M, alarm hl:np_load_avg=0.526367/2.0, alarm hl:mem_free=176.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.405762/2.3, alarm hl:np_load_long=0.550293/2.5, alarm hl:cpu=40.400000/98, alarm hl:mem_free=176.000000M/200M, al [18:42:22] s1 replag on thyme is CRITICAL: (Service Check Timed Out) [18:42:41] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2868.000000 [18:43:22] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [18:46:12] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 202975 MB (3% inode=25%): [18:48:52] [[Special:Log/newusers]] create 10 * Minassianluis341 * (New user account) [18:50:14] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10000.000000 [19:04:33] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:09:21] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [19:13:22] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [19:13:32] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.542969/1.95, alarm hl:tmp_free=30805M/100M, alarm hl:np_load_avg=0.595215/2.0, alarm hl:mem_free=280.000000M/350M, alarm hl:available=1/0 [19:14:32] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [19:15:32] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [19:21:12] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:22:32] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [19:27:13] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2098 [19:28:33] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.460449/1.95, alarm hl:tmp_free=30666M/100M, alarm hl:np_load_avg=0.431641/2.0, alarm hl:mem_free=230.000000M/350M, alarm hl:available=1/0 [19:30:52] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [19:38:42] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [19:43:33] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1936.000000 [19:47:12] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 205911 MB (3% inode=25%): [19:49:13] MySQL slave on thyme is OK: Uptime: 1094663 Threads: 11 Questions: 430023167 Slow queries: 150823 Opens: 69638 Flush tables: 1 Open tables: 2970 Queries per second avg: 392.836 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1779 [19:49:32] s1 replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1767.000000 [19:50:22] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 11063.000000 [20:04:52] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:09:22] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [20:13:32] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:15:52] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [20:17:32] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.515625/1.95, alarm hl:tmp_free=30359M/100M, alarm hl:np_load_avg=0.612793/2.0, alarm hl:mem_free=203.000000M/350M, alarm hl:available=1/0 [20:22:32] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [20:22:52] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [20:29:43] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.591797/1.95, alarm hl:tmp_free=30256M/100M, alarm hl:np_load_avg=0.552734/2.0, alarm hl:mem_free=214.000000M/350M, alarm hl:available=1/0 [20:39:51] @relpag [20:42:42] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [20:47:24] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 202935 MB (3% inode=25%): [20:50:23] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10617.000000 [21:00:42] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.731445/1.95, alarm hl:tmp_free=30080M/100M, alarm hl:np_load_avg=0.516602/2.0, alarm hl:mem_free=319.000000M/350M, alarm hl:available=1/0 [21:04:55] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:09:32] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [21:09:43] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [21:10:33] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.175781/1.10, alarm hl:np_load_long=0.858398/1.55, alarm hl:mem_free=9785.000000M/500M, alarm hl:tmp_free=14491M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.175781/1.00, alarm hl:np_load_long=0.858398/1.50, alarm hl:mem_free=9785.000000M/600M, alarm hl:tmp_free=1 [21:11:33] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [21:13:42] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [21:15:53] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [21:22:53] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [21:47:35] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 199181 MB (3% inode=24%): [21:50:34] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 9161.000000 [22:05:03] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:09:45] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [22:13:43] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [22:16:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [22:23:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [22:25:04] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.496094/1.95, alarm hl:tmp_free=29579M/100M, alarm hl:np_load_avg=0.496582/2.0, alarm hl:mem_free=254.000000M/350M, alarm hl:available=1/0 [22:26:03] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [22:26:13] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:30:44] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [22:47:44] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 196141 MB (3% inode=24%): [22:50:44] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6841.000000 [23:05:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:09:54] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [23:14:04] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [23:16:03] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [23:23:03] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [23:47:44] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 195916 MB (3% inode=24%): [23:50:44] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5660.000000