[00:02:19] Load avg. on willow is WARNING: WARNING - load average: 18.39, 16.25, 13.18 [00:03:18] Load avg. on willow is OK: OK - load average: 12.29, 14.80, 12.85 [00:06:58] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:13:48] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.005859/1.00, alarm hl:np_load_long=0.839844/1.50, alarm hl:mem_free=10734.000000M/600M, alarm hl:tmp_free=13086M/100M, alarm hl:available=1/0 [00:14:48] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [00:15:58] MySQL slave on thyme is OK: Uptime: 333070 Threads: 15 Questions: 122775063 Slow queries: 37126 Opens: 24068 Flush tables: 1 Open tables: 2767 Queries per second avg: 368.616 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1799 [00:17:18] s1 replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1794.000000 [00:18:48] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [00:19:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:20:49] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [00:30:49] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [00:31:48] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [00:31:58] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [00:32:19] Load avg. on willow is WARNING: WARNING - load average: 14.71, 15.18, 13.57 [00:33:19] Load avg. on willow is OK: OK - load average: 10.17, 13.73, 13.16 [00:35:29] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1856.000000 [00:35:58] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1867 [00:44:58] MySQL slave on thyme is OK: Uptime: 334813 Threads: 12 Questions: 123150427 Slow queries: 37536 Opens: 24148 Flush tables: 1 Open tables: 2767 Queries per second avg: 367.818 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1784 [00:45:28] s1 replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1779.000000 [00:49:58] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 92262 [00:50:28] Load avg. on willow is WARNING: WARNING - load average: 15.08, 13.35, 12.74 [00:56:07] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 294259 MB (5% inode=33%): [00:56:48] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [01:18:48] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [01:19:59] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:20:58] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [01:31:48] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [01:32:00] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [01:32:08] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:50:08] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93360 [01:56:08] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 294478 MB (5% inode=33%): [02:18:58] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [02:19:58] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:21:08] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [02:31:58] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [02:32:08] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [02:32:59] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [02:37:01] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.147461/1.10, alarm hl:np_load_long=0.709961/1.55, alarm hl:mem_free=10695.000000M/500M, alarm hl:tmp_free=12890M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.147461/1.00, alarm hl:np_load_long=0.709961/1.50, alarm hl:mem_free=10695.000000M/600M, alarm hl:tmp_free [02:38:01] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [02:50:11] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 94027 [02:50:41] Load avg. on willow is WARNING: WARNING - load average: 15.58, 15.97, 16.10 [02:53:42] Load avg. on willow is OK: OK - load average: 8.78, 12.68, 14.76 [02:56:11] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292636 MB (5% inode=33%): [02:56:41] Load avg. on willow is WARNING: WARNING - load average: 15.08, 16.18, 15.95 [02:59:41] Load avg. on willow is OK: OK - load average: 7.72, 12.64, 14.61 [03:19:01] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [03:20:01] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:21:41] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [03:32:11] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [03:32:31] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [03:33:17] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [03:50:21] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 94868 [03:56:31] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292626 MB (5% inode=32%): [04:11:20] Sun Grid Engine execd on wolfsbane is WARNING: short-sol@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.420899/1.10, alarm hl:np_load_long=0.529785/1.55, alarm hl:mem_free=326.000000M/500M, alarm hl:tmp_free=12972M/200M, alarm hl:available=1/0: medium-sol@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.420899/1.00, alarm hl:np_load_long=0.529785/1.50, alarm hl:mem_free=326.000000M/600M, alarm hl:tmp_free= [04:14:21] Sun Grid Engine execd on wolfsbane is OK: short-sol@wolfsbane OK: medium-sol@wolfsbane OK [04:15:20] /sql on cassia is WARNING: DISK WARNING - free space: /sql 129477 MB (10% inode=99%): [04:17:21] /sql on cassia is OK: DISK OK - free space: /sql 132541 MB (11% inode=99%): [04:18:20] Sun Grid Engine execd on wolfsbane is WARNING: short-sol@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.438476/1.10, alarm hl:np_load_long=0.483887/1.55, alarm hl:mem_free=471.000000M/500M, alarm hl:tmp_free=12989M/200M, alarm hl:available=1/0: medium-sol@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.438476/1.00, alarm hl:np_load_long=0.483887/1.50, alarm hl:mem_free=471.000000M/600M, alarm hl:tmp_free= [04:19:21] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [04:20:21] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:22:01] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [04:23:21] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.089844/1.00, alarm hl:np_load_long=0.759765/1.50, alarm hl:mem_free=9925.000000M/600M, alarm hl:tmp_free=14672M/100M, alarm hl:available=1/0 [04:24:21] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [04:32:15] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [04:32:43] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [04:32:52] Load avg. on willow is WARNING: WARNING - load average: 13.35, 16.08, 14.40 [04:33:21] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [04:33:51] Load avg. on willow is OK: OK - load average: 9.33, 14.41, 13.93 [04:50:22] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 95778 [04:56:31] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292614 MB (5% inode=32%): [05:03:01] Load avg. on willow is WARNING: WARNING - load average: 11.44, 15.20, 14.18 [05:04:01] Load avg. on willow is OK: OK - load average: 9.19, 14.02, 13.83 [05:13:21] [[Special:Log/newusers]] create 10 * Caixater76 * (New user account) [05:19:23] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [05:20:22] /sql on cassia is WARNING: DISK WARNING - free space: /sql 129641 MB (10% inode=99%): [05:20:22] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:22:01] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:22:22] /sql on cassia is OK: DISK OK - free space: /sql 132290 MB (11% inode=99%): [05:22:30] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [05:31:22] /sql on cassia is WARNING: DISK WARNING - free space: /sql 129913 MB (10% inode=99%): [05:32:25] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [05:33:01] Load avg. on willow is WARNING: WARNING - load average: 10.06, 15.11, 14.38 [05:33:22] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [05:33:41] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [05:34:11] Load avg. on willow is OK: OK - load average: 7.15, 13.44, 13.84 [05:37:01] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [05:49:22] /sql on cassia is OK: DISK OK - free space: /sql 134569 MB (11% inode=99%): [05:51:22] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93919 [05:56:22] /sql on cassia is WARNING: DISK WARNING - free space: /sql 129939 MB (10% inode=99%): [05:56:31] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292605 MB (5% inode=32%): [06:03:10] Load avg. on willow is WARNING: WARNING - load average: 12.89, 16.79, 15.47 [06:05:11] Load avg. on willow is OK: OK - load average: 9.70, 14.48, 14.77 [06:09:11] Load avg. on willow is WARNING: WARNING - load average: 12.46, 16.37, 15.78 [06:09:23] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.009766/1.00, alarm hl:np_load_long=0.666992/1.50, alarm hl:mem_free=10403.000000M/600M, alarm hl:tmp_free=14584M/100M, alarm hl:available=1/0 [06:10:21] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [06:20:22] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [06:20:22] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:22:41] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [06:32:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [06:33:31] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [06:33:41] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [06:45:31] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.154297/1.10, alarm hl:np_load_long=0.806640/1.55, alarm hl:mem_free=11008.000000M/500M, alarm hl:tmp_free=14533M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.154297/1.00, alarm hl:np_load_long=0.806640/1.50, alarm hl:mem_free=11008.000000M/600M, alarm hl:tmp_free [06:46:31] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [06:51:32] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93430 [06:56:41] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292576 MB (5% inode=32%): [07:03:11] Load avg. on willow is WARNING: WARNING - load average: 14.14, 17.71, 16.13 [07:16:10] Load avg. on willow is OK: OK - load average: 10.64, 13.64, 14.93 [07:20:41] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [07:20:51] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:21:11] Load avg. on willow is WARNING: WARNING - load average: 12.49, 14.52, 15.04 [07:23:01] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [07:23:41] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.014649/1.00, alarm hl:np_load_long=0.766602/1.50, alarm hl:mem_free=10916.000000M/600M, alarm hl:tmp_free=14501M/100M, alarm hl:available=1/0 [07:24:41] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [07:32:41] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [07:34:01] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [07:34:02] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [07:34:41] /sql on cassia is OK: DISK OK - free space: /sql 130729 MB (11% inode=99%): [07:36:01] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 37256 MB (9% inode=99%): [07:41:01] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24438 MB (5% inode=99%): [07:51:40] /sql on cassia is WARNING: DISK WARNING - free space: /sql 130020 MB (10% inode=99%): [07:52:31] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93896 [07:57:01] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292504 MB (5% inode=32%): [07:57:02] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:00:01] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 45230 MB (11% inode=99%): [08:01:41] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [08:03:01] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 33626 MB (8% inode=99%): [08:09:01] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24385 MB (5% inode=99%): [08:14:51] /sql on cassia is OK: DISK OK - free space: /sql 131061 MB (11% inode=99%): [08:20:51] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [08:20:51] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:23:10] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [08:32:51] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [08:34:00] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [08:35:01] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [08:52:34] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93662 [08:57:04] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292470 MB (5% inode=32%): [08:57:54] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [09:20:55] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [09:20:55] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:23:44] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [09:32:55] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [09:34:35] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [09:35:36] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [09:39:04] SMTP on ortelius is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:40:04] SMTP on ortelius is OK: SMTP OK - 7.931 sec. response time [09:52:45] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93620 [09:57:33] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292117 MB (5% inode=32%): [09:57:55] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [10:02:55] /sql on cassia is WARNING: DISK WARNING - free space: /sql 129278 MB (10% inode=99%): [10:04:57] /sql on cassia is OK: DISK OK - free space: /sql 133715 MB (11% inode=99%): [10:12:55] /sql on cassia is WARNING: DISK WARNING - free space: /sql 129781 MB (10% inode=99%): [10:21:14] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:21:54] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [10:24:14] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [10:32:54] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [10:34:45] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [10:36:33] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:52:45] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93937 [10:57:33] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 292078 MB (5% inode=32%): [10:58:04] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [11:00:04] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.317383/1.10, alarm hl:np_load_long=0.734375/1.55, alarm hl:mem_free=11145.000000M/500M, alarm hl:tmp_free=14234M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.317383/1.00, alarm hl:np_load_long=0.734375/1.50, alarm hl:mem_free=11145.000000M/600M, alarm hl:tmp_free [11:17:05] Sun Grid Engine execd on wolfsbane is WARNING: short-sol@wolfsbane exceedes load threshold: alarm hl:np_load_short=1.619141/1.10, alarm hl:np_load_long=0.802246/1.55, alarm hl:mem_free=1831.000000M/500M, alarm hl:tmp_free=12468M/200M, alarm hl:available=1/0: medium-sol@wolfsbane exceedes load threshold: alarm hl:np_load_short=1.619141/1.00, alarm hl:np_load_long=0.802246/1.50, alarm hl:mem_free=1831.000000M/600M, alarm hl:tmp_fre [11:18:05] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [11:19:04] Sun Grid Engine execd on wolfsbane is OK: short-sol@wolfsbane OK: medium-sol@wolfsbane OK [11:21:15] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:22:05] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [11:24:35] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [11:33:14] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [11:34:49] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [11:36:42] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [11:42:02] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:53:02] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93374 [11:57:42] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 291929 MB (5% inode=32%): [11:58:12] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [11:59:12] SSH on adenia is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:00:02] SSH on adenia is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [12:03:11] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.138672/1.10, alarm hl:np_load_long=0.822265/1.55, alarm hl:mem_free=10508.000000M/500M, alarm hl:tmp_free=14109M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.138672/1.00, alarm hl:np_load_long=0.822265/1.50, alarm hl:mem_free=10508.000000M/600M, alarm hl:tmp_free [12:04:16] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [12:06:37] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [12:21:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:22:16] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [12:24:43] 3(reopened) [TS-1369] SSL certificate problem, bad/outdated HTTPS CA <10https://jira.toolserver.org/browse/TS-1369> (Marlen Caemmerer) [12:24:47] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [12:26:42] 3(commented) [TS-1369] SSL certificate problem, bad/outdated HTTPS CA <10https://jira.toolserver.org/browse/TS-1369> (Marlen Caemmerer) [12:33:18] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [12:35:08] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [12:36:48] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:51:57] hello all [12:53:07] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 93847 [12:57:47] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 291891 MB (5% inode=32%): [12:58:18] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [13:03:28] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:21:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:22:27] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [13:24:57] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [13:33:28] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [13:35:28] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [13:36:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [13:53:17] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 94292 [13:57:56] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 291297 MB (5% inode=32%): [13:59:12] @replag [13:59:12] JRWR: s2-user-c: 11s [-]; s3-rr-a: 30s [-]; s3-user: 30s [-]; s5-rr-a: 42s [-]; s5-user: 42s [-]; s5-user-c: 11s [-]; s6-rr-a: 1d 2h 11m 29s [-]; s6-user: 1d 2h 11m 29s [-] [14:21:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:22:38] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [14:25:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [14:33:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [14:36:06] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [14:37:17] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [14:47:17] /sql on cassia is WARNING: DISK WARNING - free space: /sql 126966 MB (10% inode=99%): [14:53:28] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 94533 [14:58:17] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 290512 MB (5% inode=32%): [15:06:17] /sql on cassia is OK: DISK OK - free space: /sql 132852 MB (11% inode=99%): [15:19:44] 3(commented) [TS-1369] SSL certificate problem, bad/outdated HTTPS CA <10https://jira.toolserver.org/browse/TS-1369> (Krinkle) [15:22:17] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:23:16] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [15:25:27] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [15:34:18] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [15:36:47] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [15:37:17] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.075195/1.00, alarm hl:np_load_long=0.770508/1.50, alarm hl:mem_free=11077.000000M/600M, alarm hl:tmp_free=13810M/100M, alarm hl:available=1/0 [15:37:17] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [15:40:17] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [15:42:53] DaBPunkt, do you know why my stored procedures in mysql keep vanishing from time to time? [15:53:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 94994 [15:56:53] omg, the interwikibots are back! [15:57:07] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:59:17] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.164062/1.10, alarm hl:np_load_long=0.885742/1.55, alarm hl:mem_free=10816.000000M/500M, alarm hl:tmp_free=13794M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.164062/1.00, alarm hl:np_load_long=0.885742/1.50, alarm hl:mem_free=10816.000000M/600M, alarm hl:tmp_free [15:59:18] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 287642 MB (5% inode=32%): [16:02:18] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [16:06:37] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [16:08:17] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.422851/1.10, alarm hl:np_load_long=1.009766/1.55, alarm hl:mem_free=10558.000000M/500M, alarm hl:tmp_free=13678M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.422851/1.00, alarm hl:np_load_long=1.009766/1.50, alarm hl:mem_free=10558.000000M/600M, alarm hl:tmp_free [16:10:41] dschwen: usind differend servers? [16:11:04] hmm, I'm alway connecting to the same one [16:11:34] sql-s?-user.toolserver.org [16:12:14] well, no big deal, I'll just add the commands to create the procedures to the beginning of the scripts that use them [16:12:20] just puzzled me a bit [16:12:40] maybe it is related to resyncing the servers from a fresh dump? [16:13:01] aren't the stored procedures stored in a special system table? [16:13:13] maybe that gets overwritten by reimporting [16:22:17] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:23:17] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [16:25:47] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [16:35:20] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [16:37:07] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [16:37:27] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [16:42:27] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.968750/1.10, alarm hl:np_load_long=1.143555/1.55, alarm hl:mem_free=10749.000000M/500M, alarm hl:tmp_free=13724M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.968750/1.00, alarm hl:np_load_long=1.143555/1.50, alarm hl:mem_free=10749.000000M/600M, alarm hl:tmp_free [16:46:27] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [16:53:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 95748 [16:57:16] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 39839 MB (9% inode=99%): [16:59:28] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 283855 MB (5% inode=32%): [17:05:20] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24032 MB (5% inode=99%): [17:14:17] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 96365 MB (23% inode=99%): [17:18:06] Environment IPMI on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:18:47] Environment IPMI on adenia is OK: ok: temperature ok fan ok voltage ok chassis ok [17:22:07] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:22:38] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:23:16] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [17:25:47] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [17:36:18] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [17:37:06] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [17:37:38] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [17:53:38] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 96256 [17:59:38] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 280512 MB (5% inode=32%): [18:23:17] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [18:23:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:25:47] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [18:28:27] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.357422/1.10, alarm hl:np_load_long=0.746094/1.55, alarm hl:mem_free=11102.000000M/500M, alarm hl:tmp_free=13584M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.357422/1.00, alarm hl:np_load_long=0.746094/1.50, alarm hl:mem_free=11102.000000M/600M, alarm hl:tmp_free [18:36:17] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [18:37:18] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [18:38:38] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [18:47:27] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [18:53:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 96545 [18:56:44] DaBPunkt: hi :) around ? [18:59:38] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 277719 MB (5% inode=31%): [19:05:09] yes [19:06:28] DaBPunkt: just to ask you : I need a database access for the MMP… must I fill a JIRA request ? [19:06:45] yes, please [19:08:01] ok… in https://jira.toolserver.org/browse/TS ? [19:08:09] yes [19:08:32] ok thx [19:10:23] DaBPunkt: BTW, I saw that an MMP called "interwikibot" already existed… do you know what it is ? [19:11:53] Toto_Azero: that was from the old try to harmonize the bot-situation. If you like, you can have it instead of the langlinksbot-MMP [19:12:13] yeah, I would prefer if you don't mind :) [19:12:39] do I need to erase all the files I had created on the current MMP langlinksbot ? [19:13:46] yes, but it has not to be done now [19:14:31] Toto_Azero: how was your login-name? [19:14:38] totoazero [19:15:56] ok, aded you to the MMP. The MP have had already mysql-access, but I re-new the password just to be sure [19:16:45] ok many thanks :) [19:16:59] no problem [19:23:17] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [19:23:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:25:57] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [19:36:17] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [19:37:07] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:37:17] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [19:38:48] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [19:46:36] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [19:53:47] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 97652 [19:59:48] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 277436 MB (5% inode=31%): [20:17:43] DaBPunkt: i've got a script in qw for almost 7 hours.. and i have a question about memory alocation with SGE, i've defined 50M max, but got an error [20:23:23] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [20:23:52] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:26:02] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:36:25] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [20:37:34] Sun Grid Engine execd on willow is CRITICAL: medium-sol@willow in error state: longrun-sol@willow in error state [20:38:54] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [20:53:53] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 98159 [21:00:43] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 277226 MB (5% inode=31%): [21:02:43] / on willow is WARNING: DISK WARNING - free space: / 21995 MB (20% inode=99%): [21:02:54] /tmp on willow is WARNING: DISK WARNING - free space: / 21991 MB (20% inode=99%): [21:06:03] Alchimista: please try to submit the job again [21:10:47] nacht ts [21:16:34] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [21:23:34] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [21:24:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:26:14] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [21:36:36] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [21:39:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [21:40:34] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.793945/1.95, alarm hl:tmp_free=21467M/100M, alarm hl:np_load_avg=0.814941/2.0, alarm hl:mem_free=240.000000M/350M, alarm hl:available=1/0 [21:43:34] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [21:53:54] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 99010 [21:58:34] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.834961/1.95, alarm hl:tmp_free=21061M/100M, alarm hl:np_load_avg=0.817383/2.0, alarm hl:mem_free=192.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.834961/2.3, alarm hl:np_load_long=0.839356/2.5, alarm hl:cpu=70.700000/98, alarm hl:mem_free=192.000000M/200M, al [22:00:43] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 277079 MB (5% inode=31%): [22:02:43] / on willow is WARNING: DISK WARNING - free space: / 20859 MB (19% inode=98%): [22:02:54] /tmp on willow is WARNING: DISK WARNING - free space: / 20838 MB (19% inode=98%): [22:23:34] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [22:24:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:26:24] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [22:37:34] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [22:37:53] [[Special:Log/newusers]] create 10 * Maverickxx7876 * (New user account) [22:39:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [22:53:54] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 99883 [23:00:45] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 276897 MB (5% inode=31%): [23:02:45] / on willow is WARNING: DISK WARNING - free space: / 20381 MB (19% inode=98%): [23:02:54] /tmp on willow is WARNING: DISK WARNING - free space: / 20372 MB (19% inode=98%): [23:09:35] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.841309/1.95, alarm hl:tmp_free=20287M/100M, alarm hl:np_load_avg=0.882812/2.0, alarm hl:mem_free=201.000000M/350M, alarm hl:available=1/0 [23:10:34] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [23:23:35] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [23:24:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:26:25] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [23:37:44] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [23:39:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [23:46:47] [[User:Maverickxx7876]] !NM 10https://wiki.toolserver.org/w/index.php?oldid=7267&rcid=9697 * Maverickxx7876 * (+611) (Created page with "Guten Tag zusammen, haben Selbige sich durchaus fortgesetzt attraktiv welches die ersten Lungenkarzinom Symptome sind und wie diese gegenseitig merklich tätig sein? Dadurch lu...") [23:53:53] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 101472 [23:57:44] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.887207/1.95, alarm hl:tmp_free=19917M/100M, alarm hl:np_load_avg=0.781738/2.0, alarm hl:mem_free=285.000000M/350M, alarm hl:available=1/0 [23:59:46] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK