[23:59:58] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [00:03:18] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3612 [00:03:37] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3620.000000 [00:03:47] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:13:26] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 194749 [00:21:57] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 245472 MB (4% inode=29%): [00:27:58] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.563965/1.95, alarm hl:tmp_free=35684M/100M, alarm hl:np_load_avg=0.533691/2.0, alarm hl:mem_free=233.000000M/350M, alarm hl:available=1/0 [00:30:58] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [00:41:27] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:44:57] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [00:50:06] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [00:56:27] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:00:57] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [01:03:27] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4801 [01:03:36] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [01:03:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4810.000000 [01:12:57] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.840820/1.95, alarm hl:tmp_free=35606M/100M, alarm hl:np_load_avg=0.850586/2.0, alarm hl:mem_free=266.000000M/350M, alarm hl:available=1/0 [01:13:27] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 196235 [01:13:57] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [01:22:57] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 245554 MB (4% inode=29%): [01:24:22] @replag [01:24:22] Krinkle: s1-rr-a: 1h 27m 41s [+0.19 s/s]; s1-user: 1h 27m 41s [+0.19 s/s]; s2-user: 34s [-0.05 s/s]; s3-rr-a: 31s [-0.01 s/s]; s3-user: 31s [-0.01 s/s]; s5-rr-a: 22s [+0.00 s/s]; s5-user: 22s [+0.00 s/s]; s6-rr-a: 2d 6h 34m 9s [+0.23 s/s] [01:24:23] Krinkle: s6-user: 2d 6h 34m 9s [+0.23 s/s] [01:41:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:42:36] /sql on thyme is WARNING: DISK WARNING - free space: /sql 192898 MB (20% inode=99%): [01:45:57] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [01:50:17] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [01:53:37] /sql on thyme is OK: DISK OK - free space: /sql 203907 MB (21% inode=99%): [01:56:46] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:58:57] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.525879/1.95, alarm hl:tmp_free=35222M/100M, alarm hl:np_load_avg=0.615723/2.0, alarm hl:mem_free=342.000000M/350M, alarm hl:available=1/0 [02:00:57] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [02:01:57] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [02:03:28] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6680 [02:03:47] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6699.000000 [02:06:37] /sql on thyme is WARNING: DISK WARNING - free space: /sql 189768 MB (19% inode=99%): [02:11:57] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.771484/1.95, alarm hl:tmp_free=35150M/100M, alarm hl:np_load_avg=0.730469/2.0, alarm hl:mem_free=266.000000M/350M, alarm hl:available=1/0 [02:13:27] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 197122 [02:22:57] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 244322 MB (4% inode=29%): [02:24:36] /sql on thyme is OK: DISK OK - free space: /sql 213961 MB (22% inode=99%): [02:41:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:45:57] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [02:50:27] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [02:50:43] 3(created) [ACCAPP-517] Account request to conduct a research on Indonesian Wikipedia quality improvement strategy; Account Approval; New Account <10https://jira.toolserver.org/browse/ACCAPP-517> (IvanLanin ) [02:56:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [02:57:42] [[User:IvanLanin]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7280&rcid=9730 * IvanLanin * (+34) (homewiki) [03:00:57] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [03:03:07] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.765625/1.95, alarm hl:tmp_free=34042M/100M, alarm hl:np_load_avg=0.783203/2.0, alarm hl:mem_free=329.000000M/350M, alarm hl:available=1/0 [03:03:58] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8528.000000 [03:04:27] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8546 [03:12:06] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [03:13:27] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 197134 [03:23:07] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.565918/1.95, alarm hl:tmp_free=33485M/100M, alarm hl:np_load_avg=0.670410/2.0, alarm hl:mem_free=239.000000M/350M, alarm hl:available=1/0 [03:23:57] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 243071 MB (4% inode=28%): [03:27:07] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [03:42:56] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:43:06] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.665527/1.95, alarm hl:tmp_free=33150M/100M, alarm hl:np_load_avg=0.648926/2.0, alarm hl:mem_free=305.000000M/350M, alarm hl:available=1/0 [03:46:17] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [03:47:07] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [03:50:27] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [03:56:56] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [04:01:07] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [04:04:27] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 10391 [04:04:46] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10399.000000 [04:06:07] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.718750/1.95, alarm hl:tmp_free=32914M/100M, alarm hl:np_load_avg=0.687012/2.0, alarm hl:mem_free=237.000000M/350M, alarm hl:available=1/0 [04:07:07] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [04:13:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 195057 [04:24:07] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 243375 MB (4% inode=28%): [04:30:12] [[Wiki server assignments]] ! 10https://wiki.toolserver.org/w/index.php?diff=7281&oldid=7246&rcid=9731 * 91.198.174.202 * (-1) (updated page) [04:36:08] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.696289/1.95, alarm hl:tmp_free=32457M/100M, alarm hl:np_load_avg=0.697754/2.0, alarm hl:mem_free=232.000000M/350M, alarm hl:available=1/0 [04:42:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:45:17] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.478516/1.10, alarm hl:np_load_long=0.876953/1.55, alarm hl:mem_free=7206.000000M/500M, alarm hl:tmp_free=14899M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.478516/1.00, alarm hl:np_load_long=0.876953/1.50, alarm hl:mem_free=7206.000000M/600M, alarm hl:tmp_free=1 [04:47:07] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [04:47:07] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [04:49:17] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [04:50:37] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [04:56:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [05:01:06] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.498535/1.95, alarm hl:tmp_free=31843M/100M, alarm hl:np_load_avg=0.876953/2.0, alarm hl:mem_free=174.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.498535/2.3, alarm hl:np_load_long=0.720215/2.5, alarm hl:cpu=63.000000/98, alarm hl:mem_free=174.000000M/200M, al [05:01:07] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [05:02:07] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [05:04:37] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 12355 [05:04:46] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 12358.000000 [05:14:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 189748 [05:18:56] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:21:18] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.849609/1.95, alarm hl:tmp_free=31274M/100M, alarm hl:np_load_avg=0.625000/2.0, alarm hl:mem_free=263.000000M/350M, alarm hl:available=1/0 [05:23:56] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [05:24:06] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 243262 MB (4% inode=28%): [05:28:17] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [05:37:17] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.505859/1.95, alarm hl:tmp_free=30856M/100M, alarm hl:np_load_avg=0.626465/2.0, alarm hl:mem_free=287.000000M/350M, alarm hl:available=1/0 [05:42:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:47:16] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [05:50:36] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [05:52:06] /sql on cassia is WARNING: DISK WARNING - free space: /sql 121966 MB (10% inode=99%): [05:56:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [06:01:16] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [06:04:36] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 13771 [06:04:46] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 13778.000000 [06:14:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 185155 [06:24:06] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 243147 MB (4% inode=28%): [06:26:57] /sql on thyme is WARNING: DISK WARNING - free space: /sql 192390 MB (20% inode=99%): [06:42:56] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:45:57] /sql on thyme is OK: DISK OK - free space: /sql 226162 MB (23% inode=99%): [06:47:17] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [06:50:36] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [06:56:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [07:01:17] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [07:04:36] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 15703 [07:04:47] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15702.000000 [07:14:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 179235 [07:24:17] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 243005 MB (4% inode=28%): [07:38:57] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:42:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:47:27] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [07:50:47] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [07:56:56] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [08:02:18] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [08:04:37] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16245 [08:05:06] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 16240.000000 [08:13:27] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [08:14:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 177443 [08:24:17] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 242831 MB (4% inode=28%): [08:42:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:44:27] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1842.000000 [08:47:28] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [08:50:57] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [08:56:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [09:02:29] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [09:04:39] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 15591 [09:05:18] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15583.000000 [09:15:37] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 175207 [09:20:28] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1792.000000 [09:24:17] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 242655 MB (4% inode=28%): [09:37:27] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1975.000000 [09:43:17] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:48:28] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [09:50:56] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [09:57:17] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:03:27] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [10:04:39] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 14727 [10:05:26] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14747.000000 [10:10:27] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3630.000000 [10:15:47] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 172089 [10:18:11] [[Special:Log/newusers]] create 10 * Airsoftguy7 * (New user account) [10:24:17] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 245622 MB (4% inode=29%): [10:28:27] SMTP on willow is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:29:19] SMTP on willow is OK: SMTP OK - 0.107 sec. response time [10:43:18] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:48:38] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [10:51:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [10:55:07] Load avg. on willow is CRITICAL: CRITICAL - load average: 88.15, 43.17, 20.50 [10:56:28] SMTP on willow is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:57:19] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:57:28] SMTP on willow is OK: SMTP OK - 7.977 sec. response time [11:00:29] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.252930/1.10, alarm hl:np_load_long=0.690430/1.55, alarm hl:mem_free=10061.000000M/500M, alarm hl:tmp_free=14515M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.252930/1.00, alarm hl:np_load_long=0.690430/1.50, alarm hl:mem_free=10061.000000M/600M, alarm hl:tmp_free [11:01:08] Load avg. on willow is WARNING: WARNING - load average: 11.00, 23.77, 19.67 [11:02:38] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [11:03:28] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [11:04:58] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 14960 [11:05:19] hello all [11:05:21] @replag [11:05:23] DaBPunkt: s1-rr-a: 4h 9m 5s [+0.07 s/s]; s1-user: 4h 9m 5s [+0.07 s/s]; s2-user: 2h 38m 20s [+0.00 s/s]; s2-user-c: 1h 14m 18s [+0.12 s/s]; s3-rr-a: 12s [-0.01 s/s]; s3-user: 12s [-0.01 s/s]; s5-user-c: 1h 14m 18s [+0.12 s/s]; s6-rr-a: 1d 23h 15m 53s [-0.62 s/s] [11:05:24] DaBPunkt: s6-user: 1d 23h 15m 53s [-0.62 s/s] [11:05:41] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14946.000000 [11:05:42] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.392578/1.10, alarm hl:np_load_long=0.858398/1.55, alarm hl:mem_free=9834.000000M/500M, alarm hl:tmp_free=14517M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.392578/1.00, alarm hl:np_load_long=0.858398/1.50, alarm hl:mem_free=9834.000000M/600M, alarm hl:tmp_free=1 [11:07:17] Load avg. on willow is OK: OK - load average: 4.07, 10.38, 14.73 [11:10:28] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4055.000000 [11:15:57] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 169782 [11:24:28] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 243519 MB (4% inode=28%): [11:43:28] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:48:38] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [11:51:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [11:57:28] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:03:39] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [12:05:06] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 14420 [12:06:07] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14442.000000 [12:11:06] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4752.000000 [12:16:06] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 168809 [12:23:06] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.487793/1.95, alarm hl:tmp_free=37339M/100M, alarm hl:np_load_avg=0.479980/2.0, alarm hl:mem_free=290.000000M/350M, alarm hl:available=1/0 [12:24:06] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [12:25:06] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 241674 MB (4% inode=28%): [12:28:56] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:29:07] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.601562/1.95, alarm hl:tmp_free=37285M/100M, alarm hl:np_load_avg=0.585449/2.0, alarm hl:mem_free=206.000000M/350M, alarm hl:available=1/0 [12:41:09] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3579.000000 [12:44:08] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:49:08] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [12:51:47] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [12:58:08] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:59:08] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1605.000000 [13:04:09] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [13:05:08] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 15780 [13:06:09] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15798.000000 [13:16:08] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 167824 [13:18:38] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [13:22:09] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.484375/1.95, alarm hl:tmp_free=36632M/100M, alarm hl:np_load_avg=0.530273/2.0, alarm hl:mem_free=327.000000M/350M, alarm hl:available=1/0 [13:25:08] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 239684 MB (4% inode=28%): [13:27:09] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [13:28:42] 3(created) [ACCAPP-518] Uploading templates at es.wiki; Account Approval; New Account <10https://jira.toolserver.org/browse/ACCAPP-518> (Mr.Ajedrez) [13:38:08] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.494141/1.95, alarm hl:tmp_free=36180M/100M, alarm hl:np_load_avg=0.562988/2.0, alarm hl:mem_free=236.000000M/350M, alarm hl:available=1/0 [13:44:08] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:45:38] SMF on hemlock is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [13:46:38] SMF on hemlock is OK: OK - all services online [13:48:41] [[User:Dab/Debian-Packages]] 10https://wiki.toolserver.org/w/index.php?diff=7282&oldid=7214&rcid=9733 * Dab * (+7) (/* C(++/#)/Mono */ ) [13:49:08] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [13:51:58] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [13:58:08] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [14:04:09] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [14:05:09] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 15653 [14:06:09] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15645.000000 [14:06:55] 3(resolved) [ENWPONE-29] Web tool appears to list duplicate projects for individual names, and many projects pages don't actually appear to exist in the toolserver db <10https://jira.toolserver.org/browse/ENWPONE-29> (CBM) [14:17:08] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 165742 [14:25:19] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 243148 MB (4% inode=28%): [14:38:28] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=3.275391/1.95, alarm hl:tmp_free=35713M/100M, alarm hl:np_load_avg=1.572754/2.0, alarm hl:mem_free=461.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=3.275391/2.3, alarm hl:np_load_long=0.867188/2.5, alarm hl:cpu=78.900000/98, alarm hl:mem_free=461.000000M/200M, al [14:39:28] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [14:44:28] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:45:28] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.816406/1.95, alarm hl:tmp_free=35685M/100M, alarm hl:np_load_avg=1.232422/2.0, alarm hl:mem_free=249.000000M/350M, alarm hl:available=1/0 [14:49:28] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [14:52:08] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [14:56:17] SMTP on willow is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:57:08] SMTP on willow is OK: SMTP OK - 0.175 sec. response time [14:58:28] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [15:04:39] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [15:05:18] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 15554 [15:06:28] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15563.000000 [15:14:27] @replag [15:14:27] DaBPunkt: s1-user: 4h 22m 10s [-]; s2-user: 2h 21m 27s [-]; s2-user-c: 16s [-]; s5-user-c: 16s [-]; s6-rr-a: 1d 20h 47m 38s [-]; s6-user: 1d 20h 47m 38s [-] [15:17:17] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 160893 [15:25:28] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 239736 MB (4% inode=28%): [15:30:55] @preplag [15:30:57] :/ [15:36:37] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.697266/1.10, alarm hl:np_load_long=0.796875/1.55, alarm hl:mem_free=8518.000000M/500M, alarm hl:tmp_free=14176M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.697266/1.00, alarm hl:np_load_long=0.796875/1.50, alarm hl:mem_free=8518.000000M/600M, alarm hl:tmp_free=1 [15:38:38] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [15:44:38] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:49:48] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [15:52:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [15:58:38] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [16:00:13] [[Debian/Installation]] N 10https://wiki.toolserver.org/w/index.php?oldid=7283&rcid=9734 * Dab * (+468) (Created page with "*Enable the "network/dhcp-server"-service on hemlock (make sure to disable it again at the end!). *Choose "installer" as user-account with a password of your choice (the user wil...") [16:00:53] [[Debian/Installation]] 10https://wiki.toolserver.org/w/index.php?diff=7284&oldid=7283&rcid=9735 * Dab * (+39) (+structure) [16:04:40] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [16:05:18] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17475 [16:06:49] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17500.000000 [16:08:33] [[Debian/Installation]] 10https://wiki.toolserver.org/w/index.php?diff=7285&oldid=7284&rcid=9736 * Dab * (+198) (*save*) [16:08:37] DaBPunkt: hi :) around ? [16:08:48] Toto_Azero: back in 1h [16:09:08] ok… it was just for a question [16:11:03] Ilmari Karonen * [Toolserver-l] Rename MMPs to "public tools" [was: Toolserver user phe account expired] [16:17:18] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 152449 [16:22:44] will it be easy to move stuff to MMP? [16:25:01] I have tools in three parts: one is several python scripts as bots and they can be the easiest one. the second is a django project with some meta modules coupled. the third is a modified mediawiki and some bots are running as maintenance scripts [16:25:27] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 238555 MB (4% inode=28%): [16:45:07] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:48:35] [[Special:Log/newusers]] create 10 * VCP510-exam * (New user account) [16:50:06] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [16:53:06] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [16:53:17] [[Special:Log/newusers]] create 10 * Coltonjoyce959 * (New user account) [16:59:07] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [17:05:17] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [17:06:06] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17349 [17:07:16] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17339.000000 [17:15:17] DaBPunkt: there ? [17:18:06] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 119715 [17:26:06] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 238347 MB (4% inode=28%): [17:28:44] [[VCP510 exam]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7286&rcid=9739 * VCP510-exam * (+238) (Created page with "Just $1 for got CertClick VMware VCP510 exam , The VCP510 test contains real VCP510 Certification Questions an...") [17:32:27] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.726562/1.95, alarm hl:tmp_free=48689M/100M, alarm hl:np_load_avg=0.656738/2.0, alarm hl:mem_free=316.000000M/350M, alarm hl:available=1/0 [17:34:27] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [17:37:29] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.401855/1.95, alarm hl:tmp_free=48699M/100M, alarm hl:np_load_avg=0.511719/2.0, alarm hl:mem_free=285.000000M/350M, alarm hl:available=1/0 [17:45:29] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:50:29] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [17:52:08] /sql on cassia is WARNING: DISK WARNING - free space: /sql 122758 MB (10% inode=99%): [17:53:48] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [17:59:08] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:59:29] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [18:05:25] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [18:06:26] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17341 [18:07:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 17355.000000 [18:08:54] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [18:18:25] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 110567 [18:26:26] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 238214 MB (4% inode=28%): [18:33:40] re [18:45:34] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:50:02] Krinkle * Re: [Toolserver-l] Rename MMPs to "public tools" [was: Toolserver user phe account expired] [18:50:35] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [18:54:04] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [18:59:02] Danny B. * Re: [Toolserver-l] Rename MMPs to "public tools" [was: Toolserver user phe account expired] [18:59:35] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [19:05:55] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [19:07:24] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 18086 [19:07:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18105.000000 [19:09:54] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 127238 MB (20% inode=99%): [19:18:34] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 103629 [19:22:17] [[Debian/Installation]] 10https://wiki.toolserver.org/w/index.php?diff=7287&oldid=7285&rcid=9740 * Dab * (+315) (/* Pre-puppet */ *save*) [19:26:34] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 238038 MB (4% inode=28%): [19:27:02] Ilmari Karonen * Re: [Toolserver-l] Rename MMPs to "public tools" [was: Toolserver user phe account expired] [19:37:47] 3(created) [TS-1387] Multi-maintainer project for Commons MIME type statistics; Toolserver: Accounts; Minor Task <10https://jira.toolserver.org/browse/TS-1387> (Ilmari Karonen) [19:38:08] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.471680/1.95, alarm hl:tmp_free=48070M/100M, alarm hl:np_load_avg=0.488281/2.0, alarm hl:mem_free=280.000000M/350M, alarm hl:available=1/0 [19:39:07] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [19:42:01] multi-maintainer project != non-expiring tool [19:43:08] What happens if I find an XSS venerability, doesn't that mean the tool get permanently shutdown? [19:46:07] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:51:16] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [19:54:46] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:00:07] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [20:06:07] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [20:07:37] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 18958 [20:08:07] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18964.000000 [20:10:07] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126963 MB (20% inode=99%): [20:18:38] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 95431 [20:26:37] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 237858 MB (4% inode=28%): [20:40:40] [[Debian/Installation]] 10https://wiki.toolserver.org/w/index.php?diff=7288&oldid=7287&rcid=9741 * Dab * (+54) (/* Pre-puppet */ ) [20:44:26] s1 replag on rosemary is CRITICAL: (Service Check Timed Out) [20:44:36] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 209.000000 [20:44:49] [[Debian/Groups]] N 10https://wiki.toolserver.org/w/index.php?oldid=7289&rcid=9742 * Dab * (+315) (first draft) [20:46:26] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:46:28] [[Debian/Groups]] 10https://wiki.toolserver.org/w/index.php?diff=7290&oldid=7289&rcid=9743 * Dab * (+76) (+already used numbers) [20:51:27] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [20:52:27] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.404785/1.95, alarm hl:tmp_free=47702M/100M, alarm hl:np_load_avg=0.445312/2.0, alarm hl:mem_free=348.000000M/350M, alarm hl:available=1/0 [20:53:02] Krinkle * Re: [Toolserver-l] Rename "MMP" [20:54:57] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:56:27] Sun Grid Engine execd on willow is OK: testqueue@willow disabled: medium-sol@willow OK: longrun-sol@willow OK [20:58:44] [[Special:Log/newusers]] create 10 * Jwqdg7 * (New user account) [20:59:19] [[Debian/Groups]] 10https://wiki.toolserver.org/w/index.php?diff=7291&oldid=7290&rcid=9745 * Dab * (+11) (/* 200-299 */ +munin) [21:00:27] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [21:06:26] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [21:07:37] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 19281 [21:08:18] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 19293.000000 [21:09:27] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.819336/1.10, alarm hl:np_load_long=0.677735/1.55, alarm hl:mem_free=9359.000000M/500M, alarm hl:tmp_free=13668M/200M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.819336/1.00, alarm hl:np_load_long=0.677735/1.50, alarm hl:mem_free=9359.000000M/600M, alarm hl:tmp_free=1 [21:10:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126676 MB (20% inode=99%): [21:10:27] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [21:16:42] [[Debian/Groups]] 10https://wiki.toolserver.org/w/index.php?diff=7292&oldid=7291&rcid=9746 * Dab * (+13) (/* 200-299 */ +oident) [21:18:46] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 85427 [21:26:47] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 237674 MB (4% inode=28%): [21:41:12] [[Debian/Groups]] 10https://wiki.toolserver.org/w/index.php?diff=7293&oldid=7292&rcid=9747 * Dab * (+22) (/* 200-299 */ ) [21:46:27] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:51:27] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [21:55:17] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [21:59:06] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:59:06] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:59:47] SMF on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:47] SMF on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:47] / on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:47] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:47] /tmp on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:47] SMF on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:48] /sql on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:57] / on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:57] SMTP on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:59:57] /tmp on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:57] SMF on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:59:57] SMF on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:07] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:00:07] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:00:07] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:07] Load avg. on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:07] /tmp on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:17] Load avg. on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:17] / on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:17] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:27] Load avg. on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:27] /tmp on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:27] / on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:27] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [22:00:57] MySQL on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [22:00:57] s4 replag on z-dat-s4-a is CRITICAL: (Service Check Timed Out) [22:00:57] /tmp on hyacinth is OK: DISK OK - free space: /tmp 1809 MB (100% inode=99%): [22:00:57] / on hyacinth is OK: DISK OK - free space: / 8292 MB (27% inode=85%): [22:01:06] MySQL slave on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [22:01:32] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 74557 MB (18% inode=99%): [22:01:32] / on z-dat-s4-a is OK: DISK OK - free space: / 8292 MB (27% inode=85%): [22:01:32] /tmp on z-dat-s6-a is OK: DISK OK - free space: /tmp 1810 MB (99% inode=99%): [22:01:32] SMF on z-dat-s6-a is OK: OK - all services online [22:01:32] SMF on z-dat-s4-a is OK: OK - all services online [22:01:37] /tmp on z-dat-s4-a is OK: DISK OK - free space: /tmp 1808 MB (99% inode=99%): [22:01:37] s4 replag on z-dat-s4-a is OK: QUERY OK: SELECT ts_rc_age() returned 168.000000 [22:01:37] SMF on z-dat-s3-a is OK: OK - all services online [22:01:37] SMF on hyacinth is OK: OK - all services online [22:01:37] / on z-dat-s7-a is OK: DISK OK - free space: / 8292 MB (27% inode=85%): [22:01:37] MySQL on z-dat-s3-a is OK: Uptime: 2252453 Threads: 24 Questions: 2492660415 Slow queries: 139470 Opens: 23738022 Flush tables: 1 Open tables: 16384 Queries per second avg: 1106.642 [22:01:37] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 91383 MB (22% inode=99%): [22:01:38] Load avg. on z-dat-s4-a is OK: OK - load average: 0.50, 1.58, 2.47 [22:01:47] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [22:01:47] SMTP on z-dat-s6-a is OK: SMTP OK - 0.004 sec. response time [22:01:48] Load avg. on z-dat-s7-a is OK: OK - load average: 0.78, 1.62, 2.48 [22:01:48] / on z-dat-s6-a is OK: DISK OK - free space: / 8292 MB (27% inode=85%): [22:01:48] MySQL slave on z-dat-s3-a is OK: Uptime: 2252463 Threads: 24 Questions: 2492665379 Slow queries: 139471 Opens: 23738065 Flush tables: 1 Open tables: 16384 Queries per second avg: 1106.639 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 272 [22:01:48] /sql on z-dat-s6-a is OK: DISK OK - free space: /sql 143571 MB (14% inode=99%): [22:01:57] MySQL on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [22:01:57] MySQL slave on z-dat-s7-a is CRITICAL: (Service Check Timed Out) [22:01:57] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [22:01:57] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [22:01:57] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [22:01:58] Load avg. on z-dat-s6-a is OK: OK - load average: 1.13, 1.67, 2.48 [22:01:58] /tmp on z-dat-s7-a is OK: DISK OK - free space: /tmp 1874 MB (99% inode=99%): [22:01:58] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [22:02:07] MySQL on z-dat-s7-a is OK: Uptime: 2252480 Threads: 8 Questions: 817239438 Slow queries: 70563 Opens: 7125108 Flush tables: 1 Open tables: 6533 Queries per second avg: 362.817 [22:02:07] MySQL slave on z-dat-s7-a is OK: Uptime: 2252480 Threads: 8 Questions: 817239440 Slow queries: 70563 Opens: 7125108 Flush tables: 1 Open tables: 6533 Queries per second avg: 362.817 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 188 [22:02:17] SMF on z-dat-s7-a is OK: OK - all services online [22:06:27] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [22:07:37] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20778 [22:09:17] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 20816.000000 [22:10:06] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126433 MB (20% inode=99%): [22:18:56] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 62966 [22:26:47] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 237513 MB (4% inode=28%): [22:46:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:51:37] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [22:54:02] Ryan Kaldari * Re: [Toolserver-l] Rename "MMP" [22:55:27] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [22:58:03] OQ * Re: [Toolserver-l] Rename "MMP" [22:58:46] 3(updated) [ACCAPP-499] Account for Wikisource-specific IRC bot <10https://jira.toolserver.org/browse/ACCAPP-499> (inductiveload) [23:00:37] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [23:06:42] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default offline: svc:/system/cluster/scsymon-srv:default [23:07:52] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 21754 [23:08:22] /sql on thyme is WARNING: DISK WARNING - free space: /sql 194037 MB (20% inode=99%): [23:09:22] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21793.000000 [23:10:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126268 MB (20% inode=99%): [23:19:01] MySQL slave on z-dat-s6-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 45856 [23:20:22] /sql on thyme is OK: DISK OK - free space: /sql 222513 MB (23% inode=99%): [23:26:53] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 237446 MB (4% inode=28%): [23:32:22] /sql on thyme is WARNING: DISK WARNING - free space: /sql 190956 MB (19% inode=99%): [23:41:10] nacht ts [23:46:22] /sql on thyme is OK: DISK OK - free space: /sql 228850 MB (23% inode=99%): [23:46:43] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:51:42] DiskSuite on turnera is CRITICAL: CRITICAL - submirror d42 of mirror d40 is Needs and submirror d32 of mirror d30 is Needs and submirror d22 of mirror d20 is Needs and submirror d12 of mirror d10 is Needs [23:56:22] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk