[00:07:30] 2013/07/05 00:07 WARN wolfsbane /tmp DISK WARNING - free space: / 4018 MB (13% inode=92%): [00:08:30] 2013/07/05 00:07 WARN wolfsbane / DISK WARNING - free space: / 3993 MB (13% inode=92%): [07:07:53] 2013/07/05 07:01 WARN z-dat-s2-b wikidata replag QUERY WARNING: 'SELECT ts_rc_age()' returned 2034.000000 [07:18:50] 2013/07/05 07:18 OK z-dat-s2-b wikidata replag QUERY OK: 'SELECT ts_rc_age()' returned 1742.000000 [07:41:52] 2013/07/05 07:40 CRIT z-dat-s4-a /sql CHECK_NRPE: Socket timeout after 30 seconds. [07:42:52] 2013/07/05 07:42 OK yarrow aliasd TCP OK - 0.002 second response time on port 984 [500 Not found.] [07:43:51] 2013/07/05 07:40 CRIT z-dat-s3-a MySQL Can't connect to MySQL server on 'z-dat-s3-a' (111) [07:43:51] 2013/07/05 07:40 CRIT z-dat-s3-a MySQL slave Can't connect to MySQL server on 'z-dat-s3-a' (111) [07:43:51] 2013/07/05 07:42 WARN z-dat-s4-a /sql DISK WARNING - free space: /sql 66210 MB (10% inode=99%): [08:09:51] 2013/07/05 08:09 OK z-dat-s3-a MySQL Uptime: 1634 Threads: 30 Questions: 996 Slow queries: 3 Opens: 174 Flush tables: 1 Open tables: 146 Queries per second avg: 0.609 [08:09:51] 2013/07/05 08:09 OK z-dat-s3-a MySQL slave Uptime: 1634 Threads: 30 Questions: 996 Slow queries: 3 Opens: 174 Flush tables: 1 Open tables: 146 Queries per second avg: 0.609 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1753 [08:13:52] 2013/07/05 08:10 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1961 [08:50:57] [[Special:Log/newusers]] create 10 * JurgenNL * (New user account) [09:09:53] 2013/07/05 09:05 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 5.567 second response time [09:11:53] 2013/07/05 09:11 WARN ortelius toolserver.org HTTP HTTP WARNING: HTTP/1.1 200 OK - 239 bytes in 0.967 second response time [09:12:53] 2013/07/05 09:12 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 4.687 second response time [09:13:54] 2013/07/05 09:13 WARN ortelius toolserver.org HTTP HTTP WARNING: HTTP/1.1 200 OK - 239 bytes in 0.854 second response time [09:14:54] 2013/07/05 09:14 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 7.301 second response time [09:23:54] 2013/07/05 09:23 WARN ortelius toolserver.org HTTP HTTP WARNING: HTTP/1.1 200 OK - 239 bytes in 0.679 second response time [09:24:55] 2013/07/05 09:24 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 3.151 second response time [09:24:55] 2013/07/05 09:18 WARN z-dat-s2-b wikidata replag QUERY WARNING: 'SELECT ts_rc_age()' returned 1921.000000 [09:26:55] 2013/07/05 09:26 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.370 second response time [10:04:58] 2013/07/05 10:04 OK z-dat-s2-b wikidata replag QUERY OK: 'SELECT ts_rc_age()' returned 1700.000000 [10:55:02] 2013/07/05 10:54 OK z-dat-s4-a /sql DISK OK - free space: /sql 67425 MB (11% inode=99%): [10:56:02] 2013/07/05 10:49 CRIT wolfsbane Load avg. CRITICAL - load average: 15.27, 33.00, 21.29 [10:58:02] 2013/07/05 10:57 WARN wolfsbane Load avg. WARNING - load average: 6.18, 23.66, 19.20 [10:58:02] 2013/07/05 10:51 WARN z-dat-s2-b wikidata replag QUERY WARNING: 'SELECT ts_rc_age()' returned 2104.000000 [11:02:02] 2013/07/05 10:55 WARN z-dat-s4-a /sql DISK WARNING - free space: /sql 66947 MB (10% inode=99%): [11:04:02] 2013/07/05 11:03 OK wolfsbane Load avg. OK - load average: 4.64, 10.55, 14.50 [11:26:02] 2013/07/05 11:19 WARN z-dat-s2-b MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1837 [11:27:02] 2013/07/05 11:26 OK z-dat-s2-b MySQL slave Uptime: 4071815 Threads: 9 Questions: 3748307530 Slow queries: 75209 Opens: 29290907 Flush tables: 1 Open tables: 256 Queries per second avg: 920.549 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1693 [11:44:02] 2013/07/05 11:37 CRIT rosemary RAID ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [12:19:03] 2013/07/05 12:18 CRIT z-dat-s2-b wikidata replag QUERY CRITICAL: 'SELECT ts_rc_age()' returned 3636.000000 [12:30:04] 2013/07/05 12:23 WARN z-dat-s2-b MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2049 [12:51:46] 2013/07/05 12:39 WARN wolfsbane / DISK WARNING - free space: / 6252 MB (20% inode=92%): [12:51:46] 2013/07/05 12:40 WARN wolfsbane /tmp DISK WARNING - free space: / 6225 MB (20% inode=92%): [13:53:09] 2013/07/05 13:52 WARN z-dat-s2-b MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3573 [14:24:11] 2013/07/05 14:23 OK z-dat-s2-b MySQL slave Uptime: 4082435 Threads: 6 Questions: 3759359100 Slow queries: 75423 Opens: 29385568 Flush tables: 1 Open tables: 256 Queries per second avg: 920.861 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1761 [15:04:14] 2013/07/05 15:03 WARN z-dat-s2-b wikidata replag QUERY WARNING: 'SELECT ts_rc_age()' returned 3550.000000 [15:16:15] 2013/07/05 15:15 OK z-dat-s2-b wikidata replag QUERY OK: 'SELECT ts_rc_age()' returned 1757.000000 [17:13:21] 2013/07/05 17:06 WARN ptolemy Load avg. WARNING - load average: 18.48, 17.72, 12.19 [17:17:22] 2013/07/05 17:16 OK ptolemy Load avg. OK - load average: 10.51, 14.41, 12.13 [20:51:32] 2013/07/05 20:51 CRIT wolfsbane /tmp DISK CRITICAL - free space: / 3295 MB (10% inode=92%): [20:52:32] 2013/07/05 20:52 WARN wolfsbane /tmp DISK WARNING - free space: / 3335 MB (11% inode=92%): [20:57:32] 2013/07/05 20:56 CRIT wolfsbane / DISK CRITICAL - free space: / 3289 MB (10% inode=92%): [20:57:32] 2013/07/05 20:57 CRIT wolfsbane /tmp DISK CRITICAL - free space: / 3294 MB (10% inode=92%): [21:12:33] 2013/07/05 21:11 OK wolfsbane / DISK OK - free space: / 6325 MB (21% inode=92%): [21:12:33] 2013/07/05 21:12 OK wolfsbane /tmp DISK OK - free space: / 6318 MB (21% inode=92%): [21:22:33] 2013/07/05 21:15 WARN wolfsbane / DISK WARNING - free space: / 6260 MB (20% inode=92%): [21:22:34] 2013/07/05 21:16 WARN wolfsbane /tmp DISK WARNING - free space: / 6245 MB (20% inode=92%):