[07:20:04] ho [07:20:04] hi [07:20:04] a while back someone generated a dynamically-updated page on toolserv for me [07:20:04] but now it's dead [07:20:04] http://toolserver.org/~nikola/grep.php?pattern=^.{0,9}%24&lang=en&wiki=wikipedia&ns=6 [07:20:04] I would strongly appreciate it if someone could create a new version of this page, thank ys [07:20:31] 2013/08/02 05:24 WARN z-dat-s1-b MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3462 [07:20:33] 2013/08/02 05:32 OK z-dat-s1-b MySQL slave Uptime: 55661 Threads: 8 Questions: 35767407 Slow queries: 945 Opens: 2162 Flush tables: 1 Open tables: 622 Queries per second avg: 642.593 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1651 [07:21:01] [[Special:Log/newusers]] create 10 * Luciana327 * (New user account) [09:18:45] 2013/08/02 09:10 CRIT z-dat-s2-b MySQL slave (Service Check Timed Out) [10:15:49] 2013/08/02 10:09 CRIT yarrow aliasd Connection refused [11:30:55] 2013/08/02 11:27 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1991 [11:41:54] 2013/08/02 11:40 CRIT z-dat-s3-a MySQL slave (Service Check Timed Out) [11:41:54] 2013/08/02 11:40 ?? z-dat-s4-a /sql CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [11:41:54] 2013/08/02 11:39 CRIT z-dat-s7-a MySQL slave Can't connect to MySQL server on 'z-dat-s7-a' (111) [11:43:54] 2013/08/02 11:40 ?? hyacinth RAID CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [11:44:54] 2013/08/02 11:40 CRIT z-dat-s3-a MySQL Can't connect to MySQL server on 'z-dat-s3-a' (110) [11:44:54] 2013/08/02 11:40 CRIT z-dat-s4-a MySQL (Service Check Timed Out) [11:44:54] 2013/08/02 11:39 CRIT z-dat-s6-a MySQL Can't connect to MySQL server on 'z-dat-s6-a' (110) [11:44:54] 2013/08/02 11:39 CRIT z-dat-s6-a MySQL slave Can't connect to MySQL server on 'z-dat-s6-a' (110) [11:45:54] 2013/08/02 11:39 CRIT hyacinth SSH CRITICAL - Socket timeout after 10 seconds [11:45:54] 2013/08/02 11:40 CRIT z-dat-s4-a MySQL slave (Service Check Timed Out) [11:46:54] 2013/08/02 11:40 ?? hyacinth / CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [11:46:54] 2013/08/02 11:46 CRIT hyacinth /tmp CHECK_NRPE: Socket timeout after 30 seconds. [11:46:55] 2013/08/02 11:40 CRIT hyacinth SMTP CRITICAL - Socket timeout after 10 seconds [11:46:55] 2013/08/02 11:46 CRIT z-dat-s3-a /sql CHECK_NRPE: Socket timeout after 30 seconds. [11:46:55] 2013/08/02 11:46 CRIT z-dat-s6-a /sql CHECK_NRPE: Socket timeout after 30 seconds. [11:46:55] 2013/08/02 11:46 CRIT z-dat-s7-a /tmp CHECK_NRPE: Socket timeout after 30 seconds. [11:46:55] 2013/08/02 11:46 CRIT z-dat-s7-a Load avg. CHECK_NRPE: Socket timeout after 30 seconds. [11:46:56] 2013/08/02 11:39 CRIT z-dat-s7-a MySQL Can't connect to MySQL server on 'z-dat-s7-a' (111) [11:47:54] 2013/08/02 11:47 CRIT hyacinth / CHECK_NRPE: Socket timeout after 30 seconds. [11:47:55] 2013/08/02 11:46 CRIT hyacinth Environment IPMI CHECK_NRPE: Socket timeout after 30 seconds. [11:47:55] 2013/08/02 11:46 CRIT hyacinth Load avg. CHECK_NRPE: Socket timeout after 30 seconds. [11:47:55] 2013/08/02 11:46 CRIT hyacinth RAID CHECK_NRPE: Socket timeout after 30 seconds. [11:47:55] 2013/08/02 11:46 CRIT z-dat-s3-a / CHECK_NRPE: Socket timeout after 30 seconds. [11:47:55] 2013/08/02 11:46 CRIT z-dat-s3-a /tmp CHECK_NRPE: Socket timeout after 30 seconds. [11:47:55] 2013/08/02 11:46 CRIT z-dat-s3-a Load avg. CHECK_NRPE: Socket timeout after 30 seconds. [11:47:56] 2013/08/02 11:46 CRIT z-dat-s4-a / CHECK_NRPE: Socket timeout after 30 seconds. [11:47:56] 2013/08/02 11:46 CRIT z-dat-s4-a /sql CHECK_NRPE: Socket timeout after 30 seconds. [11:47:57] 2013/08/02 11:46 CRIT z-dat-s4-a /tmp CHECK_NRPE: Socket timeout after 30 seconds. [11:47:57] 2013/08/02 11:46 CRIT z-dat-s4-a Load avg. CHECK_NRPE: Socket timeout after 30 seconds. [11:47:58] 2013/08/02 11:46 CRIT z-dat-s6-a / CHECK_NRPE: Socket timeout after 30 seconds. [11:47:58] 2013/08/02 11:46 CRIT z-dat-s6-a /tmp CHECK_NRPE: Socket timeout after 30 seconds. [11:47:59] 2013/08/02 11:46 CRIT z-dat-s6-a Load avg. CHECK_NRPE: Socket timeout after 30 seconds. [11:48:55] 2013/08/02 11:48 OK hyacinth / DISK OK - free space: / 9334 MB (31% inode=86%): [11:48:55] 2013/08/02 11:48 OK hyacinth /tmp DISK OK - free space: /tmp 47021 MB (99% inode=99%): [11:48:55] 2013/08/02 11:47 OK hyacinth Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [11:48:55] 2013/08/02 11:47 OK hyacinth Load avg. OK - load average: 0.06, 0.20, 0.94 [11:48:55] 2013/08/02 11:48 OK hyacinth RAID OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [11:48:55] 2013/08/02 11:48 OK hyacinth SMTP SMTP OK - 0.078 sec. response time [11:48:55] 2013/08/02 11:48 OK hyacinth SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [11:48:56] 2013/08/02 11:47 OK z-dat-s3-a / DISK OK - free space: / 9334 MB (31% inode=86%): [11:48:56] 2013/08/02 11:48 OK z-dat-s3-a /sql DISK OK - free space: /sql 183282 MB (18% inode=99%): [11:48:57] 2013/08/02 11:47 OK z-dat-s3-a /tmp DISK OK - free space: /tmp 46985 MB (99% inode=99%): [11:48:57] 2013/08/02 11:47 OK z-dat-s3-a Load avg. OK - load average: 0.88, 0.41, 0.97 [11:48:58] 2013/08/02 11:47 OK z-dat-s4-a / DISK OK - free space: / 9334 MB (31% inode=86%): [11:53:55] 2013/08/02 11:53 OK z-dat-s6-a MySQL Uptime: 329 Threads: 5 Questions: 325 Slow queries: 0 Opens: 69 Flush tables: 1 Open tables: 58 Queries per second avg: 0.987 [11:53:55] 2013/08/02 11:53 WARN z-dat-s6-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2779 [11:55:55] 2013/08/02 11:54 OK z-dat-s4-a MySQL slave Uptime: 610253 Threads: 1 Questions: 20405048 Slow queries: 9 Opens: 101 Flush tables: 1 Open tables: 75 Queries per second avg: 33.437 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1627 [11:56:55] 2013/08/02 11:55 OK z-dat-s7-a MySQL Uptime: 481 Threads: 1 Questions: 6727 Slow queries: 0 Opens: 242 Flush tables: 1 Open tables: 229 Queries per second avg: 13.985 [11:56:55] 2013/08/02 11:55 WARN z-dat-s7-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2918 [11:57:55] 2013/08/02 11:57 OK z-dat-s6-a MySQL slave Uptime: 569 Threads: 9 Questions: 112465 Slow queries: 8 Opens: 230 Flush tables: 1 Open tables: 216 Queries per second avg: 197.653 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1523 [12:00:56] 2013/08/02 11:59 OK z-dat-s7-a MySQL slave Uptime: 721 Threads: 5 Questions: 121373 Slow queries: 6 Opens: 495 Flush tables: 1 Open tables: 482 Queries per second avg: 168.339 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1485 [12:08:56] 2013/08/02 12:08 OK z-dat-s3-a MySQL Uptime: 1239 Threads: 2 Questions: 10 Slow queries: 0 Opens: 17 Flush tables: 1 Open tables: 10 Queries per second avg: 0.8 [12:29:57] 2013/08/02 12:26 WARN z-dat-s5-b MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1948 [12:32:57] 2013/08/02 12:26 WARN z-dat-s5-b wikidata replag QUERY WARNING: 'SELECT ts_rc_age()' returned 2100.000000 [12:40:57] 2013/08/02 12:40 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3600 [12:51:58] 2013/08/02 12:51 OK z-dat-s3-a MySQL slave Uptime: 3820 Threads: 23 Questions: 3414096 Slow queries: 198 Opens: 19164 Flush tables: 1 Open tables: 12671 Queries per second avg: 893.742 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1683 [15:12:07] 2013/08/02 15:11 OK z-dat-s5-b MySQL slave Uptime: 1824423 Threads: 12 Questions: 2404943172 Slow queries: 17419 Opens: 1539490 Flush tables: 1 Open tables: 256 Queries per second avg: 1318.193 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1737 [15:12:08] 2013/08/02 15:11 OK z-dat-s5-b wikidata replag QUERY OK: 'SELECT ts_rc_age()' returned 1737.000000 [15:23:07] 2013/08/02 15:18 CRIT ortelius Load avg. CRITICAL - load average: 29.45, 25.83, 14.45 [15:24:07] 2013/08/02 15:23 WARN ortelius Load avg. WARNING - load average: 17.72, 23.50, 14.37 [15:27:07] 2013/08/02 15:26 OK ortelius Load avg. OK - load average: 3.64, 14.69, 12.52 [15:41:08] 2013/08/02 15:37 WARN web.amaranth fisheye.toolserver.org HTTP WARNING: HTTP/1.1 200 OK - 274 bytes in 15.845 second response time [15:47:07] 2013/08/02 15:46 OK web.amaranth fisheye.toolserver.org HTTP OK: HTTP/1.1 200 OK - 273 bytes in 9.988 second response time [16:10:09] @replag [16:10:10] Merlissimo: s1-rr-a-wd: 2w 3d 11h 41m 37s [+1.00 s/s]; s1-user-wd: 1d 19h 45m 49s [+1.00 s/s]; s2-rr: 7h 10m 39s [+0.36 s/s]; s2-user: 7h 10m 39s [+0.36 s/s]; s2-user-c: error; s2-user-wd: 3d 20h 29m 40s [+1.00 s/s]; s3-user: 36s [+0.00 s/s]; s3-user-wd: 3d 20h 29m 40s [+1.00 s/s] [16:10:11] Merlissimo: s4-user-wd: 1d 19h 45m 49s [+1.00 s/s]; s5-rr-a-wd: 1d 19h 45m 49s [+1.00 s/s]; s5-user: 7m 42s [+0.01 s/s]; s6-user-wd: 3d 20h 29m 41s [+1.00 s/s]; s7-user-wd: 3d 20h 29m 41s [+1.00 s/s] [18:33:48] amette: s2 is having problems. reading query result does not return anything (connecting and sending query to server works) [19:30:08] Hi Merlissimo : sorry, but I haven't really advanced to caring about databases yet... still working on Jira, etc. - right now wouldn't know what to touch for fixing that... :-| [19:30:27] should I shoot an e-mail to the other admins or do you do that? [19:31:08] i don't know which of them is available atm [19:31:42] at this time none of them seems to... just address us all and the first one availabe will take care of it [19:31:55] not working example is: mysql -hsql-s2 -e "select * from zhwiki_p.recentchanges limit 1;" [19:32:13] so probably you better write an e-mail as you can describe the problem better [19:33:26] the interesting thing is that there is the munin mysql plugin is not available on s2 [19:33:32] http://munin.toolserver.org/ [19:34:02] hmmm, true... that looks way less monitored... [19:34:18] so i would guess a file system problem [19:34:45] that shouldn't make the plugins not working, should it? [19:34:54] were they available earlier? [19:35:48] havn't watch there for some weeks, but before it was available [19:35:59] hmmm, ok [19:45:36] amette: i'll open a jira ticket and assign it to DaB. [22:03:16] [[Special:Log/newusers]] create 10 * Jackson491 * (New user account) [23:08:43] Merlissimo: ok [23:08:48] and good night everyone