[00:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 107403 MB (17% inode=99%): [00:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:37:31] / on wolfsbane is WARNING: DISK WARNING - free space: / 5996 MB (20% inode=93%): [00:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [00:55:02] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4371 [00:56:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:57:02] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3489 [00:57:11] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3398.000000 [01:06:02] MySQL slave on thyme is OK: Uptime: 5115903 Threads: 3 Questions: 1979970934 Slow queries: 691353 Opens: 262984 Flush tables: 2 Open tables: 2776 Queries per second avg: 387.22 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1770 [01:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103628 MB (17% inode=99%): [01:06:11] s1 replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1715.000000 [01:07:08] @replag [01:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:37:32] / on wolfsbane is WARNING: DISK WARNING - free space: / 5906 MB (19% inode=93%): [01:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [01:56:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103800 MB (17% inode=99%): [02:09:06] hi [02:09:09] anyone around? [02:12:52] ToAruShiroiNeko: just ask your question [02:14:02] Load avg. on willow is WARNING: WARNING - load average: 17.02, 17.31, 14.39 [02:16:16] I am hoping someone will run a query that can help me fill http://meta.wikimedia.org/wiki/Requests_for_comment/Sysop_abuse_on_the_Turkish_Wikipedia/Voters [02:16:33] I want to identify voters whom dont have worthwhile edits anywhere [02:18:06] "access" is all noteworthy flags [02:18:10] like sysops and etc [02:19:02] Load avg. on willow is OK: OK - load average: 11.31, 14.99, 14.38 [02:22:32] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2023.000000 [02:23:02] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2042 [02:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:33:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:37:32] / on wolfsbane is WARNING: DISK WARNING - free space: / 5814 MB (19% inode=93%): [02:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [02:56:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:06:01] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 106616 MB (17% inode=99%): [03:13:32] / on wolfsbane is OK: DISK OK - free space: / 10001 MB (33% inode=93%): [03:18:02] Load avg. on willow is WARNING: WARNING - load average: 11.92, 15.51, 14.60 [03:19:02] Load avg. on willow is OK: OK - load average: 10.82, 14.57, 14.32 [03:22:32] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3317.000000 [03:23:02] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3314 [03:29:09] @replag [03:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:32:02] /sql on thyme is WARNING: DISK WARNING - free space: /sql 120678 MB (12% inode=99%): [03:33:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:34:22] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:36:02] / on damiana is WARNING: DISK WARNING - free space: / 14199 MB (19% inode=95%): [03:39:02] / on damiana is OK: DISK OK - free space: / 23165 MB (32% inode=95%): [03:49:11] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [03:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [03:56:11] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 106875 MB (17% inode=99%): [04:11:06] @replag [04:11:21] yay, all relatively low [04:15:02] MySQL slave on rosemary is OK: Uptime: 11278657 Threads: 28 Questions: 4566653146 Slow queries: 963766 Opens: 58885 Flush tables: 6 Open tables: 3434 Queries per second avg: 404.893 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1775 [04:15:32] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1733.000000 [04:30:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:52:22] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [04:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [04:56:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 105436 MB (17% inode=99%): [05:24:22] Load avg. on ortelius is WARNING: WARNING - load average: 15.09, 16.66, 13.14 [05:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:47:11] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 33920 MB (2% inode=98%): [05:49:21] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 22.62, 22.77, 20.11 [05:52:22] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [05:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [05:56:21] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 106192 MB (17% inode=99%): [06:06:22] Load avg. on ortelius is WARNING: WARNING - load average: 9.57, 16.95, 19.25 [06:17:02] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 37349 MB (9% inode=99%): [06:25:22] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 135085 MB (13% inode=99%): [06:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:52:22] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [06:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [06:56:21] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 106748 MB (17% inode=99%): [07:06:22] Load avg. on ortelius is WARNING: WARNING - load average: 12.61, 15.74, 16.32 [07:09:21] Load avg. on ortelius is OK: OK - load average: 6.51, 11.82, 14.67 [07:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:52:22] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [07:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [07:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 106430 MB (17% inode=99%): [08:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:33:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:38:02] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 22455 MB (5% inode=99%): [08:52:22] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [08:53:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [08:56:02] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 26423 MB (6% inode=99%): [08:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:57:01] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 22082 MB (5% inode=99%): [09:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103508 MB (17% inode=99%): [09:24:02] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 37208 MB (9% inode=99%): [09:24:02] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 98462 MB (10% inode=98%): [09:24:02] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 98462 MB (10% inode=98%): [09:30:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:52:21] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [09:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [09:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 103958 MB (17% inode=99%): [10:11:44] hmm? [10:11:49] hello? [10:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:52:22] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [10:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [10:56:21] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 106090 MB (17% inode=99%): [11:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:33:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:52:22] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [11:53:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [11:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104084 MB (17% inode=99%): [12:23:52] hello all [12:28:56] heyo [12:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:33:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:38:00] @replag [12:45:36] does anyone know why https://toolserver.org/~enwp10/ is being ropey? [12:45:46] pretty much everything is returning 403 Forbidden [12:46:09] ah, okay [12:46:11] I see - https://en.wikipedia.org/wiki/Wikipedia_talk:Version_1.0_Editorial_Team/Index#WP_1.0_bot_temporarily_down_2012-8-7 [12:52:21] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 28422 [12:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [12:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 105277 MB (17% inode=99%): [13:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:52:22] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 11913 [13:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [13:56:21] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:05:10] @replag [14:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 102962 MB (16% inode=99%): [14:17:22] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3383 [14:22:21] MySQL slave on z-dat-s3-a is OK: Uptime: 356022 Threads: 19 Questions: 307306025 Slow queries: 31048 Opens: 4211461 Flush tables: 1 Open tables: 16384 Queries per second avg: 863.165 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1508 [14:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:53:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [14:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104978 MB (17% inode=99%): [15:30:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:32:02] /sql on thyme is WARNING: DISK WARNING - free space: /sql 120078 MB (12% inode=99%): [15:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [15:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104736 MB (17% inode=99%): [16:24:22] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:28:51] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [16:30:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:45:09] Damn, what is the svn server slow....... [16:45:37] svn: Server sent unexpected return value (504 Gateway Time-out) in response to MERGE request [16:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [16:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104720 MB (17% inode=99%): [17:12:10] dschwen: This is probably the right channel. And no, I haven't put a JIRA ticket up yet. [17:12:29] apmon: I was in wm-labs [17:12:44] Just the emails we had with nosy [17:13:03] Do you want to put up the jira ticket? [17:13:08] ok, a ticket might be a good idea. DaB will see it [17:13:18] and there will be a record of the work the admins do [17:13:37] I could, but you should at least comment on it [17:13:50] reminds me, I need to update the maintanance jira ticket for the re-import [17:16:14] https://jira.toolserver.org/browse/TS-1494 [17:30:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:47:11] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 33902 MB (2% inode=98%): [17:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [17:56:21] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104710 MB (17% inode=99%): [18:25:22] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 134737 MB (13% inode=99%): [18:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:33:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:38:02] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24216 MB (5% inode=99%): [18:53:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [18:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104700 MB (17% inode=99%): [19:11:02] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 37999 MB (9% inode=99%): [19:30:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [19:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104692 MB (17% inode=99%): [20:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:34:11] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [20:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104681 MB (17% inode=99%): [21:24:02] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 100040 MB (10% inode=98%): [21:24:02] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 100040 MB (10% inode=98%): [21:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [21:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104672 MB (17% inode=99%): [22:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:33:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [22:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:06:02] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 104666 MB (17% inode=99%): [23:17:22] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [23:30:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:33:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:34:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:47:22] s4 replag on z-dat-s4-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2209.000000 [23:53:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [23:56:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default