[00:15:57] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 301181 MB (5% inode=68%): [00:20:57] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10802.000000 [00:21:06] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10806.000000 [00:21:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10808.000000 [00:21:08] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10812.000000 [00:21:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10822.000000 [00:21:27] wikidata replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 6.000000 [00:21:57] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10860.000000 [00:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [00:24:07] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:26:27] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:26:28] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:26:47] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:29:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:29:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:32:27] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 26360 MB (4% inode=99%): [00:34:17] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [00:37:35] DaBPunkt: I was just talking in #wikimedia-operations that apparently some changes need to be made now that wikidata has made a few changes, are those already planned? [00:39:53] duh: I was told somewhen in the end of last year that wikidata will move SOMEWHEN. Few minutes ago I saw the import into s5 by luck. Fortunaly wikidata is still small and so I can re-import it quite easily tomorrow [00:40:10] but a little warning in advantage would have been cool [00:40:14] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [00:40:40] ah ok [00:40:48] if you can do it tomorrow that would be awesome :) [00:42:55] duh: lets see how the import on s5 works tonight [00:43:21] ok [00:53:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:04:06] DaB. * [Toolserver-announce] Wikidata-moving [01:04:33] wikidata replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2182.000000 [01:04:53] s5 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2206.000000 [01:06:24] nacht ts [01:16:14] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 304252 MB (5% inode=68%): [01:21:02] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14402.000000 [01:21:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14414.000000 [01:21:14] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14415.000000 [01:21:14] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14419.000000 [01:21:23] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14423.000000 [01:21:53] Load avg. on ortelius is WARNING: WARNING - load average: 23.79, 18.31, 11.43 [01:22:53] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14519.000000 [01:24:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [01:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:26:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:26:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:27:23] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3623.000000 [01:28:54] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3645.000000 [01:29:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:30:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:32:06] Marc A. Pelletier * Re: [Toolserver-l] Wikidata-moving [01:33:24] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 26070 MB (4% inode=99%): [01:34:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [01:39:13] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 68963 MB (7% inode=99%): [01:39:53] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2143 [01:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [01:40:53] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 41.29, 24.94, 19.23 [01:44:54] MySQL slave on z-dat-s2-b is OK: Uptime: 541086 Threads: 12 Questions: 421158167 Slow queries: 8947 Opens: 18770797 Flush tables: 1 Open tables: 64 Queries per second avg: 778.357 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1635 [01:46:53] Load avg. on ortelius is WARNING: WARNING - load average: 10.70, 19.62, 19.41 [01:51:41] Coren: Not to curb your enthusiam, but this treatment of Toolserver re: schema changes dates back years, even under River's adminship, so: Good luck! :-) [01:52:15] scfc_de`: They actually pay me to make sure that things like that don't happen. [01:52:28] Well, no. They *will* pay me for that. :-) [01:52:52] The ts isn't an opponent, it's our ally and needs to be treated as such. [01:53:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:54:53] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 41.51, 22.92, 19.81 [01:55:13] I don't think that's (was) opposition, just ignorance (because nobody paid for that :-)). But past is past, future is future, :-). [01:56:08] Oh, I /know/ there were no ill feelings -- but things like that grate and annoy, and are completely unnecessary. [01:56:26] Yep. [02:14:53] Load avg. on ortelius is WARNING: WARNING - load average: 14.91, 14.89, 15.68 [02:16:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 303816 MB (5% inode=68%): [02:16:54] Load avg. on ortelius is OK: OK - load average: 7.53, 12.57, 14.74 [02:21:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18002.000000 [02:21:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18013.000000 [02:21:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18015.000000 [02:21:13] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18019.000000 [02:21:14] [[Special:Log/newusers]] create 10 * Dogphoenix * (New user account) [02:21:23] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18023.000000 [02:22:09] [[User talk:Dogphoenix]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7822&rcid=21573 * Dogphoenix * (+1395) (Created page with "Phoenix Dog Boarding Just Got Brilliant! With its year-round sun and popular art scene, Phoenix is property to around 1,000,000 men and women. Considering that canine are rega...") [02:22:53] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18120.000000 [02:24:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [02:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:26:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:26:54] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:27:23] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7222.000000 [02:28:52] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7246.000000 [02:29:34] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:30:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:33:23] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 26118 MB (4% inode=99%): [02:34:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [02:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [02:48:06] [[Special:Log/delete]] delete 10 * Betacommand * (deleted "[[02User talk:Dogphoenix10]]": content was: "Phoenix Dog Boarding Just Got Brilliant! With its year-round sun and popular art scene, Phoenix is property to around 1,000,000 men and women. Considerin..." (and the only contributor was "[[Special:Contributions/Dogphoenix|Dogphoenix]]")) [02:48:27] [[Special:Log/block]] block 10 * Betacommand * (blocked [[02User:Dogphoenix10]] with an expiry time of infinite (account creation disabled): Spam) [02:53:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:16:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 303173 MB (5% inode=68%): [03:21:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21602.000000 [03:21:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21614.000000 [03:21:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21615.000000 [03:21:13] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21618.000000 [03:21:23] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21624.000000 [03:22:53] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21719.000000 [03:24:14] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:26:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:26:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:27:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10822.000000 [03:28:53] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10846.000000 [03:29:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:30:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:33:23] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 25663 MB (4% inode=99%): [03:34:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [03:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [03:53:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:07:53] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2093 [04:16:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 301634 MB (5% inode=67%): [04:21:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25202.000000 [04:21:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25215.000000 [04:21:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25217.000000 [04:21:13] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25219.000000 [04:21:23] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25224.000000 [04:22:53] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25320.000000 [04:24:14] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [04:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:26:43] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:26:53] MySQL slave on z-dat-s2-b is OK: Uptime: 550806 Threads: 8 Questions: 432331662 Slow queries: 9562 Opens: 18962324 Flush tables: 1 Open tables: 64 Queries per second avg: 784.907 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1485 [04:26:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:27:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14423.000000 [04:28:53] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14447.000000 [04:29:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:30:23] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:33:23] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 25403 MB (4% inode=99%): [04:34:24] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [04:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [04:53:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:16:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 302375 MB (5% inode=67%): [05:21:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28802.000000 [05:21:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28815.000000 [05:21:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28816.000000 [05:21:13] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28819.000000 [05:21:23] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28830.000000 [05:22:53] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28920.000000 [05:24:14] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [05:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:26:43] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:26:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:27:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18023.000000 [05:28:53] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18046.000000 [05:29:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:30:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:33:23] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 25234 MB (4% inode=99%): [05:35:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [05:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [05:53:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:16:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 301941 MB (5% inode=67%): [06:21:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32402.000000 [06:21:12] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32416.000000 [06:21:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32417.000000 [06:21:14] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32420.000000 [06:21:23] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32429.000000 [06:22:52] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32519.000000 [06:24:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [06:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:26:43] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:26:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:27:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21623.000000 [06:28:53] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21646.000000 [06:29:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:30:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:31:32] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [06:31:43] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [06:31:43] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [06:31:53] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [06:32:43] [[Special:Log/newusers]] create 10 * Pipat.poonP2 * (New user account) [06:33:23] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 25124 MB (4% inode=99%): [06:35:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [06:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [06:53:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:16:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 301624 MB (5% inode=67%): [07:21:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36002.000000 [07:21:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36016.000000 [07:21:14] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36017.000000 [07:21:14] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36019.000000 [07:21:23] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36030.000000 [07:22:53] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36120.000000 [07:24:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [07:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:26:43] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:26:53] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:27:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25224.000000 [07:28:53] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25247.000000 [07:29:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:30:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:31:32] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [07:31:42] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [07:31:43] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [07:31:54] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [07:33:22] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 24962 MB (4% inode=99%): [07:35:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [07:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [07:53:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:16:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 301292 MB (5% inode=67%): [08:16:52] @replag [08:16:52] Merlissimo: s1-rr-a-wd: 10h 56m 1s [+0.59 s/s]; s1-user-wd: 10h 56m 1s [+0.59 s/s]; s2-rr: 10s [-0.19 s/s]; s2-user: 10s [-0.19 s/s]; s2-user-c: error; s2-user-wd: 10h 56m 1s [+0.59 s/s]; s3-user: 7h 44m 29s [+0.41 s/s]; s4-user-wd: 10h 56m 1s [+0.59 s/s] [08:16:54] Merlissimo: s5-rr-a: 7h 48m 52s [+0.11 s/s]; s5-user: 7h 48m 52s [+0.11 s/s]; s5-user-c: 1m 28s [-0.00 s/s]; s5-user-wd: 7h 48m 50s [+0.42 s/s]; s6-user-wd: 10h 56m 1s [+0.59 s/s]; s7-user-wd: 10h 56m 1s [+0.59 s/s] [08:17:12] @replag [08:17:12] Merlissimo: s1-rr-a-wd: 10h 56m 20s [+0.98 s/s]; s1-user-wd: 10h 56m 20s [+0.98 s/s]; s2-user-c: error; s2-user-wd: 10h 56m 20s [+0.98 s/s]; s3-user: 7h 44m 48s [+0.98 s/s]; s4-user-wd: 10h 56m 20s [+0.98 s/s]; s5-rr-a: 7h 49m 11s [+0.98 s/s]; s5-user: 7h 49m 11s [+0.98 s/s] [08:17:13] Merlissimo: s5-user-c: 1m 25s [-0.15 s/s]; s5-user-wd: 7h 49m 9s [+0.98 s/s]; s6-user-wd: 10h 56m 20s [+0.98 s/s]; s7-user-wd: 10h 56m 20s [+0.98 s/s] [08:21:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39602.000000 [08:21:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39615.000000 [08:21:14] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39616.000000 [08:21:14] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39620.000000 [08:21:24] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39630.000000 [08:22:53] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39719.000000 [08:24:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [08:24:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:26:43] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:26:54] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:27:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:28:33] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28824.000000 [08:28:53] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28846.000000 [08:29:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:30:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:31:33] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [08:31:42] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [08:31:43] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [08:31:53] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [08:33:23] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 24721 MB (4% inode=99%): [08:35:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [08:40:13] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [08:53:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:17:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 300940 MB (5% inode=67%): [09:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43219.000000 [09:21:30] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43230.000000 [09:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43263.000000 [09:22:11] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43275.000000 [09:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43277.000000 [09:23:01] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43323.000000 [09:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:25:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [09:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:27:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:27:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32453.000000 [09:29:31] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32485.000000 [09:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:31:40] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [09:32:02] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [09:32:02] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [09:32:02] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [09:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 24482 MB (3% inode=99%): [09:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [09:41:11] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [10:15:56] [[Special:Log/newusers]] create 10 * Wyy065 * (New user account) [10:17:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 300421 MB (5% inode=67%): [10:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46820.000000 [10:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46832.000000 [10:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46864.000000 [10:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46876.000000 [10:22:31] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46889.000000 [10:23:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:23:22] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46939.000000 [10:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:25:10] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [10:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:27:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:27:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36054.000000 [10:29:31] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36085.000000 [10:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:31:41] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [10:32:01] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [10:32:02] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [10:32:02] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [10:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 24245 MB (3% inode=99%): [10:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [10:41:21] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [11:02:27] [[GeoHack]] ! 10https://wiki.toolserver.org/w/index.php?diff=7823&oldid=7664&rcid=21578 * 115.67.34.38 * (-9301) ('''GeoHack''' is a modified version of [[mw:Extension:Gis/map sources|map sources]] from [[w:User:Egil|Egil Kvaleberg]]'s [[mw:Extension:Gis|gis extension]]. It is designed to do simple HTML replacem) [11:04:25] [[GeoHack]] ! 10https://wiki.toolserver.org/w/index.php?diff=7824&oldid=7823&rcid=21579 * Phe * (+9301) (Undo revision 7823 by [[Special:Contributions/115.67.34.38|115.67.34.38]] ([[User talk:115.67.34.38|talk]])) [11:13:51] [[GeoHack]] ! 10https://wiki.toolserver.org/w/index.php?diff=7825&oldid=7824&rcid=21580 * 115.67.34.38 * (-9301) (== Query ===== language ===The requested language version (e.g.''language''.wikipedia.org), falls back to English if the template page does not exist.) [11:14:49] [[GeoHack]] ! 10https://wiki.toolserver.org/w/index.php?diff=7826&oldid=7825&rcid=21581 * 115.67.34.38 * (+8) (=== language ===The requested language version (e.g. <code>''language''.wikipedia.org</code>), falls back to English if the template page does not exist.) [11:17:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 300023 MB (5% inode=67%): [11:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50422.000000 [11:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50432.000000 [11:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50464.000000 [11:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50476.000000 [11:22:31] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50489.000000 [11:23:11] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50539.000000 [11:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:25:10] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [11:27:00] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:27:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:27:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39653.000000 [11:29:31] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39684.000000 [11:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:31:41] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [11:32:01] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [11:32:02] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [11:32:02] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [11:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 23992 MB (3% inode=99%): [11:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [11:41:21] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [11:44:03] [[Logging in]] !M 10https://wiki.toolserver.org/w/index.php?diff=7827&oldid=6995&rcid=21582 * Pipat.poonP2 * (-5936) (You might be asked to enter a passphrase when authenticating;thisis''not''your Toolserverpassword,it's only used to encrypt yourprivate key on your owncomputer,in case someone steals it.This passphras) [12:17:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 299608 MB (5% inode=67%): [12:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54022.000000 [12:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54033.000000 [12:22:02] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54064.000000 [12:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54076.000000 [12:22:31] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54089.000000 [12:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54139.000000 [12:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:25:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [12:27:02] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:27:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:27:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43253.000000 [12:29:32] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43285.000000 [12:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:31:41] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [12:32:01] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [12:32:01] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [12:32:01] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [12:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 23714 MB (3% inode=99%): [12:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [12:41:21] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [12:53:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:13:27] [[Logging in]] ! 10https://wiki.toolserver.org/w/index.php?diff=7828&oldid=7827&rcid=21583 * Phe * (+5936) (Undo revision 7827 by [[Special:Contributions/Pipat.poonP2|Pipat.poonP2]] ([[User talk:Pipat.poonP2|talk]])) [13:15:33] [[GeoHack]] ! 10https://wiki.toolserver.org/w/index.php?diff=7829&oldid=7826&rcid=21584 * Phe * (-8) (Undo revision 7826 by [[Special:Contributions/115.67.34.38|115.67.34.38]] ([[User talk:115.67.34.38|talk]])) [13:17:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 299164 MB (5% inode=67%): [13:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57622.000000 [13:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57632.000000 [13:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57664.000000 [13:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57676.000000 [13:22:31] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57690.000000 [13:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57739.000000 [13:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:25:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:27:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:27:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46853.000000 [13:29:31] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 46885.000000 [13:29:44] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:31:41] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [13:32:01] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [13:32:01] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [13:32:01] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [13:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 23464 MB (3% inode=99%): [13:36:22] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [13:40:11] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 68126 MB (6% inode=99%): [13:41:21] MySQL slave on z-dat-s3-a is CRITICAL: (Return code of 139 is out of bounds) [13:42:07] DaB. * Re: [Toolserver-announce] [Toolserver-l] sql-s5 (dewiki) will be read-only tomorrow [13:53:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:09:06] DaB. * Re: [Toolserver-announce] [Toolserver-l] sql-s5 (dewiki) will be read-only tomorrow [14:16:44] I will start to repair s3 now [14:17:10] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 298720 MB (5% inode=67%): [14:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61222.000000 [14:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61233.000000 [14:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61264.000000 [14:22:10] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61277.000000 [14:22:31] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61290.000000 [14:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61339.000000 [14:24:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:25:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [14:25:54] s3 is fixed, dropped wikidatawiki_p because there is no database left [14:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:27:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:27:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50454.000000 [14:29:31] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 50484.000000 [14:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:31:41] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:32:01] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [14:32:01] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [14:32:01] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [14:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 23205 MB (3% inode=99%): [14:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [14:41:21] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 38126 [14:51:42] [[Special:Log/newusers]] create 10 * Amira reza * (New user account) [14:53:01] Load avg. on ptolemy is WARNING: WARNING - load average: 17.37, 17.94, 14.05 [14:53:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:01:01] Load avg. on ptolemy is OK: OK - load average: 8.55, 13.87, 13.91 [15:17:10] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 298243 MB (5% inode=67%): [15:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64822.000000 [15:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64833.000000 [15:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [15:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64877.000000 [15:22:31] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64889.000000 [15:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64939.000000 [15:24:36] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:25:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [15:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:27:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:27:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:29:02] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54053.000000 [15:29:30] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 54084.000000 [15:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:31:41] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:32:01] APT on mayapple is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:32:01] APT on nightshade is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:32:01] APT on yarrow is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22948 MB (3% inode=99%): [15:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [15:38:41] APT on z-dat-s2-b is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:39:01] APT on mayapple is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:39:01] APT on nightshade is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:39:01] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:41:21] MySQL slave on z-dat-s3-a is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 11527 [15:53:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:08:01] Load avg. on ptolemy is WARNING: WARNING - load average: 16.52, 15.34, 11.63 [16:09:01] Load avg. on ptolemy is OK: OK - load average: 11.18, 14.13, 11.44 [16:17:10] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 297803 MB (5% inode=67%): [16:21:21] MySQL slave on z-dat-s3-a is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3369 [16:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68422.000000 [16:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68432.000000 [16:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [16:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68476.000000 [16:22:30] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68490.000000 [16:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68539.000000 [16:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:25:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [16:26:21] MySQL slave on z-dat-s3-a is OK: Uptime: 1051413 Threads: 24 Questions: 809726323 Slow queries: 38012 Opens: 7099942 Flush tables: 1 Open tables: 16384 Queries per second avg: 770.131 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1760 [16:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:27:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:27:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:28:50] [[Operators]] N 10https://wiki.toolserver.org/w/index.php?oldid=7830&rcid=21586 * Dab * (+1073) (first draft) [16:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57654.000000 [16:29:31] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 57686.000000 [16:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:31:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22706 MB (3% inode=99%): [16:36:20] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [16:53:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:59:59] * jeremyb_ waves DaBPunkt [17:00:17] * jeremyb_ wonders if nosy knows about https://otrs-wiki.wikimedia.org/wiki/Public_userlist [17:06:38] [[Operators]] 10https://wiki.toolserver.org/w/index.php?diff=7831&oldid=7830&rcid=21587 * MZMcBride * (+7) (minor fixes) [17:07:29] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Pipat.poonP210]] with an expiry time of infinite (account creation disabled): doesn't appear to be here to contribute productively) [17:08:21] [[GeoHack]] 10https://wiki.toolserver.org/w/index.php?diff=7832&oldid=7829&rcid=21589 * MZMcBride * (+9301) (properly revert) [17:09:08] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:115.67.34.3810]] with an expiry time of 1 year (account creation disabled): inappropriate behavior) [17:09:45] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Nancyenm10]]": spam) [17:09:57] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Nancyenm10]] with an expiry time of infinite (account creation disabled): spam) [17:10:29] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Certificateseo10]]": spam) [17:10:41] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Certificateseo10]] with an expiry time of infinite (account creation disabled): spam) [17:10:48] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Nancyyo10]]": spam) [17:10:54] @replag [17:10:55] Alchimista: s1-rr-a-wd: 19h 50m 3s [+1.00 s/s]; s1-user-wd: 19h 50m 3s [+1.00 s/s]; s2-rr: 29s [-0.35 s/s]; s2-user: 29s [-0.35 s/s]; s2-user-c: error; s2-user-wd: error; s4-user-wd: 19h 50m 3s [+1.00 s/s]; s5-rr-a: 16h 42m 54s [+1.00 s/s] [17:10:56] Alchimista: s5-user: 16h 42m 54s [+1.00 s/s]; s5-user-c: 52s [-0.00 s/s]; s5-user-wd: 16h 42m 52s [+1.00 s/s]; s6-user-wd: 19h 50m 3s [+1.00 s/s]; s7-user-wd: 19h 50m 3s [+1.00 s/s] [17:11:02] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Nancyyo10]] with an expiry time of infinite (account creation disabled): spam) [17:11:29] Goddamn it. [17:12:06] DaB. * [Toolserver-announce] Restart of mysql on cassia (s5) [17:12:26] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Betty Waltun10]]": spam) [17:12:43] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02Nur mit einer ausreichende Bonität bekommt man einen Online Kredit10]]": spam) [17:12:53] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Betty Waltun10]] with an expiry time of infinite (account creation disabled): spam) [17:13:26] Is https://wiki.toolserver.org/w/index.php?title=User_talk:Chihonglee&redirect=no&rcid=14468 spam? [17:14:06] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Autoshope10]]": spam) [17:14:21] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Autoshope10]] with an expiry time of infinite (account creation disabled): spam) [17:14:44] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Maillotsuk10]]": spam) [17:14:58] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Maillotsuk10]] with an expiry time of infinite (account creation disabled): spam) [17:15:08] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Washware10]]": spam) [17:15:21] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Washware10]] with an expiry time of infinite (account creation disabled): spam) [17:15:43] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Accidentattorney10]]": spam) [17:15:52] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Accidentattorney10]] with an expiry time of infinite (account creation disabled): spam) [17:15:58] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Carattorney10]]": spam) [17:16:10] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Carattorney10]] with an expiry time of infinite (account creation disabled): spam) [17:16:23] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Packages10]]": spam) [17:16:32] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Packages10]] with an expiry time of infinite (account creation disabled): spam) [17:16:52] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User talk:Chihonglee10]]": outside project scope) [17:17:10] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 296704 MB (5% inode=67%): [17:19:35] Rawr. [17:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 72022.000000 [17:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 72033.000000 [17:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [17:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 72076.000000 [17:22:30] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 72089.000000 [17:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 72139.000000 [17:24:30] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:25:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [17:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:27:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:27:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61254.000000 [17:29:40] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 61288.000000 [17:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:31:34] Is there a library or some code that exist that easily reads all internal links on a page, or otherwise parses text for internal links? [17:31:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:31:53] I'm trying to get a list of all internal links on the page, given the text of the page [17:32:00] The MediaWiki API can do that. [17:32:22] Look at action=parse or... [17:32:55] Susan: I'm doing this for millions of pages [17:33:02] I'm not going to call the API for each one [17:33:19] Internal links are tracked. [17:33:21] In a database table. [17:33:30] I'm working on a database dump [17:33:31] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22393 MB (3% inode=99%): [17:33:36] prop=links exposes them. [17:33:42] In the MediaWiki API. [17:33:49] You can also set up a local parser if you want. [17:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [17:37:24] Yetanotherx, in each dump directory there is file called xxx-yyyy-externallinks.sql.gz which is small and contains only external links as sql statement, you can parse that or import it in a db and query data with sql [17:37:54] He wants internal links. [17:38:05] phe: Thanks for the reverts on tswiki. I cleaned up a bit as well. :-) [17:38:38] then xxx-yyyy-pagelinks.sql.gz [17:39:54] Susan, spammer on the ts nowadays :/ [17:39:55] Yeah, I saw that. Unfortunately, it's SQL, which is a bit annoying for my needs. [17:40:36] I'm wondering how effective just a regex for \[\[[^:](.*?)|.*?\]\] would be [17:40:57] Depends how much you care about accuracy. [17:41:02] {{foo|1=link}} [17:41:11] Yetanotherx, but a lot easier than parsing the full text, I think it contains really all links, if you parse a dump you'll don't get links added through template [17:41:20] Or [[hello| goodbye]] [17:41:26] Or a trillion other edge cases. [17:41:56] phe. True. [17:42:22] Really don't like working with SQL, annoyingly enough. [17:42:28] Not all internal links are really tracked either. [17:42:38] * Susan shrugs. [17:42:47] +1 [17:43:07] Susan: ? [17:43:22] Depends what you consider an internal link. [17:43:46] [{{fullurl:{{PAGENAME}}|action=edit}} foo] is somewhat internal. [17:43:51] "ISBN 1234" is somewhat internal. [17:44:07] [[File:hello|link=goodbye]] is somewhat internal. [17:44:20] I'm not looking at those edge cases. [17:44:31] If you don't care about edge cases, regex is fine. [17:44:53] Edge cases such as [[ might be weird, though [17:45:34] [[{{ns:0}}hello]] [17:45:41] [[{{FULLPAGENAME}}]] [17:46:06] {{#if:{{{1|}}}|[[link]]|[[don't link{{!}}whattttt]]}} [17:46:40] The other disadvantage to regex is that you have to deal with "shared" [[ syntax. [17:46:41] eww [17:46:50] [[Category:foo]] isn't an internal link. [17:46:58] Either is [[File:foo]]. [17:50:59] Wikimarkup was designed as eternal damnation for people who know something about language design and parsing. :-) [17:53:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:17:11] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 296049 MB (5% inode=67%): [18:21:21] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75622.000000 [18:21:31] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75633.000000 [18:22:01] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [18:22:11] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75676.000000 [18:22:31] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75690.000000 [18:23:21] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75739.000000 [18:24:30] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:25:11] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [18:27:01] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:27:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:27:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:29:01] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64854.000000 [18:29:41] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 64887.000000 [18:29:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:31:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:33:30] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22090 MB (3% inode=99%): [18:36:21] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [18:53:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:05:13] hi guys [19:05:52] Query killer is killing this query http://pastebin.com/E7v0hShj, has something wrong with it to make it happen? [19:17:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295590 MB (5% inode=67%): [19:17:37] hi, is it possible in any way to get the code of https://toolserver.org/~freaky/range.php ? [19:18:00] the account expired in the last few days [19:21:23] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79223.000000 [19:21:43] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79242.000000 [19:22:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [19:22:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79277.000000 [19:22:43] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79302.000000 [19:24:13] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79398.000000 [19:24:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:25:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [19:27:04] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:27:04] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:27:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:27:58] re [19:28:24] Wiki13: Do you have an account? Then ~freaky/public_html/range.php should be readable to you. [19:29:03] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68454.000000 [19:29:43] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68491.000000 [19:29:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:31:43] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:33:16] Raylton_away: Doesn't look sane at all. [19:33:28] revision is usually a pretty big table. [19:33:35] And subqueries are usually pretty awful performance-wise. [19:33:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 21858 MB (3% inode=99%): [19:35:08] Susan, There is another way to do? [19:36:10] What is u_alchimista_temp_p? [19:36:23] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [19:36:34] You're looking for users in the revision table where the user ID is not in your temporary table. To what end? [19:36:49] Also, you have to account for rev_user = 0 in either SQL logic or app logic. [19:36:58] That would be an optimization, I imagine. [19:37:04] If you added WHERE rev_user != 0. [19:37:11] Because you're probably scanning a lot of rows needlessly... [19:37:17] Dunno if that'll actually help, though. [19:38:11] scfc_de: sorry for the late reply, but no, I don´t have any access on the Toolserver [19:38:21] so I can´t read it [19:39:01] the tool in question is really helpfull to some Dutch users for countervandalism [19:39:51] Was it released under a free license? [19:40:03] If so, any Toolserver can probably grab a copy of it for you. [19:40:05] Susan: postive join should be faster (and then substract from total) [19:40:07] Or a root, if necessary. [19:40:21] Merlissimo: Positive join? [19:40:25] Not sure what kind of join that is. [19:40:32] like inner joins [19:40:32] You mean a regular or inner join? [19:40:36] Okay. :-) [19:40:46] And yeah, joins are almost always faster than subqueries. [19:40:52] But I don't know what Raylton_away is trying to accomplish. [19:40:56] So it's difficult to optimize. ;-) [19:41:05] If the temp table is really small, it shouldn't really matter. [19:41:12] But I imagine it's big. [19:41:16] And revision is probably big and growing. [19:42:05] The user "freaky" has GPLv2 as default license [19:42:11] Susan, u_alchimista_temp_p is something like select * from user_groups where ug_groups = 'bot' [19:42:22] So you're trying to get every non-bot user? [19:42:27] With an edit? [19:42:46] yep [19:42:59] Including IPs? [19:43:36] You probably just want to do something like "select distinct rev_user from revision;" and then just exclude in app logic. [19:43:41] no [19:43:59] DaBPunkt: is it possible then to get a copy from the cool? [19:44:01] So then be sure to add rev_user = 0 to the bot user IDs. [19:44:15] tool* [19:44:53] Wiki13: the file is world-reable AFAIS. So any user can overtake it. [19:45:50] Susan: how can a bot have the user-id 0? [19:45:55] ah ok, but I don´t have any access on the Toolserver, so I can´t grabb it myself [19:46:00] :s [19:46:11] DaBPunkt: A bot can't. I'm saying he'll want to exclude rev_user = 0 in addition to excluding all the bot user IDs. [19:46:23] He has to exclude IPs in the SQL part or in later filtering. [19:46:31] Wiki13: than either request access (takes a couble of weeks) or find a ts-user who is willing to host it :) [19:46:50] Susan: a "exclude", ok, sorry [19:47:06] http://pastebin.com/raw.php?i=E7v0hShj [19:47:09] That was the original query. [19:47:17] NOT IN :-) [19:47:29] Wiki13: Do you need the source code of the tool or do you need it hosted? [19:48:49] ehm... I think the second option. [19:48:50] Wiki13: http://toolserver.org/~dab/freaky-range.php – but that can be only temporary stay there [19:49:01] (because I can ot maintence it) [19:51:30] Susan, I wanted: [19:51:35] the number of edits total (already got) [19:51:41] the number of edits of ips (already got) [19:51:48] the number of edits of robots (I could not) [19:51:54] the number of edits of non-robots (could not) [19:52:29] the 4th is easy, the 5 is hard [19:52:38] -1 [19:52:46] Susan, I'm having trouble the last two queries [19:52:47] Current robots are easy enough to get. [19:52:51] But historical data is the real bitch. [19:52:53] Right. [19:53:04] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:53:19] I do some similar work for en.wiki. [19:53:23] thanks, but how long will it be there DaBPunkt? otherwise It would be better to get the sourcecode to host it elsewhere [19:53:23] It's not fun or pretty. :-) [19:53:37] Mostly requires manually filtering out bot names and relying on user.user_editcount. [19:53:42] Which can be wildly inaccurate. [19:54:51] Wiki13: I would prefer if an user with php-experince would overtake it. But in theory it can stay there for a while [19:55:11] if you like the source you can have that too [19:56:43] I would like to have it to be honest [19:56:47] Susan, and there would be a way to create a list of users by number of edits? [19:56:48] If it were possible I could count edits with regex [19:57:29] What. [19:57:36] Raylton_away: Look at the user table. [19:57:43] user_editcount is an integer you can look up. [19:57:45] True story. [19:57:46] Wiki13: http://toolserver.org/~dab/freaky-range.php.source – don't forget to name "freaky" and the license in your copy [20:00:06] Susan, ok, thank you so much [20:00:09] ok guys, I restart s5 now [20:00:18] thx DaBPunkt [20:00:54] Raylton_away: https://en.wikipedia.org/wiki/Wikipedia:WBE <-- en.wiki version. [20:00:58] Read the caveats. ;-) [20:01:18] https://en.wikipedia.org/wiki/Wikipedia:List_of_Wikipedians_by_number_of_edits/Configuration [20:01:22] Query is there. [20:03:58] ok, already done [20:04:36] @replag [20:04:38] Susan: s1-rr-a-wd: 22h 43m 45s [+1.00 s/s]; s1-user-wd: 22h 43m 45s [+1.00 s/s]; s2-rr: 1m 24s [-0.05 s/s]; s2-user: 1m 24s [-0.05 s/s]; s2-user-c: error; s2-user-wd: error; s3-user: 15s [-0.44 s/s]; s4-user-wd: 22h 43m 45s [+1.00 s/s] [20:04:39] Susan: s5-rr-a: 19h 21m 56s [+0.78 s/s]; s5-user: 19h 21m 56s [+0.78 s/s]; s5-user-c: 3m 48s [+0.05 s/s]; s5-user-wd: 19h 36m 34s [+1.00 s/s]; s6-user-wd: 22h 43m 45s [+1.00 s/s]; s7-user-wd: 22h 43m 45s [+1.00 s/s] [20:05:13] s6 and s7 are +1s/s? [20:05:29] Susan: 1 thing at a time [20:05:43] :-) [20:06:19] Susan, thank you so much :) [20:06:29] No problem. [20:07:47] @replag [20:07:48] DaBPunkt: s1-rr-a-wd: 22h 46m 56s [+1.00 s/s]; s1-user-wd: 22h 46m 56s [+1.00 s/s]; s2-rr: 3m 22s [+0.62 s/s]; s2-user: 3m 22s [+0.62 s/s]; s2-user-c: error; s2-user-wd: 5h 39m 9s [-2.33 s/s]; s3-user: 37s [+0.12 s/s]; s4-user-wd: 22h 46m 56s [+1.00 s/s] [20:07:49] DaBPunkt: s5-rr-a: 18h 4m 46s [-24.24 s/s]; s5-user: 18h 4m 46s [-24.24 s/s]; s5-user-c: 6m 59s [+1.00 s/s]; s5-user-wd: 19h 39m 45s [+1.00 s/s]; s6-user-wd: 22h 46m 56s [+1.00 s/s]; s7-user-wd: 22h 46m 56s [+1.00 s/s] [20:17:13] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 295143 MB (5% inode=67%): [20:20:11] DaB. * Re: [Toolserver-announce] [Toolserver-l] Restart of mysql on cassia (s5) [20:21:23] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82823.000000 [20:21:44] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82842.000000 [20:22:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 20358.000000 [20:22:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82876.000000 [20:22:43] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82902.000000 [20:24:13] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82999.000000 [20:24:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:25:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [20:27:03] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:27:04] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:27:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:29:04] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43801.000000 [20:29:43] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 72091.000000 [20:29:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:31:43] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:32:40] ok, I re-import wikidata now everywhere (without s6). I also dropped the wikidatawiki-view everywhere to not serve non-consitend data [20:33:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 21540 MB (3% inode=99%): [20:34:17] where it is finisged before I go to sleep I will start replication. Where not it has to wait for the re-import tomorrow [20:36:23] MySQL slave on cassia is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 38734 [20:53:03] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:17:12] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 294577 MB (5% inode=67%): [21:21:23] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [21:21:43] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [21:22:03] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 16805.000000 [21:22:13] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [21:22:43] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [21:24:13] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [21:24:22] MySQL slave on cassia is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3301 [21:24:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:24:47] ok, z-dat-s2-b and daphne has wikidata back [21:25:03] s5 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2774.000000 [21:25:13] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [21:26:23] MySQL slave on cassia is OK: Uptime: 5006 Threads: 18 Questions: 84611535 Slow queries: 406 Opens: 697 Flush tables: 1 Open tables: 295 Queries per second avg: 16902.24 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1691 [21:27:03] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:27:03] s5 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1116.000000 [21:27:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:27:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:29:42] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 75691.000000 [21:29:44] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:31:43] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:33:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 21211 MB (3% inode=99%): [21:37:18] wikidata-import on thyme is done too [21:51:05] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2101 [21:53:05] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:56:36] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [22:17:15] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 293906 MB (5% inode=67%): [22:18:15] wikidata replag on daphne is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3506.000000 [22:21:35] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14089.000000 [22:21:45] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [22:22:05] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 18674.000000 [22:22:25] wikidata replag on daphne is OK: QUERY OK: SELECT ts_rc_age() returned 1691.000000 [22:22:45] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 24997.000000 [22:24:15] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [22:24:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:25:15] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [22:26:05] s5 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2093.000000 [22:26:34] MySQL slave on cassia is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1965 [22:27:05] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:27:35] MySQL slave on cassia is OK: Uptime: 8677 Threads: 17 Questions: 88099937 Slow queries: 613 Opens: 975 Flush tables: 1 Open tables: 421 Queries per second avg: 10153.271 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [22:27:45] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:27:55] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:28:05] s5 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1722.000000 [22:29:45] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 79291.000000 [22:29:46] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:31:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:31:51] wikidata-import on rosemary is done too (so both s1 have wikidata again) [22:33:45] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 20885 MB (3% inode=99%): [22:37:08] @replag [22:37:09] DaBPunkt: s1-rr-a: 34s [-]; s1-rr-a-wd: 3h 3m 7s [-]; s1-user: 4m 10s [-]; s1-user-wd: 7h 42m 53s [-]; s2-rr: 50m 40s [-]; s2-user: 50m 40s [-]; s2-user-c: error; s2-user-wd: 4h 58m 30s [-] [22:37:10] DaBPunkt: s3-user-wd: error; s4-user-wd: 59s [-]; s5-user-c: 1m 14s [-]; s6-user-wd: error; s7-user-wd: 5h 30m 36s [-] [22:37:29] @replag [22:37:29] DaBPunkt: s1-rr-a: 42s [+0.38 s/s]; s1-rr-a-wd: 3h 2m 13s [-2.60 s/s]; s1-user: 4m 6s [-0.19 s/s]; s1-user-wd: 7h 41m 34s [-3.80 s/s]; s2-rr: 50m 47s [+0.34 s/s]; s2-user: 50m 47s [+0.34 s/s]; s2-user-c: error; s2-user-wd: 4h 58m 32s [+0.10 s/s] [22:37:30] DaBPunkt: s3-user-wd: error; s4-user-wd: 60s [+0.05 s/s]; s5-user-c: 1m 11s [-0.14 s/s]; s6-user-wd: error; s7-user-wd: 5h 27m 55s [-7.75 s/s] [22:51:05] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2361 [22:53:05] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:02:06] MySQL slave on z-dat-s2-b is OK: Uptime: 617713 Threads: 15 Questions: 528242756 Slow queries: 10698 Opens: 20834515 Flush tables: 1 Open tables: 64 Queries per second avg: 855.158 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1799 [23:03:38] DaBPunkt: legoktm@nightshade:~$ sql wikidatawiki_p [23:03:38] ERROR 1049 (42000): Unknown database 'wikidatawiki_p' [23:05:32] hm [23:05:35] legoktm@nightshade:~$ mysql -h sql-s1-rr wikidatawiki_p <-- works fine [23:07:59] duh: that's a tricky problem. wikidata belonged to s3 but is outdated there now. It belongs now to s5, but have not arrived yet [23:08:10] hmm [23:08:23] so if i just need wikidata with no joins, which server should i connect to it on then? [23:08:47] we have wikidata at thyme, rosemary, daphne, z-dat-s2-b and z-dat-s7-a at the moment [23:09:19] duh: If you just need to run a query you can use anyone of it. If you like to write a tool please wait for 24h [23:09:22] ok, but all of those are outdated? [23:09:46] duh: no. [23:09:52] @replag [23:09:53] DaBPunkt: s1-rr-a-wd: 1h 49m 25s [-2.67 s/s]; s1-user: 4m 38s [+0.08 s/s]; s1-user-wd: 5h 30m 26s [-5.76 s/s]; s2-rr: 30m 42s [-0.46 s/s]; s2-user: 30m 42s [-0.46 s/s]; s2-user-c: error; s2-user-wd: 4h 26m 55s [-0.56 s/s]; s3-user-wd: error [23:09:54] DaBPunkt: s4-user-wd: 41s [-0.01 s/s]; s5-user-c: 58s [-0.05 s/s]; s6-user: 18s [-]; s6-user-wd: error; s7-user-wd: 3h 11m 15s [-3.29 s/s] [23:09:59] well its my bots that use it, but its no hurry if they dont run for 24h [23:10:22] daphne is at 41s, that's quite life [23:13:06] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1906 [23:17:15] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 292393 MB (5% inode=67%): [23:21:34] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4079.000000 [23:21:45] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 16681.000000 [23:22:05] MySQL slave on z-dat-s2-b is OK: Uptime: 618913 Threads: 13 Questions: 529721658 Slow queries: 10724 Opens: 20890463 Flush tables: 1 Open tables: 65 Queries per second avg: 855.890 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1751 [23:22:05] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15202.000000 [23:22:45] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8440.000000 [23:23:35] wikidata replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3320.000000 [23:24:15] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: Unknown database wikidatawiki_p [23:24:35] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:25:15] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [23:27:04] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:27:45] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:28:05] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:28:35] wikidata replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1639.000000 [23:29:45] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 82893.000000 [23:29:45] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:31:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:33:45] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 20513 MB (3% inode=99%): [23:39:45] wikidata replag on z-dat-s7-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3534.000000 [23:48:51] wikidata replag on z-dat-s7-a is OK: QUERY OK: SELECT ts_rc_age() returned 1707.000000 [23:53:05] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL