[00:20:33] nacht ts [00:23:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [00:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:25:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:28:36] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339324 MB (6% inode=68%): [00:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:31:06] seth * Re: [Toolserver-l] List of tools on the toolserver - Help needed [00:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37953 MB (6% inode=99%): [00:38:06] Dennis Tobar * Re: [Toolserver-l] List of tools on the toolserver - Help needed [00:52:56] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:11:06] MZMcBride * Re: [Toolserver-l] List of tools on the toolserver - Help needed [01:23:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [01:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:25:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:28:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:28:37] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 341761 MB (6% inode=68%): [01:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37740 MB (6% inode=99%): [01:36:36] Load avg. on ortelius is WARNING: WARNING - load average: 16.15, 16.33, 13.44 [01:38:37] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 69610 MB (7% inode=99%): [01:41:41] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 38.78, 25.54, 17.85 [01:44:37] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1904 [01:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:53:37] MySQL slave on z-dat-s2-b is OK: Uptime: 368809 Threads: 10 Questions: 269375936 Slow queries: 5561 Opens: 12786664 Flush tables: 1 Open tables: 65 Queries per second avg: 730.394 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1583 [02:02:36] Load avg. on ortelius is WARNING: WARNING - load average: 10.95, 17.27, 19.68 [02:23:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [02:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:25:11] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:25:11] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:25:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:28:37] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 341105 MB (6% inode=68%): [02:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:30:39] Load avg. on ortelius is OK: OK - load average: 6.50, 12.32, 14.92 [02:31:08] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37749 MB (6% inode=99%): [02:50:06] Tim Landscheidt * Re: [Toolserver-l] List of tools on the toolserver - Help needed [02:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:54:14] Coren: Now if you could just give the parameters to http://www.mediawiki.org/wiki/Toolserver/List_of_Tools/tool some meaningful names as well ... :-) [02:54:51] Bah! Named parameters are for 1990's-era quiche eating programmers! :-) [02:54:59] (Yeah, good idea) [02:55:45] But not before I changed the entry I've been fiddling with :-). [02:58:37] Load avg. on ortelius is WARNING: WARNING - load average: 15.70, 16.53, 14.95 [03:00:21] Well, actually I think I name the parameters before. My eyes hurt :-). [03:01:42] {{done}} [03:04:47] scfc_de: hey! [03:05:14] do you have some time to chat about dbreps? [03:06:08] Coren: Thanks! [03:06:22] duh: Of course. What do you want to know? [03:06:26] scfc_de: Mediawiki is teh suxx0rz at structured data. [03:06:51] Coren: I believe there's a bug to enable SMW :-). [03:06:53] scfc_de: well I was talking with Susan earlier about making it easier to run reports cross-wiki [03:07:02] O [03:07:21] duh: You're legoktm? [03:07:23] I am :) [03:07:34] I'm pretty sure Wikidata is meant to be the Ubër-SMW eventually. [03:07:40] Yes. [03:07:59] Since all the scripts in general/ pull from dbreps.ini, they only work on enwiki [03:08:39] I was thinking of using getoptions to enable other ini files (also good for testing), but haven't implemented it yet. [03:08:57] You could also use different sections for different wikis in .ini. [03:09:33] right but then you have to edit the script itself [03:09:37] i was thinking of something like [03:09:46] python brokenredirects --wikidatawiki_p [03:09:57] probably --site=enwiki_p or something [03:10:57] Yep, that's what I meant with "other sections". In dbreps.ini, you then have a section [enwiki_p] with the config for enwiki_p, etc. [03:11:13] ah ok [03:11:43] Disclaimer: My Python knowledge is very limited. There's probably "the way" to do it, which I don't know anything about. [03:12:23] we would also need a helper script/class that would be called in each of the reports to check all of the args, and gets the right settings... [03:12:25] well, [03:12:38] "the way" would probably be to make it a package with setup.py and stuff [03:13:05] but thats way more work than i have time for, so i never pursued that option [03:13:17] That's what I wrote in the mail the other day, I believe: Modules for each report, and then one "distributor". [03:13:44] I can see if I can structure the code the next days something like that and then post a pull request. [03:14:19] It's very basic Python. [03:14:23] For the most part. [03:14:26] Most reports are very simple. [03:14:36] I know :-). [03:15:37] Load avg. on ortelius is OK: OK - load average: 5.27, 10.94, 14.71 [03:23:36] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:25:14] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:25:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:28:12] We could put the check that there is a section in dbreps.ini for the "--site" param in the global distributor and pass it as one of the parameters to the report class. [03:28:36] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 340464 MB (6% inode=68%): [03:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37163 MB (6% inode=99%): [04:11:31] scfc_de: oops just say that. yeah i think that would work. [04:11:35] saw* [04:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:23:47] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [04:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:25:27] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:28:46] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339648 MB (6% inode=68%): [04:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36955 MB (6% inode=99%): [04:36:08] I'll give it a try in the next days. [05:01:16] awesome :D [05:12:08] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1845 [05:13:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1836.000000 [05:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [05:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:28:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:28:57] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 340711 MB (6% inode=68%): [05:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36738 MB (6% inode=99%): [06:12:07] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2469 [06:13:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2476.000000 [06:23:08] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [06:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:28:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:28:57] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 340596 MB (6% inode=68%): [06:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36473 MB (5% inode=99%): [06:50:47] [[Category:Tools]] ! 10https://wiki.toolserver.org/w/index.php?diff=7815&oldid=7806&rcid=21559 * 69.171.172.63 * (-162) (Blanked the page) [06:52:13] [[Category:Tools]] ! 10https://wiki.toolserver.org/w/index.php?diff=7816&oldid=7815&rcid=21560 * Legoktm * (+162) (Undo revision 7815 by [[Special:Contributions/69.171.172.63|69.171.172.63]] ([[User talk:69.171.172.63|talk]])) [07:12:07] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2441 [07:13:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2440.000000 [07:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [07:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:28:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:29:46] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 340342 MB (6% inode=68%): [07:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36282 MB (5% inode=99%): [08:12:07] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2199 [08:13:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2232.000000 [08:23:17] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:23:48] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [08:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:28:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:28:08] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1748.000000 [08:28:08] MySQL slave on rosemary is OK: Uptime: 72854 Threads: 13 Questions: 56997802 Slow queries: 14960 Opens: 7346 Flush tables: 1 Open tables: 2963 Queries per second avg: 782.356 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1747 [08:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339953 MB (6% inode=68%): [08:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36070 MB (5% inode=99%): [09:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [09:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:29:46] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 339551 MB (6% inode=68%): [09:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 35909 MB (5% inode=99%): [09:53:17] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:17:47] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1923 [10:17:47] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1963.000000 [10:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [10:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:28:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 338956 MB (6% inode=68%): [10:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 35741 MB (5% inode=99%): [10:51:47] wikidata replag on z-dat-s2-b is OK: QUERY OK: SELECT ts_rc_age() returned 1762.000000 [10:57:47] MySQL slave on z-dat-s2-b is OK: Uptime: 401459 Threads: 11 Questions: 298312829 Slow queries: 6245 Opens: 13715197 Flush tables: 1 Open tables: 64 Queries per second avg: 743.71 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1720 [11:11:06] Silke Meyer * Re: [Toolserver-l] List of tools on the toolserver - Help needed [11:13:06] Silke Meyer * Re: [Toolserver-l] List of tools on the toolserver - Help needed [11:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:24:03] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [11:24:27] [[Special:Log/newusers]] create 10 * Myakubchy * (New user account) [11:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 338412 MB (6% inode=68%): [11:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 35481 MB (5% inode=99%): [12:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [12:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 337957 MB (6% inode=68%): [12:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 35249 MB (5% inode=99%): [12:47:46] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2170 [12:58:47] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2023.000000 [13:12:47] wikidata replag on z-dat-s2-b is OK: QUERY OK: SELECT ts_rc_age() returned 1748.000000 [13:19:47] MySQL slave on z-dat-s2-b is OK: Uptime: 409979 Threads: 10 Questions: 302617052 Slow queries: 6420 Opens: 13962175 Flush tables: 1 Open tables: 64 Queries per second avg: 738.128 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1772 [13:22:56] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [13:25:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 337385 MB (6% inode=68%): [13:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 34988 MB (5% inode=99%): [13:38:47] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 69720 MB (7% inode=99%): [14:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [14:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 336856 MB (6% inode=68%): [14:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 34739 MB (5% inode=99%): [14:43:41] I've trouble with a try ... finally clause with python and SGE, if the task is terminated in any way the finally clause run, but if the task is terminated with qdel, the finally clause never run, any idea ? [14:52:42] phe: Sorry, can't help you :/ [14:53:52] phe: this is because if the process is killed, python has no chance to run [14:54:10] phe: maybe there is a way to intercept some signal [14:55:13] hello all [14:55:41] Hi DaBPunkt [14:56:10] hello DaBPunkt [15:00:07] DaB. * Re: [Toolserver-l] List of tools on the toolserver - Help needed [15:02:06] DaB. * Re: [Toolserver-l] List of tools on the toolserver - Help needed [15:02:22] saper, yeps, was looking that, finally clause doesn't run on signal, and SGE use signal to terminate a task, the way is either to intercept signal or to install an atexit handler (both untested) [15:03:27] what do you plan to do if I may ask? [15:05:31] DaBPunkt, my bot queue task and do them one by one, I wanted to save the task queue in a finally clause and restore the task queue when the bot restart [15:07:05] phe: the list is in memory for normal? [15:07:09] another option is to use a user database, adding/removing jobs from it, but that was so simple to do it in a finally clause w/o user database ;( [15:07:13] DaBPunkt, yeps [15:09:02] DaBPunkt, do you know if SGE use SIGTERM to kill a task ? [15:09:33] you can tell SGE to send you a signal (so you have enough time to clean-up), but that helps not with crashes. Maybe you should use a file-based-approch if you don't like databases? [15:14:12] well, anyway the finally clause use already do that, I can save the file at each new request, a bit more IO than a database as I'll save the whole queue, but the queue is always small [15:14:48] but I see now the good point of using a user database :) [15:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:23:46] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [15:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 334579 MB (6% inode=68%): [15:31:08] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 34459 MB (5% inode=99%): [16:00:46] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2054 [16:01:57] DaBPunkt: just wondering again about amaranth. (AIUI, it's a fairly critical host?? so trying to make sure the ticket doesn't stall) [16:02:47] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1975.000000 [16:08:06] Peter Körner * Re: [Toolserver-l] New Rule: SGE-constraint for bots [16:09:46] wikidata replag on z-dat-s2-b is OK: QUERY OK: SELECT ts_rc_age() returned 1763.000000 [16:10:47] MySQL slave on z-dat-s2-b is OK: Uptime: 420239 Threads: 10 Questions: 307982899 Slow queries: 6580 Opens: 14330879 Flush tables: 1 Open tables: 64 Queries per second avg: 732.875 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1768 [16:14:33] jeremyb_: that's nice of you. The problem is that I have no idea how to provide the requested informations at the moment [16:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:23:46] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [16:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:25:36] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 333388 MB (6% inode=68%): [16:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 34181 MB (5% inode=99%): [16:39:06] Kai Nissen * Re: [Toolserver-l] New Rule: SGE-constraint for bots [16:53:06] DaB. * Re: [Toolserver-l] New Rule: SGE-constraint for bots [17:00:35] question, is there a way to run a db query to find pages that have interwiki links that are not from wikidata? [17:01:44] Betacommand: there is the langlinks table. [17:02:10] But of course there is no "This was cpied from wikidata"-column [17:03:02] DaBPunkt: was hoping there was some way to differentiate the data, but I might be SOL [17:03:30] IE wikidata not populate lanlinks [17:06:07] Betacommand: AFAIK the langlinks-table cointains only the "interwikilinks" that are in the article TEXT (=the old version). The "interwikilinks" from wikidata should not be in there. But of corse there is no way telling if a bot copied them over (for unknown reasons) [17:18:09] "Users can request their toolserver access to be restored by filing a request on JIRA in the TS project (use the "issues" menu on the top). Note that you may need to create a jira account first. " [17:18:10] awesome [17:19:07] "A system error occurred while creating your user account." [17:19:22] @replag [17:19:28] Betacommand: s1-rr-a-wd: 40s [-0.00 s/s]; s1-user-wd: 42s [-0.44 s/s]; s2-user-c: error; s2-user-wd: 41s [-0.00 s/s]; s3-user: 11s [-0.00 s/s]; s4-user-wd: 40s [-0.00 s/s]; s5-user-c: 2m 15s [+0.00 s/s]; s5-user-wd: 1m 13s [+0.00 s/s] [17:19:29] Betacommand: s6-user-wd: 42s [-0.00 s/s]; s7-user-wd: 40s [-0.00 s/s] [17:20:40] anybody about who can un-expire my account? or at least create a JIRA account for me so i can request it be done? [17:22:25] DaBPunkt: poke ^ [17:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [17:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [17:23:59] DaBPunkt: I dont think that is accurate any more, I just tried: select count(*) from langlinks where ll_from = 287847; on enwiki and got 46 it should be 0 [17:25:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:28:13] rofl http://toolserver.org/~nicdumz/expired.html "403 user account expired" [17:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 331754 MB (6% inode=68%): [17:31:08] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 33942 MB (5% inode=99%): [17:37:05] brion: What's funny? [17:37:42] scfc_de: the tool to check for expired accounts is on an expired account, so can't be accessed. [17:38:09] Ah, okay, I didn't know the origin of the URL. [17:38:32] knowing is half the battle :) [17:40:07] DaBPunkt: Did you brion's request for his account to be renewed? [17:49:30] brion: I believe Reedy and DanielK_WMDE should be able to renew your account as well, but DaBPunkt will probably be back later. [17:49:43] spiffy. i'll just hang out for a while, no rush :) [17:54:37] brion: long time no see :P [17:54:44] hey all [17:56:07] /tmp on ortelius is WARNING: DISK WARNING - free space: /tmp 3472 MB (17% inode=99%): [18:00:58] brion: Extended your account, Have to run now, cu [18:01:04] thanks DaBPunkt :D [18:06:15] hmm, still can't get in [18:06:24] http://toolserver.org/~brion/extensions/ shows me as still expired [18:06:34] and no luck logging in with what allegedly is my password [18:08:08] /tmp on ortelius is OK: DISK OK - free space: /tmp 7614 MB (38% inode=99%): [18:10:47] brion: The web server configuration gets only updated daily (or hourly?). But if you can't login, there seems to be another problem. [18:11:29] Received disconnect from 91.198.174.202: 2: Too many authentication failures for brion [18:13:51] Sorry, I don't know how to fix that (but I could tell you that the web configuration is updated daily at 0:00Z :-), modules/webserver/manifests/zws.pp). [18:14:24] i'll wait and see :) thanks [18:20:03] For what it's worth, looking at the LDAP entries I see no difference between your account and mine. Have you changed your SSH key? [18:20:23] i might have… [18:20:30] … but in theory i can still login with password right? [18:20:40] or did someone disable that and still require passwords for something? :P [18:22:01] I don't remember ever being able to login with passwords ... ?! [18:22:58] hmmmm [18:23:08] maybe i have to track down an old key. sighhhh [18:23:26] Probably best if you ask Reedy or DanielK_WMDE to just replace ~brion/.ssh/authorized_keys. [18:23:29] i ain't used tool server in ages, i just want the files used by gadgets to stay :P [18:23:40] fun fun [18:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [18:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:25:36] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 330912 MB (6% inode=68%): [18:31:08] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 33563 MB (5% inode=99%): [18:46:47] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2154 [18:47:47] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2122.000000 [18:52:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:15:48] Load avg. on ptolemy is WARNING: WARNING - load average: 17.34, 16.62, 11.70 [19:16:47] Load avg. on ptolemy is OK: OK - load average: 10.79, 14.97, 11.43 [19:16:47] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3608 [19:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [19:25:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:25:47] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3486 [19:26:26] re [19:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 330277 MB (6% inode=68%): [19:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 33284 MB (5% inode=99%): [19:47:46] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3048.000000 [19:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:11:47] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3626 [20:13:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3645.000000 [20:23:47] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [20:24:47] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3558.000000 [20:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:25:47] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3438 [20:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:29:47] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 329857 MB (6% inode=68%): [20:31:08] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 32997 MB (5% inode=99%): [20:38:47] MySQL slave on z-dat-s2-b is OK: Uptime: 436319 Threads: 12 Questions: 322931699 Slow queries: 6952 Opens: 15006837 Flush tables: 1 Open tables: 64 Queries per second avg: 740.127 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1782 [20:45:46] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2001 [20:55:47] MySQL slave on z-dat-s2-b is OK: Uptime: 437339 Threads: 10 Questions: 323290776 Slow queries: 6984 Opens: 15058888 Flush tables: 1 Open tables: 64 Queries per second avg: 739.222 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1795 [21:21:47] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2631 [21:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:23:57] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [21:24:57] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3302.000000 [21:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:29:07] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:29:57] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 329295 MB (6% inode=68%): [21:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 32741 MB (5% inode=99%): [21:40:57] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3607.000000 [21:41:57] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3589.000000 [21:42:57] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3602.000000 [21:52:46] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3651 [21:56:47] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3528 [21:57:57] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3525.000000 [22:22:47] MySQL slave on z-dat-s2-b is OK: Uptime: 442559 Threads: 7 Questions: 326087788 Slow queries: 7125 Opens: 15273447 Flush tables: 1 Open tables: 64 Queries per second avg: 736.823 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1641 [22:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:23:57] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:23:57] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [22:25:08] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:25:08] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:29:57] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 328536 MB (6% inode=68%): [22:31:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 32414 MB (5% inode=99%): [22:43:36] mm openstreetmap (edit= seems down :( [22:44:03] :-( [22:44:15] DaBPunkt: Did you see https://toolserver.org/~brion/extensions/ was still returning 403: expired? [22:44:31] Susan: that will fixed at midnight [22:44:36] yay [22:44:38] or is it urgend? [22:45:03] Sounds like midnight is fine. I thought there was a delay. I didn't realize it was daily. [22:48:57] wikidata replag on z-dat-s2-b is OK: QUERY OK: SELECT ts_rc_age() returned 1701.000000 [23:22:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:23:57] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:23:58] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [23:25:07] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:25:07] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:25:37] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:28:07] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:29:08] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:29:57] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 326713 MB (6% inode=68%): [23:31:08] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 32163 MB (5% inode=99%): [23:35:23] brion: Have you solved the SSH key business? [23:39:48] scfc_de: haven't touched it. if my files work, i have no need to log in again [23:39:51] until the next expiry :P [23:41:35] The latter could be solved much easier by you executing "acctrenew" than again waiting for a root :-). [23:46:17] MySQL slave on z-dat-s6-a is CRITICAL: (Return code of 139 is out of bounds) [23:50:23] hehe