[00:06:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:11:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:20:41] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:22:41] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [00:24:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:36:51] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24401 MB (5% inode=99%): [00:37:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:37:51] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 25175 MB (6% inode=99%): [00:39:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:52:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [00:52:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42927 MB (7% inode=99%): [01:06:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:11:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:20:41] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:22:42] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [01:25:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:37:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:39:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:52:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [01:52:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41280 MB (6% inode=99%): [02:06:42] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:11:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:20:41] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:22:42] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [02:25:42] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:38:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:39:42] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:52:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42780 MB (6% inode=99%): [02:53:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [03:07:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:12:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:20:41] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:22:41] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [03:26:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:32:51] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24433 MB (5% inode=99%): [03:35:52] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 24955 MB (6% inode=99%): [03:38:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:40:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:52:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42755 MB (6% inode=99%): [03:53:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [04:07:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:12:42] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:20:40] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:23:41] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [04:26:42] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:38:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:40:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:52:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42702 MB (6% inode=99%): [04:54:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [05:07:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:13:42] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:20:41] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:23:40] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [05:27:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:38:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:40:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:53:01] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 5123 MB (0% inode=92%): [05:53:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42625 MB (6% inode=99%): [05:55:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [06:07:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:13:42] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:20:41] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:23:41] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [06:25:41] APT on yarrow is CRITICAL: APT CRITICAL: 13 packages available for upgrade (9 critical updates). [06:27:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:31:41] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [06:31:41] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [06:38:01] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 44839 MB (11% inode=99%): [06:38:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:40:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:44:51] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24470 MB (5% inode=99%): [06:49:51] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 24502 MB (6% inode=99%): [06:53:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42557 MB (6% inode=99%): [06:55:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [07:07:42] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:13:42] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:20:42] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:25:41] APT on yarrow is CRITICAL: APT CRITICAL: 13 packages available for upgrade (9 critical updates). [07:27:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:31:41] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [07:31:41] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [07:32:51] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1994.000000 [07:36:51] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1705.000000 [07:38:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:39:52] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24471 MB (5% inode=99%): [07:40:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:53:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42455 MB (6% inode=99%): [07:55:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [08:07:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:12:01] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 116987 MB (12% inode=99%): [08:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:21:40] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:25:42] APT on yarrow is CRITICAL: APT CRITICAL: 13 packages available for upgrade (9 critical updates). [08:27:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:31:41] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [08:31:41] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [08:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:40:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:46:51] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 25477 MB (6% inode=99%): [08:47:50] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24389 MB (5% inode=99%): [08:53:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42386 MB (6% inode=99%): [08:55:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [09:08:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:21:41] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:26:42] APT on yarrow is CRITICAL: APT CRITICAL: 13 packages available for upgrade (9 critical updates). [09:28:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:31:41] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [09:31:41] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [09:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:40:01] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 44716 MB (10% inode=99%): [09:40:42] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:49:01] jira.toolserver.org on web.amaranth is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Temporarily Unavailable - 170 bytes in 0.472 second response time [09:51:01] jira.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 518 bytes in 0.536 second response time [09:53:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42268 MB (6% inode=99%): [09:54:01] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 26784 MB (6% inode=99%): [09:55:01] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24298 MB (5% inode=99%): [09:55:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [10:09:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:21:50] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:26:51] APT on yarrow is CRITICAL: APT CRITICAL: 13 packages available for upgrade (9 critical updates). [10:28:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:31:51] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [10:31:51] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [10:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:53:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42118 MB (6% inode=99%): [10:56:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [10:58:12] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 87739 MB (9% inode=98%): [10:58:12] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 87739 MB (9% inode=98%): [11:09:02] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:26:50] APT on yarrow is CRITICAL: APT CRITICAL: 13 packages available for upgrade (9 critical updates). [11:28:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:31:50] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [11:31:50] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [11:38:12] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 44744 MB (11% inode=99%): [11:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:45:12] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 44588 MB (10% inode=99%): [11:54:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 42030 MB (6% inode=99%): [11:56:11] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [12:10:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:17:00] hello all [12:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:26:51] APT on yarrow is CRITICAL: APT CRITICAL: 13 packages available for upgrade (9 critical updates). [12:29:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:31:51] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [12:31:51] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [12:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:54:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41900 MB (6% inode=99%): [12:56:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [12:59:02] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 25041 MB (6% inode=99%): [13:10:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:18:51] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:21:50] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:29:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:31:51] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [13:31:51] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [13:39:05] Hello, Any administrator at orts-wiki, please? [13:39:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:42:34] Avocato: wrong channel. try #wikimedia-tech [13:55:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41792 MB (6% inode=99%): [13:57:11] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [13:57:28] DaBPunkt: nosy was looking for something earlier? [14:11:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:14:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:21:50] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:29:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:31:51] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [14:31:51] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [14:39:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:55:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41657 MB (6% inode=99%): [14:55:51] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [14:56:39] soon, my little apt, soon [14:57:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [15:02:06] http://www.quickmeme.com/meme/3qw3hh/ ;-) [15:12:02] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:14:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:21:50] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:28:50] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:29:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:31:51] APT on nightshade is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [15:31:51] APT on mayapple is CRITICAL: APT CRITICAL: 9 packages available for upgrade (9 critical updates). [15:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:55:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41621 MB (6% inode=99%): [15:58:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [15:59:51] APT on mayapple is OK: APT OK: 0 packages available for upgrade (0 critical updates). [16:01:51] APT on nightshade is OK: APT OK: 0 packages available for upgrade (0 critical updates). [16:12:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:14:07] DaB. * [Toolserver-l] Python on linux now working (was:Re: Please move more longrunning-tasks to linux) [16:14:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:20:01] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24391 MB (5% inode=99%): [16:21:50] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:29:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:32:06] Dr. Trigon * Re: [Toolserver-l] tool to explore MediaWiki API [16:39:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:55:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41529 MB (6% inode=99%): [16:58:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [17:08:01] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 24575 MB (6% inode=99%): [17:09:02] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1866.000000 [17:12:02] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:25:01] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1715.000000 [17:30:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:39:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:53:13] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 21244 MB (1% inode=97%): [17:55:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41400 MB (6% inode=99%): [17:58:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [18:01:02] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24472 MB (5% inode=99%): [18:13:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:14:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:21:50] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:31:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:55:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41284 MB (6% inode=99%): [18:58:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [19:13:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:14:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:31:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:41:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:48:37] DaBPunkt: Hello! May I ask you a question? [19:48:45] sure [19:49:22] Some of my bots did not run yesterday and today... I was talking to Merlissimo and he suggested you could have a look into the cron log? [19:49:47] from where? [19:49:59] submit (clematis?) [19:50:02] ok. [19:50:34] I thinks it's the line: [19:50:39] 0 0 * * * qcronsub -l h_rt=06:00:00 -l virtual_free=200M -N subster_meta $HOME/pywikipedia/bot_control.py -subster -cron -family:meta -lang: [19:50:55] ok, midnight [19:51:36] strange... it does not show up in SGE... [19:54:37] I need a moment. I have problems to find the right log-file [19:55:15] (... no problem - thanks btw? ;) [19:56:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41222 MB (6% inode=99%): [19:58:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [19:59:11] /sql on z-dat-s3-a is CRITICAL: DISK CRITICAL - free space: /sql 58348 MB (5% inode=97%): [19:59:11] /sql on z-dat-s6-a is CRITICAL: DISK CRITICAL - free space: /sql 58348 MB (5% inode=97%): [20:02:59] @replag [20:03:03] Merlissimo: s2-user: 2h 24m 5s [+0.01 s/s]; s3-rr-a: 26s [-0.00 s/s]; s3-user: 27s [-0.00 s/s] [20:05:20] DaBPunkt: (still searching...?) [20:05:29] Sep 13 00:00:02 hawthorn /OPT/TS/SBIN/CROND[5184]: [ID 529850 cron.error] (CRON) CAN'T FORK (child_process): Not enough space [20:07:35] out of memory [20:07:35] ...I did not get any e-mail about this... ?!? [20:07:37] puh, not sge ;-) [20:07:52] out of memory again...?? [20:08:23] I WONDER WHY VIXIE CRON PUTS ALL ITS MESSAGES IN UPPER CASE? [20:08:27] The strange thing is that hawthorn has 500 freee RAM + 1.42GB cache [20:08:28] why was that not re-started/-tried...? [20:08:59] and this should be small job... [20:09:27] DrTrigon: it died before sge could get the job [20:10:06] ...so how to solve that kind of issues? btw.: what whas the issue? ;) [20:10:08] My advice would be to choose another start-time than midnight [20:10:21] like 0 23 [20:10:49] like Merlissimo suggested not at 00:00 but may be at 00:07 ? [20:10:56] yes [20:12:01] ok, I will do this - BUT what if others do the same and we have the same situation again? Write a script the randomly changes the times? what is the "correct" solution? [20:12:11] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 116959 MB (12% inode=99%): [20:12:22] 40 tasks were successfull executed – I wonder how many failed [20:13:31] the point is; it worked for years - then it stopped some weeks (or months ago) - then memory was bought and it worked again - an now it stopped again... ?!? [20:14:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:14:08] how can I (we) solve that issue permanently? [20:14:27] DrTrigon: do not add new users or new tools [20:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:15:02] ;) ok that was a good one... ;) but seriously? [20:15:12] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 91010 MB (9% inode=98%): [20:15:12] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 91010 MB (9% inode=98%): [20:15:16] of course this is no prefered solution, so the only other way is to grow [20:15:57] but in this sepcial case: I will write an email to the ML asking the users to spread there tasks more [20:16:00] so... does this mean new memory is up to be bought and installed? [20:17:03] yeah, but then you have no control or whatsoever at what times the will schedule, and then say at 00:05 will be the next bunch which causes problems... [20:17:40] is ther no mechanism, e.g. in cron to avoid this (e.g. by delaying some jobs like anacron) [20:18:03] DrTrigon: I can not promise you anything. WMDE and I are not on good therms in questions of spending money [20:18:20] I still wait for the build-host [20:18:27] ;) [20:19:48] what about cron should have handeled that correctly (e.g. by delaying) anyway?? [20:20:28] DrTrigon: If you can write a patch I'm sure the cron-developers wil be happy ;) [20:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:21:57] ...what about the email that would have been sent usually?? [20:22:53] DrTrigon: if there is not enough space, nothing will happen. No run, no feedback, no mail [20:24:22] so I have to write my script in a manner that allow to start it e.g. 5 times a day but it runs just once - for example... I see - now I understand...! ;)) [20:24:37] DrTrigon: You don't need to :) [20:24:45] SGE does that for you [20:25:27] not I cases where my script stops after running and runs just a few seconds...? or am I wrong??? [20:25:42] there is no problem to run your cron-task every hour (or so). If the job is already running, SGE will do nothing. [20:25:51] DrTrigon: yes [20:26:01] if it's only run for short time [20:27:13] ?!? now I am confused... ;) [20:27:57] DrTrigon: If the task runs only for short time, you have to take care yourself, yes. If it is a long-running task, SGE handles it for you [20:29:09] it's a pitty... ;)) but thanks for the suggestion! [20:29:30] so I will set my jobs to 07 and look how that works! [20:29:58] may I ask you another question? related to TS-1449? [20:31:27] yes, but I think you will not like my answer [20:31:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:33:24] ;) I see - is there at least a more recent version in sight somewhen in future? or will there never ever be one? (so I have to look for an alternative?) [20:38:36] DaBPunkt: ^^^? [20:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:39:48] DrTrigon: pdfimages package version in testing is 0.18.4. That will become live somewhen in spring. ffprobe has no testing-version at the moment, there I need investigation. [20:42:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:49:04] aaaa, that's bad... ffprobe would be more intressting at the moment... are we using th one from debian, ubuntu or are they the same? anyway THANKS A LOT for your help!!!!! :) [20:49:15] DaBPunkt: ^^^ [20:49:32] DrTrigon: debian [20:49:51] I can take a look tomorrow of you like [20:51:07] DaBPunkt: if you are asking that way: "yes I like!" ;) but do not give up all of your other life just for that ... :) THANKS AGAIN!!! [20:53:04] DaBPunkt: meanwhile I will look for an alternative (I doubt there is one) and may be how to compile it ourselves... [20:54:45] DaBPunkt: I forgot to mention this one: http://stackoverflow.com/questions/11229660/ffprobe-with-print-json-doesnt-print-anything (may be you can use that?) [20:56:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41136 MB (6% inode=99%): [20:58:12] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [21:01:04] DaB. * [Toolserver-announce] When to execute cron-tasks [21:02:17] DrTrigon: the main problem with these user-generated repositories is the same if I do compile it myself: Unfixed security bugs. Most people have not the time to read every security-notice. [21:03:43] I see... and third-party is even worse since you don't know what security-notices the author has read... ;)) [21:04:18] (so we are back to the build host again... I see the dilemma know... ;) [21:14:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:14:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:19:34] DrTrigon: it looks like the ffprobe-programm moved from its own package to the libav-tools-package [21:21:12] sounds promising...? [21:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:22:04] André Karwath * Re: [Toolserver-l] When to execute cron-tasks [21:22:23] DaBPunkt: ...sounds promising...? so just install "libav-tools-package"? [21:22:37] DrTrigon: I try [21:30:27] DaBPunkt: ? *hope* ? [21:31:07] DrTrigon: I installed it, but it looks not like there is a json-output. You can try yourself at nightshade if you like [21:31:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:35:09] DaBPunkt: au-contraire... it look's very promising, but I need a few minutes to finally test it... stay tuned! [21:39:05] it is a little bit strange: the homepage says that 0.8.3 is the last 0.8 version (fom june). I installed 0.8.3 – but avprobe does not understand -of AFAIS [21:39:13] but I let the testing to you, DrTrigon [21:39:40] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:42:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:44:04] DaB. * Re: [Toolserver-l] When to execute cron-tasks [21:49:01] DaBPunkt: yes it is somehow strange indeed! 'json' is not working you are right but at least it gives more output than the previous one - and now I can use it at least (even though I have to adopt my script... ;))) THANKS A LOT! [21:49:21] and may be there will be an update in the package soon... :) [21:49:22] no problem [21:49:58] thanks for all the fish!! ;) [21:50:08] so I have to go to bed soon... ;)) good night and good look [21:56:13] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 41066 MB (6% inode=99%): [21:58:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [22:00:55] I will leave for bed too [22:01:18] nacht ts [22:04:51] APT on mayapple is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [22:06:51] APT on nightshade is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [22:08:51] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [22:14:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:14:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:31:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:39:41] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:43:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:56:13] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 40942 MB (6% inode=99%): [22:58:13] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [23:04:51] APT on mayapple is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [23:06:51] APT on nightshade is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [23:08:51] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [23:14:02] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:14:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:21:51] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:31:51] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:36:04] Dr. Trigon * Re: [Toolserver-l] Python on linux now working [23:39:52] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:43:01] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:43:07] Dr. Trigon * Re: [Toolserver-l] When to execute cron-tasks [23:45:12] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 43715 MB (10% inode=99%): [23:57:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 40905 MB (6% inode=99%): [23:59:11] SRaid on nightshade is CRITICAL: NRPE: Unable to read output