[10:55:35] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:55:44] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:55:44] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:55:54] /tmp on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:55:54] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:55:54] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:56:03] /sql on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:56:03] /tmp on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:56:23] /tmp on z-dat-s4-a is OK: DISK OK - free space: /tmp 3539 MB (99% inode=99%): [10:56:34] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [10:56:34] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 131088 MB (32% inode=99%): [10:56:35] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [10:56:35] /tmp on z-dat-s7-a is OK: DISK OK - free space: /tmp 3537 MB (99% inode=99%): [10:56:45] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [10:56:45] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [11:07:43] SSH on nightshade.mgmt is CRITICAL: Server answer: [11:08:03] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:09:04] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:09:23] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 62376 MB (6% inode=99%): [11:10:44] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 213381 MB (4% inode=27%): [12:03:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [12:07:55] SSH on nightshade.mgmt is CRITICAL: Server answer: [12:08:04] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:09:04] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:09:25] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 62294 MB (6% inode=99%): [12:10:43] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 213010 MB (4% inode=27%): [13:03:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [13:08:05] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:08:53] SSH on nightshade.mgmt is CRITICAL: Server answer: [13:09:04] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:09:24] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 62213 MB (6% inode=99%): [13:10:43] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 212532 MB (3% inode=27%): [14:08:54] SSH on nightshade.mgmt is CRITICAL: Server answer: [14:09:04] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:09:04] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:09:24] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 62115 MB (6% inode=99%): [14:10:43] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 212133 MB (3% inode=27%): [14:33:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [15:09:04] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:09:05] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:09:23] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 62012 MB (6% inode=99%): [15:09:56] SSH on nightshade.mgmt is CRITICAL: Server answer: [15:10:44] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 211691 MB (3% inode=27%): [15:23:04] Sun Grid Engine execd on ortelius is WARNING: all.q@ortelius exceedes load threshold: alarm hl:np_load_avg=1.242188/1.00, alarm hl:mem_free=12130.000000M/100M [15:30:04] Sun Grid Engine execd on ortelius is OK: all.q@ortelius OK [15:33:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [15:55:19] DaBPunkt: I had seen that you created a malloc.o under /opts/ts/lib. Great, thank you! [15:55:53] apmon: yes, I sent you a irc-line. Please test it if I have done everything in a correct way [15:56:05] Hoever, if I saw it correctly, it there is only a 32bit version. Is it possible to have it in 64bit as well? [15:56:17] Ah, I missed the irc-line. [15:56:45] I'll move tirex (which is 32 bit) over to /opt/ts/lib then, to test [15:57:05] why that? [15:57:33] why 64 bit? Postgresql is compiled as 64 bit [15:57:40] no, why the moving [15:58:51] sorry, was imprecise. I'll change tirex from using LD_PRELOAD=~osm/tools/dlmalloc/32/dlmalloc.so to LD_PRELOAD=/opt/ts/lib/malloc.o [16:01:37] ah ok [16:01:55] I will look for x64-compilation this evening [16:02:19] wonderful, thank you. [16:08:56] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 127239 MB (20% inode=99%): [16:09:14] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:09:14] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:09:24] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 61912 MB (6% inode=99%): [16:09:55] SSH on nightshade.mgmt is CRITICAL: Server answer: [16:10:55] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 211375 MB (3% inode=27%): [16:33:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [16:47:33] Load avg. on daphne is WARNING: WARNING - load average: 16.10, 13.12, 8.89 [16:51:34] Load avg. on daphne is OK: OK - load average: 14.90, 14.33, 10.39 [17:07:33] Load avg. on daphne is WARNING: WARNING - load average: 15.12, 14.76, 13.18 [17:09:13] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:09:14] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:09:56] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 127182 MB (20% inode=99%): [17:09:56] SSH on nightshade.mgmt is CRITICAL: Server answer: [17:10:24] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 61799 MB (6% inode=99%): [17:10:55] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 211007 MB (3% inode=27%): [17:29:56] Sun Grid Engine execd on nightshade is WARNING: all.q@nightshade exceedes load threshold: alarm hl:np_load_avg=1.533691/1.50, alarm hl:mem_free=755.000000M/100M [17:32:56] Sun Grid Engine execd on nightshade is OK: all.q@nightshade OK: longrun@nightshade OK [18:01:53] is willow ok? [18:03:56] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [18:04:07] Alchimista: I'm on it right now [18:04:26] TParis: i'm too, but i can't download anything with ftp [18:04:47] Ohh, well then I don't know. I'm using SVN. [18:04:56] hi [18:08:41] Gnumarcoo: hi [18:09:14] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:09:14] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:09:55] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 127130 MB (20% inode=99%): [18:09:55] SSH on nightshade.mgmt is CRITICAL: Server answer: [18:11:23] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 61683 MB (6% inode=99%): [18:11:55] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 210530 MB (3% inode=27%): [18:23:10] Alchimista: :) [18:29:55] Sun Grid Engine execd on nightshade is WARNING: longrun@nightshade exceedes load threshold: alarm hl:np_load_long=0.546387/2.00, alarm hl:mem_free=228.000000M/250M [18:31:04] Sun Grid Engine execd on nightshade is OK: all.q@nightshade OK: longrun@nightshade OK [18:37:33] Load avg. on hyacinth is WARNING: WARNING - load average: 18.80, 15.52, 12.24 [18:53:34] Load avg. on hyacinth is OK: OK - load average: 12.46, 14.59, 14.83 [18:57:03] Sun Grid Engine execd on nightshade is WARNING: longrun@nightshade exceedes load threshold: alarm hl:np_load_long=0.605957/2.00, alarm hl:mem_free=195.000000M/250M [18:58:04] Sun Grid Engine execd on nightshade is OK: all.q@nightshade OK: longrun@nightshade OK [19:09:14] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:09:15] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:10:54] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 127063 MB (20% inode=99%): [19:10:55] SSH on nightshade.mgmt is CRITICAL: Server answer: [19:11:24] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 61574 MB (6% inode=99%): [19:12:55] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 209479 MB (3% inode=27%): [19:22:55] /aux0 on daphne is CRITICAL: DISK CRITICAL - free space: /aux0 65665 MB (6% inode=99%): [19:24:23] 3(commented) [MNT-1169] Delete empty osm-dirs <10https://jira.toolserver.org/browse/MNT-1169> (DaB.) [19:24:23] 3(resolved) [MNT-1168] Delete old osm-titles. <10https://jira.toolserver.org/browse/MNT-1168> (DaB.) [19:33:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [19:44:20] apmon: ping [19:47:34] Load avg. on hyacinth is WARNING: WARNING - load average: 16.83, 14.91, 13.29 [19:54:06] toolserver: 03dab * r1124 10/trunk/ts-specs/ (3 files in 2 dirs): [19:54:06] toolserver: -Configured dlmalloc as lib-package. [19:54:06] toolserver: -Create a base-file for a 32- and a 64-bit-compilation. [19:59:34] Load avg. on hyacinth is OK: OK - load average: 12.46, 14.62, 14.36 [20:04:35] Load avg. on hyacinth is WARNING: WARNING - load average: 15.95, 15.34, 14.73 [20:09:25] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:09:25] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:11:03] SSH on nightshade.mgmt is CRITICAL: Server answer: [20:11:34] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 61473 MB (6% inode=99%): [20:11:54] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 127012 MB (20% inode=99%): [20:13:04] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 209038 MB (3% inode=27%): [20:20:34] Load avg. on hyacinth is OK: OK - load average: 12.01, 13.83, 14.87 [20:33:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [20:36:21] 3(created) [DRTRIGON-107] DrTrigonBot without Botflag (on some pages); DrTrigon: DrTrigonBot - pywikipedia; New Feature <10https://jira.toolserver.org/browse/DRTRIGON-107> (drtrigon) [20:47:17] DaBPunkt: Pong [20:47:53] apmon: I installed the 64-bit-version and renamed the 32-bit-version. do you need a symlink to the 32-bit-version? [20:48:38] I'll just need to change the tirex startup script where I had just updated it [20:49:02] ok, that I will let the symlink to for some days [20:49:08] +be [20:50:07] Currently there is malloc.32.o malloc.64.o and malloc.o? [20:50:32] malloc.o is a symlink that points to malloc.32.o [20:51:20] Ah, you already have the symlink and are asking if you can remove it? [20:51:27] yes [20:53:30] DaBPunkt: I have restarted tirex-backend with malloc.32.o , so there shouldn't be anything using the previous name anymore [20:53:48] ok, then I will remove the symlink [20:54:06] OK [20:54:36] (It didn't matter as long as the programm is running, but it matters if the programm is restarted and looks for the lib) [21:10:24] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:10:24] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:11:04] SSH on nightshade.mgmt is CRITICAL: Server answer: [21:11:35] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 61357 MB (6% inode=99%): [21:12:05] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 126946 MB (20% inode=99%): [21:13:05] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 208646 MB (3% inode=26%): [21:33:54] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [22:10:35] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:10:35] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:11:15] SSH on nightshade.mgmt is CRITICAL: Server answer: [22:11:34] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 62178 MB (6% inode=99%): [22:13:04] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 126890 MB (20% inode=99%): [22:14:04] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 208180 MB (3% inode=26%): [23:04:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [23:11:24] SSH on nightshade.mgmt is CRITICAL: Server answer: [23:11:34] SMF on damiana.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:11:35] SMF on turnera.esi is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:11:45] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 62073 MB (6% inode=99%): [23:13:14] /sql on ptolemy is WARNING: DISK WARNING - free space: /.alt.s10b/sql 126852 MB (20% inode=99%): [23:14:14] /aux0 on hemlock is CRITICAL: DISK CRITICAL - free space: /aux0 207209 MB (3% inode=26%): [23:46:14] fisheye.toolserver.org on web.amaranth is WARNING: HTTP WARNING: HTTP/1.1 200 OK - 273 bytes in 16.871 second response time [23:46:46] nacht ts [23:48:02] fisheye.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 272 bytes in 10.686 second response time