[00:26:23] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16283 [00:27:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 16263.000000 [00:32:43] Free Memory on damiana is WARNING: WARNING - 6.5% (545180 kB) free! [00:35:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:38:44] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:40:43] Free Memory on damiana is OK: OK - 7.2% (601392 kB) free. [00:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:52:43] Free Memory on damiana is CRITICAL: CRITICAL - 4.8% (399148 kB) free! [00:54:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 39531 MB (6% inode=99%): [00:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [00:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:26:24] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 14811 [01:27:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14751.000000 [01:35:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:38:42] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:54:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37755 MB (6% inode=99%): [01:57:43] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (393160 kB) free! [01:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [01:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:00:42] Free Memory on damiana is WARNING: WARNING - 5.1% (425728 kB) free! [02:01:43] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (410928 kB) free! [02:21:43] Free Memory on damiana is WARNING: WARNING - 6.6% (552344 kB) free! [02:25:43] Load avg. on willow is WARNING: WARNING - load average: 19.88, 16.74, 14.37 [02:26:23] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 10737 [02:27:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10662.000000 [02:35:43] Load avg. on willow is OK: OK - load average: 13.12, 14.98, 14.80 [02:35:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:38:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:40:43] Free Memory on damiana is OK: OK - 7.2% (605516 kB) free. [02:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:54:44] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 39112 MB (6% inode=99%): [02:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [02:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:06:43] Load avg. on willow is WARNING: WARNING - load average: 15.70, 16.39, 15.51 [03:24:43] Load avg. on willow is OK: OK - load average: 13.17, 14.16, 14.93 [03:26:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6136 [03:27:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6081.000000 [03:31:43] Load avg. on willow is WARNING: WARNING - load average: 22.87, 19.53, 16.95 [03:36:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:38:43] Load avg. on willow is OK: OK - load average: 9.95, 13.34, 14.95 [03:38:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:43:43] Free Memory on damiana is CRITICAL: CRITICAL - 3.2% (270560 kB) free! [03:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:45:43] Load avg. on adenia is WARNING: WARNING - load average: 16.37, 15.74, 13.92 [03:51:42] Free Memory on damiana is WARNING: WARNING - 5.5% (460712 kB) free! [03:52:43] Load avg. on adenia is OK: OK - load average: 13.59, 14.84, 14.23 [03:54:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 38945 MB (6% inode=99%): [03:58:42] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [03:59:02] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3569.000000 [03:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:59:23] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3515 [03:59:43] Load avg. on adenia is WARNING: WARNING - load average: 16.18, 15.82, 14.96 [04:01:43] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (395228 kB) free! [04:06:44] Load avg. on willow is WARNING: WARNING - load average: 14.38, 15.61, 14.62 [04:11:42] Free Memory on damiana is WARNING: WARNING - 5.2% (437096 kB) free! [04:12:43] Load avg. on adenia is OK: OK - load average: 8.42, 13.37, 14.79 [04:14:43] Free Memory on damiana is OK: OK - 8.4% (706964 kB) free. [04:17:43] Load avg. on willow is OK: OK - load average: 11.63, 14.43, 14.82 [04:25:45] What is wrong with the path with everything from sge? [04:25:59] php, svn, qstat, qdel nothing is found [04:26:08] do I need to prefix /usr/bin/env for everything everywhere? [04:26:20] even inside php [04:27:23] MySQL slave on rosemary is OK: Uptime: 12489002 Threads: 25 Questions: 5138697513 Slow queries: 1459516 Opens: 135912 Flush tables: 6 Open tables: 3735 Queries per second avg: 411.457 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1734 [04:28:03] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1673.000000 [04:28:20] It works fine from submit host (clematis) ssh, but when qcronsub does it, it is broken [04:34:22] /usr/bin/env qstat -f -r [04:34:23] /usr/bin/env: qstat: No such file or directory [04:34:29] even with env it doesn't work [04:35:04] (that's pasted from the .o file) [04:36:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:54:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 38779 MB (6% inode=99%): [04:57:23] Hm.. dumping env from the job shows me this [04:57:25] PATH=/var/sge/tmp/335411.1.longrun-lx:/usr/local/bin:/bin:/usr/bin [04:57:34] compared to [04:57:37] PATH=/sge/GE/bin/sol-amd64:/sge/GE/bin/sol-amd64:/opt/local/bin:/opt/ts/gnu/bin:/opt/ts/bin:/opt/ts/sbin:/opt/ts/mysql/5.1/bin:/opt/ts/perl/5.12/bin:/opt/ts/perl/5/bin:/opt/ts/python/2.7/bin:/opt/ts/php/5.3/bin:/opt/ts/ruby/1.8/bin:/opt/ts/tcl/8.5/bin:/usr/ccs/bin:/sge/GE/bin/sol-amd64:/usr/bin:/usr/sbin:/usr/sfw/bin:/opt/ts/postgres/9.0/bin:/usr/cluster/bin:/opt/jobserver/bin [04:57:58] I thought the -b y option would make it run in place, not just "not copying the file over" but really in place [04:58:13] There is a SGE_O_PATH variable as well [04:58:14] Interesting [04:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [04:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:00:12] This is messed up, /usr/bin/env doesn't help at all because the environment is different all together. It fails the same wether I include it or not, that's good [05:00:43] but it still don't work. Should I start every time it shells out with "export PATH=$SGE_O_PATH:$PATH; real command here" that seems silly [05:01:13] shelling out from python or php (or even bash) should just work. I'll file in JIRA, I hope I'm doing it wrong. [05:02:43] Free Memory on damiana is WARNING: WARNING - 6.0% (503928 kB) free! [05:05:43] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (392876 kB) free! [05:26:44] Free Memory on damiana is WARNING: WARNING - 5.9% (490460 kB) free! [05:30:43] Free Memory on damiana is CRITICAL: CRITICAL - 3.6% (302716 kB) free! [05:36:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:37:43] Free Memory on damiana is OK: OK - 7.6% (637244 kB) free. [05:37:52] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:38:13] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:38:13] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:38:43] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 30708 MB (7% inode=99%): [05:38:43] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 89617 MB (9% inode=98%): [05:38:43] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 89617 MB (9% inode=98%): [05:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:48:44] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 10983 MB (0% inode=94%): [05:51:13] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:51:13] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:51:53] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:54:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 38616 MB (6% inode=99%): [05:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [05:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:03:43] Free Memory on damiana is CRITICAL: CRITICAL - 2.5% (205740 kB) free! [06:10:45] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 90066 MB (9% inode=98%): [06:10:45] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 90066 MB (9% inode=98%): [06:11:23] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 30679 MB (7% inode=99%): [06:36:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:55:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 38401 MB (6% inode=99%): [06:58:44] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [06:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:03:43] Free Memory on damiana is CRITICAL: CRITICAL - 3.4% (282308 kB) free! [07:35:43] Free Memory on damiana is WARNING: WARNING - 6.6% (550912 kB) free! [07:36:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:38:43] Free Memory on damiana is OK: OK - 7.2% (603848 kB) free. [07:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:42:07] @replag [07:42:07] liangent: s2-user: 24s [-0.00 s/s] [07:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:55:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 38141 MB (6% inode=99%): [07:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [07:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:08:43] Free Memory on damiana is WARNING: WARNING - 6.4% (537588 kB) free! [08:14:43] Free Memory on damiana is OK: OK - 7.3% (607816 kB) free. [08:26:43] Free Memory on damiana is WARNING: WARNING - 6.7% (565288 kB) free! [08:37:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:55:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37951 MB (6% inode=99%): [08:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [08:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:37:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:47:43] Free Memory on damiana is WARNING: WARNING - 6.6% (551804 kB) free! [09:55:43] Free Memory on damiana is OK: OK - 7.3% (611100 kB) free. [09:55:44] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37718 MB (6% inode=99%): [09:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [09:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:14:42] Free Memory on damiana is CRITICAL: CRITICAL - 4.3% (360240 kB) free! [10:15:23] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24199 MB (5% inode=99%): [10:24:45] nosy: do you know if it's possible to raise max_user_connections for a user? [10:25:25] it's currently too low for the traffic WLM is getting, it seems http://lists.wikimedia.org/pipermail/wikilovesmonuments/2012-September/004227.html [10:25:28] Nemo_bis: yes it is but you need to create a ticket and put a reason in, too [10:26:11] might be reason enough [10:27:02] nosy: can be assigned to you? [10:27:20] yes [10:29:43] Free Memory on damiana is WARNING: WARNING - 5.2% (436304 kB) free! [10:30:40] nosy: https://jira.toolserver.org/browse/TS-1508 [10:30:51] can't find you in assign field [10:31:03] wow...marlen caemmerer [10:31:24] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1930 [10:32:03] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1940.000000 [10:32:34] nosy: it must hate me, still can't add you [10:33:14] its mine now [10:33:51] thanks [10:34:33] Nemo: i will check with dab this evening [10:35:03] nosy: ok thanks [10:37:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:40:42] Free Memory on damiana is OK: OK - 7.1% (596164 kB) free. [10:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:52:14] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:52:14] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:52:52] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 89160 MB (9% inode=98%): [10:52:53] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 89160 MB (9% inode=98%): [10:53:22] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 31507 MB (7% inode=99%): [10:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37442 MB (6% inode=99%): [10:57:43] Free Memory on damiana is CRITICAL: CRITICAL - 3.7% (311652 kB) free! [10:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [10:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:04:13] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:23:45] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 88839 MB (9% inode=98%): [11:26:44] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 122373 MB (12% inode=99%): [11:31:23] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3201 [11:32:02] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3133.000000 [11:36:43] Free Memory on damiana is WARNING: WARNING - 5.2% (439544 kB) free! [11:37:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:42:42] Free Memory on damiana is CRITICAL: CRITICAL - 4.5% (375352 kB) free! [11:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:45:23] MySQL slave on rosemary is OK: Uptime: 12515282 Threads: 27 Questions: 5150472838 Slow queries: 1464040 Opens: 137000 Flush tables: 6 Open tables: 3722 Queries per second avg: 411.534 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1787 [11:46:03] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1757.000000 [11:56:44] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 37122 MB (6% inode=99%): [11:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [11:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:04:23] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:04:52] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:05:12] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:06:13] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:06:23] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 30038 MB (7% inode=99%): [12:06:43] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 88492 MB (9% inode=98%): [12:06:43] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 88492 MB (9% inode=98%): [12:19:13] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [12:20:53] /sql on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:21:12] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:21:12] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:23:24] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 30217 MB (7% inode=99%): [12:37:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:40:43] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 88624 MB (9% inode=98%): [12:40:43] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 88624 MB (9% inode=98%): [12:41:42] Free Memory on damiana is OK: OK - 8.3% (696920 kB) free. [12:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:56:42] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36834 MB (6% inode=99%): [12:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [12:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:06:07] DaB. * [Toolserver-l] How you can help: puppet [13:06:45] hello all [13:06:51] hi [13:09:06] Did you know categorylinks is missing from nowiki? [13:12:06] no, is there a bug-report? [13:12:19] It's rather particular I think. Oh well, got more important wikis to worry about. [13:15:55] peculiar* [13:37:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:45:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36560 MB (5% inode=99%): [13:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [13:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:20:24] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 21855 MB (5% inode=99%): [14:37:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:48:33] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 36289 MB (8% inode=99%): [14:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36304 MB (5% inode=99%): [14:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [14:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:37:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 36035 MB (5% inode=99%): [15:58:42] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [15:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:33:03] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2059.000000 [16:37:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:40:03] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1712.000000 [16:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:56:44] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 35753 MB (5% inode=99%): [16:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [16:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:37:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:48:43] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 15993 MB (1% inode=96%): [17:53:12] /sql on z-dat-s3-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:53:12] /sql on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:53:42] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 88484 MB (9% inode=98%): [17:53:43] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 88484 MB (9% inode=98%): [17:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 35412 MB (5% inode=99%): [17:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [17:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:37:44] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 35114 MB (5% inode=99%): [18:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [18:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:17:40] @replag [19:17:41] multichill: s3-rr-a: 23s [-0.00 s/s]; s3-user: 23s [-0.00 s/s] [19:19:06] @replag [19:19:07] multichill: s3-rr-a: 22s [-0.01 s/s]; s3-user: 22s [-0.01 s/s] [19:19:12] @replag s1 [19:19:13] multichill: No replag found. Use "@replag all" to see all lags. [19:19:16] @replag all [19:19:16] multichill: s1-rr-a: 3s [-0.03 s/s]; s1-user: 3s [-0.03 s/s]; s2-user: 4s [-0.00 s/s]; s2-user-c: 1s [-0.00 s/s]; s3-rr-a: 4s [-1.83 s/s]; s3-user: 4s [-1.83 s/s]; s4-rr-a: 1s [-]; s4-user: 1s [-0.00 s/s] [19:19:17] multichill: s5-rr-a: 1s [-0.00 s/s]; s5-user: 1s [-0.00 s/s]; s5-user-c: 1s [-0.00 s/s]; s6-rr-a: 1s [-0.05 s/s]; s6-user: 1s [-0.05 s/s]; s7-rr-a: 2s [-0.00 s/s]; s7-user: 2s [-0.00 s/s] [19:19:26] yah! [19:22:06] DaBPunkt: ping [19:23:08] passwd reset don't work, it says i don't exist. i'm quite certain (based on emails i've reviewed from months ago) that i have the username right. unless i changed my name and forgot that i did so [19:23:32] err, and i'm talking about JIRA btw ;) [19:24:14] * jeremyb pokes nosy [19:24:41] someone on the internet is wrong and his name is JIRA! [19:27:25] jeremyb: i feel poked and have a look [19:27:57] username=jeremyb btw [19:29:56] you are not a user it seems [19:30:01] but i still dont know why [19:32:19] nosy: look at the reporter of https://jira.toolserver.org/browse/ACC-29 [19:32:34] hard to believe it's been 4 years! [19:32:56] jeremyb: oh thx but you dont exist at all afais [19:33:26] nosy: errr, ??? does JIRA actually allow acct deletion? [19:33:52] it can't be more than 2 months since i last logged in [19:37:43] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:39:25] jeremyb: there is a problem with jira's accounts at the moment [19:39:34] it needs investigation [19:39:42] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:40:02] DaBPunkt: do i make a new one? is there a JIRA ticket for it? [19:40:15] obviously i can't "watch" the ticket if i can't login... ;/ [19:41:53] how many people are affected? [19:43:41] that's unknown [19:43:59] nosy: after you updated jira on amaranth – how it you restart it? by svcadm? [19:44:46] DaBPunkt: not yet unfortunatelly [19:44:58] pardon? [19:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:50:06] anyway, please hilight me when you've figure it out ;) [19:56:44] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 34814 MB (5% inode=99%): [19:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [19:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:37:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:39:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 34600 MB (5% inode=99%): [20:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [20:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:06:07] Marlen Caemmerer * [Toolserver-l] Jira Auth broken [21:07:57] i always planned to get rid of crowd once the new jira came out with built-in ldap auth that actually worked [21:37:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:39:52] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:56:42] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 34282 MB (5% inode=99%): [21:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [21:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:12:24] DaBPunkt: did you manage to review that request with nosy? [22:12:33] https://jira.toolserver.org/browse/TS-1508 [22:17:30] how do you manage to use 30 connections at the same time? Is there everything ok with the code? [22:19:26] DaBPunkt: there are thousands of users [22:20:03] I don't know the technicalities of course, but to me it doesn't seem so unlikely to go over 30 [22:20:20] It's up to you to decide if I'm right :) [22:22:59] Nemo_bis: on which server do you need the rise? [22:23:24] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24164 MB (5% inode=99%): [22:23:46] DaBPunkt: hah, you finf me unprepared [22:24:00] I've no idea.. [22:30:48] no logs? [22:37:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:40:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:45:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 33997 MB (5% inode=99%): [22:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [22:59:02] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:03:40] Nemo_bis: please leeav a comment in the bug when you found the server [23:04:01] nacht ts [23:13:23] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 31699 MB (7% inode=99%): [23:26:42] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 122676 MB (12% inode=99%): [23:37:53] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:40:23] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24411 MB (5% inode=99%): [23:40:43] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:45:02] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:56:43] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 33720 MB (5% inode=99%): [23:58:43] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [23:59:03] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default