[00:18:30] Brooke: you too, i really enjoyed you in Neighbours. [00:20:34] Heh. [00:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 31149 MB (5% inode=99%): [00:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [00:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:02:31] Free Memory on damiana is WARNING: WARNING - 6.6% (555276 kB) free! [01:03:33] that check is so wrong [01:03:53] it has at least 1.4GB free [01:08:32] Free Memory on damiana is OK: OK - 7.5% (629340 kB) free. [01:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 30974 MB (5% inode=99%): [01:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [01:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:31:12] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 94232 MB (9% inode=98%): [02:31:12] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 94232 MB (9% inode=98%): [02:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 30830 MB (5% inode=99%): [02:54:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [02:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:34:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:40:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 30686 MB (5% inode=99%): [03:54:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [03:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:16:11] Load avg. on willow is WARNING: WARNING - load average: 18.22, 16.71, 14.07 [04:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:36:32] Free Memory on damiana is WARNING: WARNING - 6.9% (577376 kB) free! [04:37:31] Free Memory on damiana is OK: OK - 8.2% (688100 kB) free. [04:38:12] Load avg. on willow is OK: OK - load average: 13.49, 14.49, 14.96 [04:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 30547 MB (4% inode=99%): [04:54:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [04:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:08:31] Free Memory on damiana is WARNING: WARNING - 6.4% (539052 kB) free! [05:15:31] Free Memory on damiana is OK: OK - 7.7% (645336 kB) free. [05:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 30366 MB (4% inode=99%): [05:47:22] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 18337 MB (1% inode=96%): [05:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [05:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:25:32] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 126622 MB (13% inode=99%): [06:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 30182 MB (4% inode=99%): [06:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [06:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:40:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 29992 MB (4% inode=99%): [07:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [07:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:15:34] @replag [08:15:35] liangent: s2-user: 40m 58s [-0.91 s/s]; s3-rr-a: 1m 43s [+0.00 s/s]; s3-user: 1m 43s [+0.00 s/s]; s7-rr-a: 13s [-0.00 s/s]; s7-user: 13s [-0.00 s/s] [08:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 29769 MB (4% inode=99%): [08:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [08:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:34:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:35:12] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2165 [09:36:01] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2212.000000 [09:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 29564 MB (4% inode=99%): [09:54:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [09:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:59:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3605 [10:00:01] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3652.000000 [10:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:24:31] Free Memory on damiana is WARNING: WARNING - 5.7% (475928 kB) free! [10:32:32] Free Memory on damiana is OK: OK - 7.9% (665756 kB) free. [10:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 29332 MB (4% inode=99%): [10:42:12] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 32957 MB (8% inode=99%): [10:51:31] Free Memory on damiana is WARNING: WARNING - 6.1% (509520 kB) free! [10:54:02] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3473.000000 [10:54:12] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3421 [10:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [10:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:59:31] Free Memory on damiana is OK: OK - 7.2% (604260 kB) free. [11:03:11] MySQL slave on rosemary is OK: Uptime: 12167149 Threads: 27 Questions: 5018978401 Slow queries: 1206857 Opens: 124554 Flush tables: 6 Open tables: 3571 Queries per second avg: 412.502 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1783 [11:04:02] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1663.000000 [11:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 29127 MB (4% inode=99%): [11:47:11] Load avg. on willow is CRITICAL: CRITICAL - load average: 68.84, 53.42, 31.93 [11:54:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [11:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:07:31] Free Memory on damiana is WARNING: WARNING - 6.4% (536040 kB) free! [12:13:32] Free Memory on damiana is OK: OK - 7.1% (594356 kB) free. [12:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:34:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:40:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 28920 MB (4% inode=99%): [12:43:31] Free Memory on damiana is WARNING: WARNING - 6.1% (508552 kB) free! [12:47:11] Load avg. on willow is CRITICAL: CRITICAL - load average: 10.67, 30.87, 54.27 [12:53:59] hello all [12:54:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [12:55:32] Free Memory on damiana is OK: OK - 7.1% (593604 kB) free. [12:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:34:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:40:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 28663 MB (4% inode=99%): [13:47:12] Load avg. on willow is CRITICAL: CRITICAL - load average: 76.05, 77.67, 72.58 [13:48:02] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2200.000000 [13:48:12] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2213 [13:54:21] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [13:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:12:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3617.000000 [14:12:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3630 [14:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:31:11] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 93450 MB (9% inode=98%): [14:31:11] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 93450 MB (9% inode=98%): [14:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:34:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:36:31] Free Memory on damiana is WARNING: WARNING - 5.9% (497380 kB) free! [14:40:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 28372 MB (4% inode=99%): [14:41:02] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3549.000000 [14:41:12] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3526 [14:41:45] @replag [14:41:46] DaBPunkt: s1-rr-a: 58m 7s [+0.07 s/s]; s1-user: 58m 7s [+0.07 s/s]; s2-user: 11m 34s [+0.02 s/s]; s3-rr-a: 52s [-0.00 s/s]; s3-user: 52s [-0.00 s/s] [14:42:32] Free Memory on damiana is OK: OK - 7.1% (594388 kB) free. [14:46:38] DaBPunkt, can you add log_page to the logging table view of commonswiki_p ? [14:47:11] Load avg. on willow is CRITICAL: CRITICAL - load average: 17.96, 53.54, 67.40 [14:49:29] Platonides: open a jira-bug, I will do it soon [14:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [14:56:32] DaBPunkt, https://jira.toolserver.org/browse/TS-1501 [14:57:16] there's another view-one a few days old, too https://jira.toolserver.org/browse/TS-1497 [14:57:30] yes, I will do them together [14:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:25:42] Platonides: should be done [15:26:26] thanks [15:29:53] np [15:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 28104 MB (4% inode=99%): [15:41:01] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2987.000000 [15:41:12] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2996 [15:47:12] Load avg. on willow is CRITICAL: CRITICAL - load average: 10.75, 14.44, 24.57 [15:49:32] Free Memory on damiana is WARNING: WARNING - 5.7% (481116 kB) free! [15:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [15:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:00:12] Load avg. on willow is WARNING: WARNING - load average: 10.93, 12.60, 19.43 [16:05:12] Load avg. on willow is CRITICAL: CRITICAL - load average: 36.45, 17.93, 19.34 [16:08:31] Free Memory on damiana is OK: OK - 7.1% (594368 kB) free. [16:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:34:47] @replag [16:34:48] liangent: s1-rr-a: 45m 0s [-0.12 s/s]; s1-user: 45m 0s [-0.12 s/s]; s2-user: 15m 27s [+0.03 s/s]; s3-rr-a: 11s [-0.01 s/s]; s3-user: 11s [-0.01 s/s] [16:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 27857 MB (4% inode=99%): [16:41:01] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2949.000000 [16:41:11] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2951 [16:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [16:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:05:12] Load avg. on willow is CRITICAL: CRITICAL - load average: 67.37, 63.82, 62.95 [17:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:23:32] Free Memory on damiana is WARNING: WARNING - 6.2% (518928 kB) free! [17:25:16] nosy: Hi [17:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:36:32] Free Memory on damiana is OK: OK - 7.1% (591800 kB) free. [17:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 27682 MB (4% inode=99%): [17:41:01] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2778.000000 [17:41:12] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2776 [17:47:21] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 19582 MB (1% inode=96%): [17:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [17:58:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:02:32] Free Memory on damiana is WARNING: WARNING - 6.2% (520048 kB) free! [18:05:12] Load avg. on willow is CRITICAL: CRITICAL - load average: 61.88, 61.99, 61.65 [18:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:25:32] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 126784 MB (13% inode=99%): [18:25:32] Free Memory on damiana is OK: OK - 7.1% (592936 kB) free. [18:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:40:12] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 27574 MB (4% inode=99%): [18:41:01] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3122.000000 [18:41:12] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3125 [18:46:11] Load avg. on willow is WARNING: WARNING - load average: 10.77, 11.27, 19.69 [18:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [18:55:11] Load avg. on willow is OK: OK - load average: 8.26, 8.73, 14.68 [18:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:22:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:23:33] nosy: ping [19:23:55] apmon: pong [19:24:27] Do you think it would be a good time to try and switch the dbs on ptolemy now? [19:24:51] apmon: i think i stilll have to prepare some stuff [19:25:03] OK [19:25:10] import user databases, change config, restart so to say [19:25:17] do you remember more? [19:25:41] no, I think that should be it, as far as I am aware [19:26:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3615.000000 [19:26:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3622 [19:27:11] I have stopped the replication and the normal rendering. So from my perspective, you should be able to restart postgres without problems if you need to [19:28:07] apmon: ok i willl let you know, first i will do the user databases [19:28:18] cheers [19:30:20] I think the old db is stable again, so you hopefully shouldn't have any crash issues while trying to dump the db-data [19:31:35] apmon: why is it stable again now? because of the postgis query that failed and you changed it? [19:32:41] Yes, it looks like the db might not have actually been corrupted, but the ST_Intersection query crashed the db so often it looked like the db was corrupted. Unfortunately we found that out too late :-S [19:33:12] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 77035 MB (12% inode=99%): [19:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:34:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:40:23] apmon: oh, ok...what about reimporting the database called gis? [19:40:37] or do i only have to reimport u_*? [19:41:45] gis appears to be empty (unless I am missing read permissions) [19:42:22] it probably was created a long time ago during test imports (as the default db name for osm2pgsql is gis) [19:42:42] true...gis is empty... [19:42:51] you probably also want to copy p_lalm [19:43:26] Oh, that is actually also empty by the looks of it [19:45:45] yes... [19:45:55] did you expect something different? [19:46:17] i will keep the old install for some days in case we forgot somethin [19:46:29] Yes, that would be good [19:48:11] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 66442 MB (10% inode=99%): [19:49:27] did not expect the kolossos db to be that big [19:49:28] wow [19:49:54] Yes, it seems to be 10-20Gb :-S [19:50:05] But I see you managed to free an extra 50Gb on /sql [19:50:06] should be 20 or something [19:50:24] yes i have too many logs [19:50:59] i still log slow queries but i can turn it off - no one reads this [19:51:49] Is it possible to make a section of those logs available for analysis? [19:52:33] Perhaps, once everything is back to normal / steady state, one could log slow queries for an hour or so and then see if anything sticks out. [19:52:52] But constant logging of slow queries is definitely not necessary [19:53:12] so i should rather keep only one hour? [19:53:53] I doubt anyone will look through 50Gb of logs ;-) [19:54:22] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [19:55:24] true - what about parsing? :D [19:55:42] ok...it is not necessary [19:56:08] i have changed the config and hope it works the way it only has the data of one day [19:56:36] kolossos-db is still dumping and currently 37 gb [19:56:40] I think there are some scripts that try and do some parsing, but probably one would have to write some new scripts first [19:56:47] ouch... [19:57:26] In what format are you dumping it? [19:57:42] text [19:57:52] uncompressed... [19:57:57] It might be easier to dump it in the binary format [19:58:07] ah, stopped dumping... [19:58:26] I think that also allows to do parallel dumping / restore [19:58:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:58:50] that might be - yes - did not think about it.... [20:02:31] Free Memory on damiana is WARNING: WARNING - 6.3% (524192 kB) free! [20:08:32] Free Memory on damiana is OK: OK - 7.3% (608852 kB) free. [20:16:36] DaBPunkt: can sqlalchemy be installed? -> http://pypi.python.org/pypi/SQLAlchemy/ [20:22:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:26:01] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3811.000000 [20:26:02] Alchimista: yes, fill in a jira-bug [20:26:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3823 [20:30:31] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 31.66, 28.46, 19.41 [20:32:32] Load avg. on ortelius is WARNING: WARNING - load average: 13.59, 22.92, 18.44 [20:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:33:34] nosy: looks like the u_kolossos import is complete now [20:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:48:22] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22926 MB (3% inode=99%): [20:54:32] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [20:54:46] apmon: sorry i have to run lets try again tomorrow - this should work then [20:55:03] OK [20:55:29] I'll reactive diff imports on the db on port 5433 for now then again [20:58:42] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:00:31] Free Memory on damiana is WARNING: WARNING - 6.3% (529972 kB) free! [21:06:32] Free Memory on damiana is OK: OK - 7.3% (608772 kB) free. [21:20:02] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1948.000000 [21:21:31] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 23.02, 21.27, 20.04 [21:22:42] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:26:02] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4615.000000 [21:26:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4620 [21:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:34:01] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1770.000000 [21:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:48:32] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22696 MB (3% inode=99%): [21:53:21] anyone know of a tool that maps geocoded articles that are missing photos to a map? [21:54:32] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [21:58:42] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:01:32] Free Memory on damiana is WARNING: WARNING - 5.5% (463340 kB) free! [22:12:32] Free Memory on damiana is OK: OK - 7.4% (620188 kB) free. [22:21:32] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 34.79, 29.75, 26.95 [22:22:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:25:08] nacht ts [22:26:01] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5438.000000 [22:26:21] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5455 [22:27:32] Free Memory on damiana is WARNING: WARNING - 5.6% (469984 kB) free! [22:33:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:33:32] Free Memory on damiana is OK: OK - 7.2% (601684 kB) free. [22:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:42:22] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 32246 MB (7% inode=99%): [22:49:21] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22335 MB (3% inode=99%): [22:54:32] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [22:56:32] Free Memory on damiana is WARNING: WARNING - 6.4% (540208 kB) free! [22:58:41] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:00:32] Free Memory on damiana is OK: OK - 7.3% (615104 kB) free. [23:21:32] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 23.13, 25.23, 26.12 [23:22:42] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:26:01] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6422.000000 [23:26:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6416 [23:33:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:34:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:49:22] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 22001 MB (3% inode=99%): [23:54:32] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [23:58:42] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default