[00:00:27] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [00:03:17] Load avg. on willow is WARNING: WARNING - load average: 19.08, 18.28, 16.27 [00:04:16] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:04:26] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [00:05:12] Hey there, I need the python module feedparser, is it possible to get that installed somehow or do it locally? [00:06:27] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [00:07:52] You can install it locally. [00:08:06] Joan: Okay, thanks. [00:08:49] Any idea how I'd do that? [00:11:38] Well, [00:11:44] I assume you checked if it was available locally? [00:11:53] Try ... [00:11:57] python -c 'import feedparser' [00:12:09] I can't remember if I installed it locally or used the global version. [00:13:27] General installation instructions are to do something like... [00:13:37] python setup.py install --prefix=$HOME [00:13:47] And then make sure you've set $PYTHONPATH properly. [00:13:50] Ask if you have any questions. [00:13:52] I'm vaguely around. [00:14:41] okay, thanks! [00:15:20] Joan: It's not available globally, I tried that already. [00:15:31] Yeah, it looks like I have it installed locally. [00:15:52] okay, sounds good [00:19:27] Sun Grid Engine execd on willow is WARNING: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.692871/2.1, alarm hl:np_load_long=2.066895/2.5, alarm hl:cpu=97.600000/97, alarm hl:mem_free=904.000000M/200M, alarm hl:available=1/0 [00:39:27] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [00:40:10] feedparser is pretty nice. [00:40:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [00:41:37] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [00:42:26] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:43:55] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [00:45:55] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 485777 MB (9% inode=45%): [00:54:06] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 521260.000000 [01:01:28] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [01:04:06] Load avg. on willow is WARNING: WARNING - load average: 17.05, 20.31, 19.02 [01:04:26] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [01:07:27] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [01:12:15] Load avg. on willow is CRITICAL: CRITICAL - load average: 22.39, 21.28, 20.03 [01:14:15] Load avg. on willow is WARNING: WARNING - load average: 18.92, 20.62, 19.96 [01:20:36] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.968750/1.75, alarm hl:np_load_avg=2.232910/2.0, alarm hl:mem_free=420.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.968750/2.1, alarm hl:np_load_long=2.369141/2.5, alarm hl:cpu=99.900000/97, alarm hl:mem_free=420.000000M/200M, alarm hl:available=1/0 [01:31:15] Load avg. on willow is CRITICAL: CRITICAL - load average: 28.01, 21.95, 20.07 [01:32:15] Load avg. on willow is WARNING: WARNING - load average: 21.11, 21.10, 19.89 [01:38:55] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 40928 MB (10% inode=99%): [01:39:35] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:41:26] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [01:41:44] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [01:42:27] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:46:05] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 487984 MB (9% inode=45%): [01:46:55] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 77311 MB (19% inode=99%): [01:55:05] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 524921.000000 [02:01:14] Load avg. on willow is CRITICAL: CRITICAL - load average: 33.13, 22.69, 19.25 [02:01:55] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 38986 MB (9% inode=99%): [02:02:15] Load avg. on willow is WARNING: WARNING - load average: 22.13, 21.55, 19.07 [02:02:36] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [02:03:55] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 77151 MB (19% inode=99%): [02:04:27] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [02:08:25] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [02:20:45] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.687012/1.75, alarm hl:np_load_avg=2.381836/2.0, alarm hl:mem_free=477.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.687012/2.1, alarm hl:np_load_long=2.332031/2.5, alarm hl:cpu=99.800000/97, alarm hl:mem_free=477.000000M/200M, alarm hl:available=1/0 [02:40:35] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [02:41:26] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [02:41:54] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [02:42:36] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:46:06] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 487226 MB (9% inode=45%): [02:52:35] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [02:55:06] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 528523.000000 [02:55:46] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.089355/1.75, alarm hl:np_load_avg=1.840820/2.0, alarm hl:mem_free=794.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.089355/2.1, alarm hl:np_load_long=1.966309/2.5, alarm hl:cpu=98.600000/97, alarm hl:mem_free=794.000000M/200M, alarm hl:available=1/0 [03:00:15] Load avg. on willow is OK: OK - load average: 11.91, 13.35, 14.89 [03:02:36] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [03:03:15] Load avg. on willow is WARNING: WARNING - load average: 16.65, 16.23, 15.91 [03:04:36] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [03:08:35] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [03:25:55] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 38736 MB (9% inode=99%): [03:30:55] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 23633 MB (5% inode=99%): [03:40:55] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [03:41:36] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [03:41:56] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [03:42:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:46:05] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 487196 MB (9% inode=45%): [03:46:56] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 82042 MB (20% inode=99%): [03:48:30] * YuviPanda wonders if there's an easy way to calculate mainspace edit counts on toolserver [03:50:36] select count(*) from revision join page on page_id = rev_page where rev_user_text = 'user name' and page_namespace = 0; [03:50:59] You can optionally specify 'and page_is_redirect = 0' if you want.. [03:51:17] Joan: won't be too slow for 3000 users? [03:52:34] hmm, I see that's how other tools are doing it [03:52:47] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [03:53:15] Load avg. on willow is OK: OK - load average: 13.05, 14.05, 14.96 [03:55:05] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 532124.000000 [03:56:45] Sun Grid Engine execd on willow is WARNING: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.721191/2.1, alarm hl:np_load_long=1.846191/2.5, alarm hl:cpu=98.100000/97, alarm hl:mem_free=841.000000M/200M, alarm hl:available=1/0 [03:59:34] Not really. [03:59:41] There's an index on rev_user_text. [03:59:57] You'll slow down if a user has a lot of non-mainspace edits and it has to do a full scan. [04:00:10] But at most you're talking about 1 million or so rows. [04:00:17] And those are the edge cases. [04:00:26] Bots, mostly. [04:00:30] If you [04:00:34] Typing is hard. [04:00:44] If you're really worried about performance, cache the results? [04:00:45] Joan: heh [04:01:00] Joan: i'm guessing there's an index on rev_user too [04:01:01] should check [04:01:05] There is. [04:01:14] But using rev_user_text is usually easier. [04:01:18] You don't need to join to the user table. [04:01:23] And it supports IPs. [04:01:26] well, this query's already involving the user table [04:01:36] Then it doesn't really matter which you use. [04:01:39] yeah [04:01:48] There's also user.user_editcount. [04:01:51] But that's all namespaces. [04:01:54] yeah but that's all namespaces [04:01:55] yeah [04:02:05] And it's wildly inaccurate. (o; [04:02:14] Counting rows in the revision table is slower. [04:02:17] Err, closer. [04:02:22] But it still has anomalies. [04:02:24] Such as page moves. [04:02:27] Close enough, though. [04:02:46] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [04:03:16] Load avg. on willow is WARNING: WARNING - load average: 14.68, 16.00, 15.47 [04:03:52] Joan: yeah. I am wondering why revision uses pageids, when they can obviously be affected by moves/delete-recreates [04:04:00] i usually see a ns:title pair everywhere [04:04:19] Moves retain page ID. [04:04:30] Delete-recreates have always been broken and rely on ar_namespace/ar_title. [04:04:33] Historical artifacts, mostly. [04:04:40] Ideally the archive table wouldn't even exist. [04:04:43] But hindsight is 20/20. [04:04:46] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [04:05:03] :) [04:05:38] There's a field for storing a deleted page's ID (ar_page_id), but it's unused, as far as I know. [04:05:42] It's a mess. [04:06:28] ns:title on revision would've been easier atleast for me though [04:07:26] Back when it was cur and old, there was a cur_namespace/cur_title pair. [04:07:29] I think. [04:07:35] But ti's been years. [04:07:38] it's [04:07:44] he [04:07:46] *heh [04:08:36] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [04:27:55] [[Special:Log/newusers]] create 10 * Payecompanies016 * (New user account) [04:29:08] [[User:Payecompanies016]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7005&rcid=9228 * Payecompanies016 * (+4494) (Created page with "Our '''[http://www.i-paye.com/ umbrella companies liverpool]''' has great importance. Our umbrella company will take care of invoicing your agency, calculating and paying your PA...") [04:38:48] [[Special:Log/delete]] delete 10 * MZMcBride * (deleted "[[02User:Payecompanies01610]]": inappropriate content) [04:39:07] [[Special:Log/block]] block 10 * MZMcBride * (blocked [[02User:Payecompanies01610]] with an expiry time of infinite (account creation disabled): inappropriate behavior) [04:40:17] Load avg. on willow is UNKNOWN: CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [04:40:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [04:41:16] Load avg. on willow is WARNING: WARNING - load average: 15.84, 16.12, 16.39 [04:41:34] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [04:42:06] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [04:42:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:46:06] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 487146 MB (9% inode=45%): [04:55:06] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 535725.000000 [04:56:57] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [04:59:57] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.774902/1.75, alarm hl:np_load_avg=1.823730/2.0, alarm hl:mem_free=806.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.774902/2.1, alarm hl:np_load_long=1.909180/2.5, alarm hl:cpu=99.800000/97, alarm hl:mem_free=806.000000M/200M, alarm hl:available=1/0 [05:02:59] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [05:04:57] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [05:09:35] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [05:41:43] Load avg. on willow is WARNING: WARNING - load average: 17.75, 17.88, 17.89 [05:41:43] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [05:41:54] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [05:43:03] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [05:43:53] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:46:13] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 487049 MB (9% inode=45%): [05:55:13] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 539327.000000 [06:00:04] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.127441/1.75, alarm hl:np_load_avg=2.382324/2.0, alarm hl:mem_free=614.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.127441/2.1, alarm hl:np_load_long=2.393066/2.5, alarm hl:cpu=100.000000/97, alarm hl:mem_free=614.000000M/200M, alarm hl:available=1/0 [06:03:04] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [06:05:44] Load avg. on willow is CRITICAL: CRITICAL - load average: 28.44, 21.52, 20.11 [06:05:53] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [06:06:43] Load avg. on willow is WARNING: WARNING - load average: 20.64, 20.57, 19.86 [06:09:43] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [06:10:43] Load avg. on willow is CRITICAL: CRITICAL - load average: 32.96, 23.31, 20.91 [06:40:45] Load avg. on willow is CRITICAL: CRITICAL - load average: 32.09, 34.54, 28.02 [06:41:44] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [06:42:23] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [06:43:23] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [06:44:02] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:45:53] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.083984/1.00, alarm hl:np_load_long=0.818360/1.50, alarm hl:mem_free=19145.000000M/350M, alarm hl:available=1/0 [06:46:24] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486946 MB (9% inode=45%): [06:46:54] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [06:54:13] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:55:23] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 542937.000000 [07:01:03] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=3.230469/1.75, alarm hl:np_load_avg=2.737305/2.0, alarm hl:mem_free=628.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=3.230469/2.1, alarm hl:np_load_long=2.897949/2.5, alarm hl:cpu=99.000000/97, alarm hl:mem_free=628.000000M/200M, alarm hl:available=1/0 [07:04:03] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [07:06:03] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [07:09:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [07:30:58] [[Special:Log/newusers]] create 10 * Coyote du 86 * (New user account) [07:33:23] [[User:Coyote du 86]] !NM 10https://wiki.toolserver.org/w/index.php?oldid=7006&rcid=9232 * Coyote du 86 * (+91) (Created page with "Coyote du 86 is the current nick name of an user an admin from [[:w:fr:|french wikipedia]].") [07:33:53] [[User:Coyote du 86]] !M 10https://wiki.toolserver.org/w/index.php?diff=7007&oldid=7006&rcid=9233 * Coyote du 86 * (+24) () [07:34:08] [[User:Coyote du 86]] !M 10https://wiki.toolserver.org/w/index.php?diff=7008&oldid=7007&rcid=9234 * Coyote du 86 * (+1) () [07:39:44] Load avg. on willow is WARNING: WARNING - load average: 14.12, 17.38, 19.70 [07:40:42] Load avg. on willow is CRITICAL: CRITICAL - load average: 22.80, 19.11, 20.16 [07:41:43] Load avg. on willow is WARNING: WARNING - load average: 18.41, 18.58, 19.90 [07:41:53] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [07:43:13] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [07:43:24] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [07:44:03] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:47:24] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486811 MB (9% inode=45%): [07:48:54] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [07:55:24] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 546541.000000 [08:01:04] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=5.689453/1.75, alarm hl:np_load_avg=3.537109/2.0, alarm hl:mem_free=575.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=5.689453/2.1, alarm hl:np_load_long=2.841309/2.5, alarm hl:cpu=100.000000/97, alarm hl:mem_free=575.000000M/200M, alarm hl:available=1/0 [08:04:04] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [08:06:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [08:08:54] Load avg. on willow is CRITICAL: CRITICAL - load average: 14.54, 21.78, 22.67 [08:09:55] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [08:19:54] Load avg. on willow is WARNING: WARNING - load average: 14.50, 17.28, 19.98 [08:20:54] Load avg. on willow is CRITICAL: CRITICAL - load average: 22.57, 19.29, 20.53 [08:23:54] Load avg. on willow is WARNING: WARNING - load average: 15.48, 17.75, 19.72 [08:41:53] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [08:43:13] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [08:43:32] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [08:44:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:48:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486678 MB (9% inode=45%): [08:55:24] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 550141.000000 [08:57:04] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.480469/1.10, alarm hl:np_load_long=0.816406/1.55, alarm hl:mem_free=18752.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.480469/1.00, alarm hl:np_load_long=0.816406/1.50, alarm hl:mem_free=18752.000000M/350M, alarm hl:available=1/0 [09:01:03] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=3.980957/1.75, alarm hl:np_load_avg=2.959473/2.0, alarm hl:mem_free=530.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=3.980957/2.1, alarm hl:np_load_long=2.620605/2.5, alarm hl:cpu=100.000000/97, alarm hl:mem_free=530.000000M/200M, alarm hl:available=1/0 [09:02:04] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [09:04:05] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [09:06:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [09:10:54] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [09:17:03] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [09:19:13] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:20:53] Load avg. on willow is WARNING: WARNING - load average: 19.03, 16.26, 17.51 [09:26:03] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.054199/1.75, alarm hl:np_load_avg=2.017090/2.0, alarm hl:mem_free=905.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.054199/2.1, alarm hl:np_load_long=2.131348/2.5, alarm hl:cpu=99.300000/97, alarm hl:mem_free=905.000000M/200M, alarm hl:available=1/0 [09:37:05] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [09:42:04] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [09:43:13] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [09:43:43] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [09:43:52] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [09:44:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:45:05] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.028320/1.00, alarm hl:np_load_long=0.799805/1.50, alarm hl:mem_free=18844.000000M/350M, alarm hl:available=1/0 [09:46:04] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [09:48:33] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486557 MB (9% inode=45%): [09:55:34] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 553747.000000 [10:04:06] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [10:06:17] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [10:07:16] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.089355/1.75, alarm hl:np_load_avg=2.336914/2.0, alarm hl:mem_free=1074.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.089355/2.1, alarm hl:np_load_long=2.226562/2.5, alarm hl:cpu=99.300000/97, alarm hl:mem_free=1074.000000M/200M, alarm hl:available=1/0 [10:11:07] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [10:20:56] Load avg. on willow is WARNING: WARNING - load average: 23.93, 19.00, 18.06 [10:31:30] hi. would like to confirm if the bug 35709 is resolved for you. anonymous git clones from gerrit on willow should _not_ report SSL certificate problems anylonger [10:34:17] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [10:37:16] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.963867/1.75, alarm hl:np_load_avg=2.020020/2.0, alarm hl:mem_free=1104.000000M/350M, alarm hl:available=1/0 [10:39:17] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [10:42:07] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [10:43:16] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:43:56] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [10:44:16] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:48:27] Duesentrieb: wanna try an anonymous git clone from gerrit on willow and tell me if SSL cert errors are gone? [10:48:36] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486423 MB (9% inode=45%): [10:48:43] i added the SSLCA cert in order to fix a bug report [10:55:36] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 557351.000000 [10:59:16] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.841797/1.10, alarm hl:np_load_long=0.819336/1.55, alarm hl:mem_free=19144.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.841797/1.00, alarm hl:np_load_long=0.819336/1.50, alarm hl:mem_free=19144.000000M/350M, alarm hl:available=1/0 [11:04:27] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [11:06:17] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [11:10:17] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [11:11:06] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [11:17:25] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.763184/1.75, alarm hl:np_load_avg=2.185547/2.0, alarm hl:mem_free=985.000000M/350M, alarm hl:available=1/0 [11:19:16] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:21:06] Load avg. on willow is WARNING: WARNING - load average: 17.33, 16.79, 17.21 [11:27:26] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [11:35:06] Load avg. on willow is OK: OK - load average: 11.29, 13.21, 14.84 [11:38:27] Sun Grid Engine execd on willow is WARNING: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.603027/2.1, alarm hl:np_load_long=1.852051/2.5, alarm hl:cpu=97.800000/97, alarm hl:mem_free=910.000000M/200M, alarm hl:available=1/0 [11:39:27] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [11:42:18] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [11:43:26] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [11:43:56] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [11:44:26] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:48:36] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486298 MB (9% inode=45%): [11:48:56] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [11:55:37] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 560956.000000 [11:56:18] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:56:27] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:56:45] SMF on z-dat-s6-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:56:48] Hi, I'm trying to use http://toolserver.org/~luxo/contributions/contributions.php? Any idea when it will be back up? [11:56:57] SMF on z-dat-s7-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:56:57] SMF on z-dat-s4-a is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:56:57] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:57:46] MySQL slave on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [11:57:46] MySQL on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [11:57:56] MySQL slave on z-dat-s3-a is OK: Uptime: 3551264 Threads: 11 Questions: 3963070700 Slow queries: 214642 Opens: 28184631 Flush tables: 1 Open tables: 16384 Queries per second avg: 1115.960 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 242 [11:57:56] MySQL on z-dat-s3-a is OK: Uptime: 3551264 Threads: 11 Questions: 3963070706 Slow queries: 214642 Opens: 28184631 Flush tables: 1 Open tables: 16384 Queries per second avg: 1115.960 [11:58:06] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [11:58:17] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [11:58:18] SMF on z-dat-s6-a is OK: OK - all services online [11:58:27] SMF on z-dat-s7-a is OK: OK - all services online [11:58:27] SMF on z-dat-s4-a is OK: OK - all services online [11:58:27] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [12:03:06] Load avg. on willow is WARNING: WARNING - load average: 17.04, 16.25, 15.05 [12:04:27] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [12:06:26] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [12:11:06] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [12:15:36] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.287109/1.75, alarm hl:np_load_avg=2.158203/2.0, alarm hl:mem_free=916.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.287109/2.1, alarm hl:np_load_long=2.008301/2.5, alarm hl:cpu=100.000000/97, alarm hl:mem_free=916.000000M/200M, alarm hl:available=1/0 [12:43:16] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [12:43:26] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:43:56] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [12:44:27] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:48:45] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486160 MB (9% inode=45%): [12:55:46] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 564561.000000 [13:03:06] Load avg. on willow is WARNING: WARNING - load average: 13.89, 15.30, 15.98 [13:04:37] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [13:06:26] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [13:11:07] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [13:15:36] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.446289/1.75, alarm hl:np_load_avg=2.282227/2.0, alarm hl:mem_free=790.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.446289/2.1, alarm hl:np_load_long=2.142090/2.5, alarm hl:cpu=99.400000/97, alarm hl:mem_free=790.000000M/200M, alarm hl:available=1/0 [13:43:18] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [13:43:46] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [13:44:06] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [13:44:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:48:57] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486039 MB (9% inode=45%): [13:55:57] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 568175.000000 [14:03:17] Load avg. on willow is WARNING: WARNING - load average: 18.51, 19.02, 18.09 [14:04:46] [[Category:Edit counters]] ! 10https://wiki.toolserver.org/w/index.php?diff=7009&oldid=6810&rcid=9235 * 87.252.227.5 * (+60) (апісаньне па-беларуску) [14:04:47] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [14:04:58] 3(commented) [DBQ-182] List of superseded images still used in the article name space <10https://jira.toolserver.org/browse/DBQ-182> (Danhash) [14:06:47] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [14:11:15] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [14:15:47] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=2.690430/1.75, alarm hl:np_load_avg=2.459473/2.0, alarm hl:mem_free=870.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=2.690430/2.1, alarm hl:np_load_long=2.340332/2.5, alarm hl:cpu=99.800000/97, alarm hl:mem_free=870.000000M/200M, alarm hl:available=1/0 [14:17:46] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.393555/1.10, alarm hl:np_load_long=0.813477/1.55, alarm hl:mem_free=18922.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.393555/1.00, alarm hl:np_load_long=0.813477/1.50, alarm hl:mem_free=18922.000000M/350M, alarm hl:available=1/0 [14:18:53] 3(resolved) [TS-1343] s6 is currently not replicating <10https://jira.toolserver.org/browse/TS-1343> (Marlen Caemmerer) [14:18:53] 3(resolved) [MNT-1221] Hard disk failure in daphne <10https://jira.toolserver.org/browse/MNT-1221> (Marlen Caemmerer) [14:18:56] 3(created) [ACCAPP-492] Renew account?; Account Approval; New Account <10https://jira.toolserver.org/browse/ACCAPP-492> (Shilad Sen) [14:20:52] 3(commented) [ACCAPP-481] I am active user in http://fa.wiktionary.org and i want this account to run bot on this wiki also updating it's statics. <10https://jira.toolserver.org/browse/ACCAPP-481> (Abdollah Abdollahi) [14:29:46] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [14:39:46] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [14:43:16] Load avg. on willow is OK: OK - load average: 10.99, 12.76, 14.78 [14:43:47] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [14:44:16] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [14:44:16] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [14:45:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:48:46] RAID on daphne is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [14:49:56] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 485853 MB (9% inode=45%): [14:56:56] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 571837.000000 [15:02:46] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.820312/1.75, alarm hl:np_load_avg=1.764160/2.0, alarm hl:mem_free=1076.000000M/350M, alarm hl:available=1/0 [15:03:47] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [15:06:46] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [15:11:16] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [15:12:16] Load avg. on willow is WARNING: WARNING - load average: 15.74, 14.09, 13.51 [15:12:47] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.912109/1.75, alarm hl:np_load_avg=1.760254/2.0, alarm hl:mem_free=386.000000M/350M, alarm hl:available=1/0 [15:13:17] Load avg. on willow is OK: OK - load average: 13.88, 13.89, 13.48 [15:43:48] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [15:44:17] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [15:44:17] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [15:45:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:49:56] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 486162 MB (9% inode=45%): [15:53:47] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.010742/1.00, alarm hl:np_load_long=0.804688/1.50, alarm hl:mem_free=17823.000000M/350M, alarm hl:available=1/0 [15:56:56] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 575437.000000 [16:02:47] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [16:03:11] who can check MySQL replication logs? we are looking for information what happened to >>select * from image where img_name = 'Flag_of_Belarus.svg'<< in commonswiki_p, it's on s4 [16:04:42] something happened around Apr 10 03:39 Flag_of_Belarus.svg [16:04:53] (on production, not toolserver) [16:07:46] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [16:11:17] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [16:18:33] saper: no reason to do that on TS. prod has those logs too [16:19:01] jeremyb: see apregos fighting the problem on #wikimedia-tech [16:19:18] saper: i do see [16:36:04] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.325195/1.75, alarm hl:np_load_avg=1.187012/2.0, alarm hl:mem_free=219.000000M/350M, alarm hl:available=1/0 [16:37:04] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [16:43:53] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.434570/1.10, alarm hl:np_load_long=0.868164/1.55, alarm hl:mem_free=18678.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.434570/1.00, alarm hl:np_load_long=0.868164/1.50, alarm hl:mem_free=18678.000000M/350M, alarm hl:available=1/0 [16:44:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [16:44:33] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [16:44:34] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [16:44:51] saper: do i have some vague memory of you saying you knew solaris? [16:45:53] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [16:46:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:48:09] http://dpaste.com/729639/plain/ [16:48:20] still could use some help that. don't have a TS acct so i can't dig mysefl [16:48:24] myself* [16:50:04] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 485598 MB (9% inode=45%): [16:54:19] jeremyb: yes [16:54:30] my first UNIX, at 2.3 [16:54:36] saper: well see above ;) [16:55:25] and please say my name when/if you reply ;) [16:57:04] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 579039.000000 [16:57:48] jeremyb: do you have access to syslog? [16:58:58] erm? [16:59:00] jeremyb: and /var/svc/log [16:59:09] i don't have access to TS period [16:59:18] why would you need access [16:59:19] ? [16:59:31] do you have `openssl s_client` installed there? [16:59:37] yes [17:00:12] i've no idea why logs would help? [17:00:24] the problem is that puppetmaster does not start for some reason (as well as some stupid ldap configuration on damiana) [17:00:58] are we talking about SMF problems? the problem is that those services do not start/or die for some reason [17:01:02] so, why does it matter if puppetmaster's running? [17:01:10] no [17:01:19] i'm talking about my paste. did you click my link? [17:01:20] it is being told to run :) [17:01:28] ah no [17:01:30] sorry [17:01:51] should be easy w8 [17:02:44] my hypothesis is that the change needs to be made on the TS not on the gerrit server [17:02:51] and that the gerrit should be reverted [17:03:04] we already confirmed it's not fixed yet [17:03:17] the question is what does fix it ;) [17:05:03] hold on will be right back [17:08:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [17:11:34] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [17:12:55] jeremyb, http://dpaste.com/729651/ [17:13:43] looks like the server dosn't have the equifax root certificate used by wikimedia [17:15:03] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.027344/1.00, alarm hl:np_load_long=0.847656/1.50, alarm hl:mem_free=18529.000000M/350M, alarm hl:available=1/0 [17:16:04] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [17:16:10] https://en.wikipedia.org/, fails too [17:16:21] I wonder if they have any kind of certificate bundle, there [17:16:26] Platonides: i think more likely is s_client uses (by default or config) no root certs. and you have to tell it where to look for them [17:16:36] Platonides: same thing s_client does for me on debian [17:16:52] Platonides: what i'm trying to find out is where does solaris keep it's certs? [17:17:32] well, I only found three files in /etc/certs [17:17:42] SUNWObjectCA SUNWSolarisCA SUNW_SunOS_5.10 [17:17:51] which are probably used for verifying packages [17:19:02] nothing found by find /usr -name \*.crt [17:19:12] no /etc/ssl ... [17:20:03] heh, there's /etc/sfw/openssl/certs/ [17:20:28] but empty [17:21:26] I'm filing a bug [17:22:13] http://dbaspot.com/solaris/367639-where-ca-certificates-solaris-10-a-2.html seems Solaris doens't have any CA bundle [17:24:03] did you try pem? [17:24:23] it has nothing to do with OS [17:24:42] erm? [17:25:11] https://jira.toolserver.org/browse/TS-1351 [17:25:21] workaround in a minute I hope [17:25:54] 3(created) [TS-1351] Create entry for C=US, O=Equifax, OU=Equifax Secure Certificate Authority for git; Toolserver: willow; Blocker Bug <10https://jira.toolserver.org/browse/TS-1351> [17:44:04] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.918945/1.10, alarm hl:np_load_long=0.859375/1.55, alarm hl:mem_free=18134.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.918945/1.00, alarm hl:np_load_long=0.859375/1.50, alarm hl:mem_free=18134.000000M/350M, alarm hl:available=1/0 [17:44:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [17:44:43] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [17:44:52] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [17:46:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:47:03] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [17:50:04] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 485385 MB (9% inode=45%): [17:52:57] 3(commented) [TS-1351] Create entry for C=US, O=Equifax, OU=Equifax Secure Certificate Authority for git <10https://jira.toolserver.org/browse/TS-1351> [17:52:59] Platonides, jeremyb, Joan: "git config --global http.sslCAPath /home/saper/cert" is the workaround [17:53:08] hah [17:53:16] very secure ;P [17:53:24] see bug comment [17:58:03] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 582701.000000 [17:58:56] 3(updated) [TS-1351] Create entry for C=US, O=Equifax, OU=Equifax Secure Certificate Authority for git <10https://jira.toolserver.org/browse/TS-1351> [18:03:05] saper: 10 18:02:21 < wikibugs> (REOPENED) Missing SSL certificates on gerrit.wikimedia.org blocking anonymous git clones - https://bugzilla.wikimedia.org/35709 +comment (Jeremy Baron) [18:08:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [18:11:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [18:11:51] jeremyb: didn't hit save :) [18:13:03] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.299805/1.75, alarm hl:np_load_avg=1.359863/2.0, alarm hl:mem_free=297.000000M/350M, alarm hl:available=1/0 [18:14:03] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [18:25:31] [[Special:Log/newusers]] create 10 * Freak222 * (New user account) [18:39:12] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:40:10] saper_: on what? [18:44:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [18:44:28] jeremyb: on the bug [18:44:45] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [18:45:04] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [18:45:04] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.102539/1.10, alarm hl:np_load_long=0.712890/1.55, alarm hl:mem_free=18809.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.102539/1.00, alarm hl:np_load_long=0.712890/1.50, alarm hl:mem_free=18809.000000M/350M, alarm hl:available=1/0 [18:46:04] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [18:46:05] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:50:04] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 485283 MB (9% inode=45%): [18:55:54] saper: i don't follow. i see no missing info... [18:56:14] unless you're saying something's not saved that i haven't seen yet [18:58:03] SSH on adenia is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:58:03] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 586302.000000 [18:58:24] jeremyb: I wanted to submit almost the same thing as you did [18:58:34] ahhhh [18:58:49] i was pretty slow though ;P [18:58:52] SSH on adenia is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [19:01:03] * jeremyb runs away [19:08:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [19:08:53] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [19:11:44] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [19:12:54] 3(assigned) [TS-1352] Add "/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA" certificate as trusted to OpenSSL configuration on willow <10https://jira.toolserver.org/browse/TS-1352> [19:13:01] 3(created) [TS-1352] Add "/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA" certificate as trusted to OpenSSL configuration on willow; Toolserver; Task <10https://jira.toolserver.org/browse/TS-1352> [19:14:53] 3(created) [ACC-244] add deleted contribs and hersfold to conflicting accounts; ACC: Internal interface; Minor New Feature <10https://jira.toolserver.org/browse/ACC-244> (Dennis Roczek) [19:24:13] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:25:02] محمد الجداوي * [Toolserver-l] Updating pywikipedia [19:41:02] Christopher David Howie * Re: [Toolserver-l] Updating pywikipedia [19:44:04] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [19:44:44] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [19:45:14] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [19:46:04] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:50:03] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 485084 MB (9% inode=45%): [19:58:04] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 589903.000000 [20:08:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [20:09:55] 3(commented) [CHTWO-38] I am never able to use this tool to transfer images either from Wikipedia, or Wikisource <10https://jira.toolserver.org/browse/CHTWO-38> (Ineuw) [20:12:45] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [20:35:02] Platonides * Re: [Toolserver-l] Updating pywikipedia [20:44:14] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [20:44:45] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [20:45:23] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:46:14] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:47:02] محمد الجداوي * Re: [Toolserver-l] Updating pywikipedia [20:50:14] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 484975 MB (9% inode=45%): [20:58:13] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 593508.000000 [21:09:04] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [21:11:02] Christopher David Howie * Re: [Toolserver-l] Updating pywikipedia [21:12:45] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [21:14:02] Christopher David Howie * Re: [Toolserver-l] Updating pywikipedia [21:18:03] bonne nuit~ [21:18:56] 3(commented) [DBQ-182] List of superseded images still used in the article name space <10https://jira.toolserver.org/browse/DBQ-182> (Hoo man) [21:19:02] Dennis Tobar * Re: [Toolserver-l] Updating pywikipedia [21:22:02] Platonides * Re: [Toolserver-l] Updating pywikipedia [21:34:02] محمد الجداوي * Re: [Toolserver-l] Updating pywikipedia [21:44:15] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [21:44:54] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [21:45:33] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [21:46:13] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:50:15] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 484872 MB (9% inode=45%): [21:58:14] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 597109.000000 [22:03:22] I managed to lose my private ssh key, how do I change de public one? [22:09:14] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [22:13:43] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [22:44:16] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [22:45:05] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [22:45:33] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [22:46:22] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:51:15] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 484827 MB (9% inode=45%): [22:53:50] [[Iterating over wikis]] ! 10https://wiki.toolserver.org/w/index.php?diff=7010&oldid=5655&rcid=9237 * Wiegels * (-172) (/* PHP */ rewritten code) [22:58:14] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 600711.000000 [23:04:15] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:08:53] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [23:09:23] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [23:13:43] MySQL slave on rosemary is CRITICAL: (Return code of 139 is out of bounds) [23:44:24] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [23:45:03] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: Unknown database enwiki_p [23:45:44] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [23:47:23] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:52:14] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 484704 MB (9% inode=45%): [23:58:15] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 604311.000000