[00:06:21] 01/01/2013 - 00:06:20 - Updating keys for johnflewis at /export/keys/johnflewis [00:24:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [00:36:24] 01/01/2013 - 00:36:23 - Updating keys for johnflewis at /export/keys/johnflewis [00:54:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [01:10:53] PROBLEM Total processes is now: WARNING on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS WARNING: 168 processes [01:15:53] RECOVERY Total processes is now: OK on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS OK: 91 processes [01:24:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [01:54:33] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [02:24:33] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [02:37:52] RECOVERY Free ram is now: OK on swift-be1.pmtpa.wmflabs 10.4.0.107 output: OK: 20% free memory [02:38:52] RECOVERY Free ram is now: OK on swift-be4.pmtpa.wmflabs 10.4.0.127 output: OK: 23% free memory [02:54:33] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [02:55:52] PROBLEM Free ram is now: WARNING on swift-be1.pmtpa.wmflabs 10.4.0.107 output: Warning: 18% free memory [03:11:52] PROBLEM Free ram is now: WARNING on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Warning: 18% free memory [03:24:33] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [03:55:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [04:25:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [04:55:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [05:05:12] PROBLEM Free ram is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Warning: 13% free memory [05:10:12] PROBLEM Free ram is now: CRITICAL on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Critical: 5% free memory [05:20:12] RECOVERY Free ram is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: OK: 496% free memory [05:25:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [05:55:54] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [06:26:13] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [06:31:52] PROBLEM Total processes is now: WARNING on parsoid-roundtrip4-8core.pmtpa.wmflabs 10.4.0.39 output: PROCS WARNING: 152 processes [06:31:52] PROBLEM Total processes is now: WARNING on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS WARNING: 155 processes [06:41:53] RECOVERY Total processes is now: OK on parsoid-roundtrip4-8core.pmtpa.wmflabs 10.4.0.39 output: PROCS OK: 148 processes [06:46:53] RECOVERY Total processes is now: OK on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS OK: 149 processes [06:56:13] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [07:26:13] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [07:56:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [08:26:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [08:56:33] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [09:26:52] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [09:56:52] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [10:27:19] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [10:57:22] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [11:21:04] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by DrTrigon link https://www.mediawiki.org/w/index.php?diff=622868 edit summary: /* Labs wide (not only bots / tools), but available for all projects */ licensing issues [11:27:23] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [11:51:52] PROBLEM Total processes is now: WARNING on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS WARNING: 151 processes [11:57:24] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [12:06:52] RECOVERY Total processes is now: OK on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS OK: 150 processes [12:14:54] PROBLEM Total processes is now: WARNING on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS WARNING: 151 processes [12:27:52] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [12:57:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [13:27:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [13:57:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [14:27:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [14:57:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [15:27:54] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [15:59:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [16:29:33] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [16:32:43] PROBLEM Free ram is now: WARNING on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: Warning: 19% free memory [16:59:33] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [17:30:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [17:38:52] Hello [17:39:35] Anyone here have any knowledge of PuTTY? [17:40:02] i've helped someone, let's see if i can help you as well [17:40:14] Ok. [18:00:53] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [18:17:28] why is labsconsole sometimes fast and sometimes slow? [18:18:53] giftpflanze: you mean the web interface? [18:19:11] i mean the wiki [18:20:03] giftpflanze: It's because we have a known problem that we haven't solved... [18:20:15] The cache process keeps crashing. When it's down, things are slow. [18:20:46] Better now, right? [18:21:20] right :) [18:21:45] is that the memcache problem? [18:21:52] This problem is right in the sweet spot where it's easy to patch over but hard to actually fix, so we just nurse it along. [18:21:58] Yep, memcached crashes. [18:23:12] i'm writing a putty help page should i name it Putty or PuTTY? [18:25:46] I have no opinion :) Whatever the authors call it [18:25:57] And, btw, thank you for writing! [18:26:06] ok, let's use titlecase :) [18:26:07] np :) [18:26:28] Having fixed one thing I am going to return to the important business of having the day off [18:26:30] i've enough experience now by remote help ;) [18:27:06] hf [18:27:15] giftpflanze: If you have a windows box that you can take screenshots from… that's a lot more than I've got [18:27:35] oh, i actually don't want to use that disk anymore :( [18:27:49] but i have putty on my linux box (though no pageant) [18:28:03] there's a linux version of putty? [18:28:06] Or are you running Wine? [18:28:08] of course :) [18:29:07] brand-name or the Free version? [18:29:13] ? [18:29:26] Just wondering if you're running CrossOver or Cedega. [18:29:33] I worked on CrossOver for years and years. [18:29:54] it's a linux package [18:30:11] Oh, nm, thought you were saying 'of course' to my question about Wine [18:30:20] i'm using no wine or other things [18:31:08] yep, I follow now. [18:31:12] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [18:31:20] OK, now I'm actually stepping away from my keyboard [18:34:50] there's a linux version of putty? < WHY WOULD YOU EVEN [18:34:53] though yes, it exists [19:01:23] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [19:25:33] PROBLEM Free ram is now: CRITICAL on dumps-bot3.pmtpa.wmflabs 10.4.0.118 output: Critical: 5% free memory [19:31:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [20:01:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [20:31:11] !log bots johnflewis: installed php5-curl on bots-4 [20:31:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [20:40:52] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [21:01:32] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [21:10:53] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [21:23:53] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [21:31:54] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [22:02:22] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [22:32:23] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [22:41:52] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [23:01:01] 01/01/2013 - 23:01:00 - Creating a home directory for chrisj at /export/keys/chrisj [23:01:53] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 21% free memory [23:02:23] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [23:06:14] 01/01/2013 - 23:06:13 - Updating keys for chrisj at /export/keys/chrisj [23:32:24] PROBLEM host: aggregator1.pmtpa.wmflabs is DOWN address: 10.4.0.79 CRITICAL - Host Unreachable (10.4.0.79) [23:34:54] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [23:51:37] 01/01/2013 - 23:51:36 - Updating keys for gifti at /export/keys/gifti [23:52:05] o.o