[00:07:05] <techman224>	 Damianz, I'm back
[00:08:13] <Damianz>	 What's the problem?
[00:08:30] <Damianz>	 also public ips are quota based, every project has 0 by default - if you didn't request an ip then it will give you an error
[00:09:03] <techman224>	 Damianz, so you have to request an public ip?
[00:09:25] <Damianz>	 yeah - we don't have that many
[00:09:34] <Damianz>	 the likes or andrewbogott_afk or Ryan_Lane can increase your quota
[00:19:42] <Ryan_Lane>	 techman224: which project?
[00:20:19] <techman224>	 Ryan_Lane, titleblacklist
[00:20:36] <Ryan_Lane>	 !resource titleblacklist
[00:20:36] <wm-bot>	 https://labsconsole.wikimedia.org/wiki/Nova_Resource:titleblacklist
[00:21:29] <Ryan_Lane>	 techman224: done
[00:26:53] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be3.pmtpa.wmflabs 10.4.0.124 output: Warning: 18% free memory
[00:32:09] <Damianz>	 Totally need a quota display on the project page
[01:08:23] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: Warning: 19% free memory
[01:09:43] <labs-nagios-wm>	 PROBLEM Total processes is now: WARNING on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS WARNING: 169 processes
[01:14:42] <labs-nagios-wm>	 RECOVERY Total processes is now: OK on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS OK: 91 processes
[01:28:23] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on bots-2.pmtpa.wmflabs 10.4.0.42 output: OK: 20% free memory
[01:41:22] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: Warning: 18% free memory
[02:21:53] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on swift-be3.pmtpa.wmflabs 10.4.0.124 output: OK: 20% free memory
[02:29:53] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be3.pmtpa.wmflabs 10.4.0.124 output: Warning: 18% free memory
[02:39:53] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on swift-be3.pmtpa.wmflabs 10.4.0.124 output: OK: 22% free memory
[03:37:53] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be3.pmtpa.wmflabs 10.4.0.124 output: Warning: 19% free memory
[04:52:52] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on swift-be3.pmtpa.wmflabs 10.4.0.124 output: OK: 20% free memory
[05:00:34] <labs-nagios-wm>	 PROBLEM Current Load is now: WARNING on parsoid-roundtrip7-8core.pmtpa.wmflabs 10.4.1.26 output: WARNING - load average: 4.99, 5.15, 5.09
[05:10:43] <labs-nagios-wm>	 PROBLEM Free ram is now: UNKNOWN on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: NRPE: Call to fork() failed
[05:10:53] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be3.pmtpa.wmflabs 10.4.0.124 output: Warning: 19% free memory
[05:13:53] <labs-nagios-wm>	 PROBLEM Disk Space is now: UNKNOWN on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: NRPE: Call to fork() failed
[05:14:23] <labs-nagios-wm>	 PROBLEM Total processes is now: UNKNOWN on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: NRPE: Call to fork() failed
[05:14:33] <labs-nagios-wm>	 PROBLEM dpkg-check is now: UNKNOWN on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: NRPE: Call to fork() failed
[05:17:23] <labs-nagios-wm>	 PROBLEM Current Users is now: UNKNOWN on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: NRPE: Call to fork() failed
[05:18:53] <labs-nagios-wm>	 PROBLEM Disk Space is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: CHECK_NRPE: Error - Could not complete SSL handshake.
[05:19:23] <labs-nagios-wm>	 PROBLEM Total processes is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: CHECK_NRPE: Error - Could not complete SSL handshake.
[05:19:33] <labs-nagios-wm>	 PROBLEM dpkg-check is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: CHECK_NRPE: Error - Could not complete SSL handshake.
[05:20:32] <labs-nagios-wm>	 PROBLEM SSH is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: Server answer:
[05:20:32] <labs-nagios-wm>	 RECOVERY Current Load is now: OK on parsoid-roundtrip7-8core.pmtpa.wmflabs 10.4.1.26 output: OK - load average: 4.42, 4.78, 4.96
[05:20:42] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: CHECK_NRPE: Error - Could not complete SSL handshake.
[05:21:42] <labs-nagios-wm>	 PROBLEM Current Load is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: CHECK_NRPE: Error - Could not complete SSL handshake.
[05:22:22] <labs-nagios-wm>	 PROBLEM Current Users is now: CRITICAL on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: CHECK_NRPE: Error - Could not complete SSL handshake.
[05:27:23] <labs-nagios-wm>	 RECOVERY Current Users is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: USERS OK - 0 users currently logged in
[05:28:53] <labs-nagios-wm>	 RECOVERY Disk Space is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: DISK OK
[05:29:23] <labs-nagios-wm>	 RECOVERY Total processes is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: PROCS OK: 124 processes
[05:29:33] <labs-nagios-wm>	 RECOVERY dpkg-check is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: All packages OK
[05:30:32] <labs-nagios-wm>	 RECOVERY SSH is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1 (protocol 2.0)
[05:30:42] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: OK: 65% free memory
[05:31:42] <labs-nagios-wm>	 RECOVERY Current Load is now: OK on dumps-bot1.pmtpa.wmflabs 10.4.0.4 output: OK - load average: 0.03, 0.55, 0.81
[05:46:32] <AMadman>	 !log cvresearch Installed HTTP_OAuth on cvresearch-bots.
[05:46:34] <labs-morebots>	 Logged the message, Master
[06:28:54] <labs-nagios-wm>	 PROBLEM Total processes is now: WARNING on parsoid-roundtrip4-8core.pmtpa.wmflabs 10.4.0.39 output: PROCS WARNING: 152 processes
[06:30:32] <labs-nagios-wm>	 PROBLEM Total processes is now: WARNING on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS WARNING: 156 processes
[06:41:32] <labs-nagios-wm>	 PROBLEM Total processes is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: PROCS WARNING: 154 processes
[06:51:34] <labs-nagios-wm>	 RECOVERY Total processes is now: OK on bots-2.pmtpa.wmflabs 10.4.0.42 output: PROCS OK: 149 processes
[06:51:34] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on dumps-bot3.pmtpa.wmflabs 10.4.0.118 output: Warning: 19% free memory
[06:55:20] <labs-home-wm>	 01/06/2013 - 06:55:19 - Updating keys for beetstra at /export/keys/beetstra
[06:55:33] <Beetstra>	 oh
[06:55:33] <labs-nagios-wm>	 PROBLEM Current Load is now: WARNING on parsoid-roundtrip7-8core.pmtpa.wmflabs 10.4.1.26 output: WARNING - load average: 4.86, 5.37, 5.27
[06:58:53] <labs-nagios-wm>	 RECOVERY Total processes is now: OK on parsoid-roundtrip4-8core.pmtpa.wmflabs 10.4.0.39 output: PROCS OK: 148 processes
[07:00:32] <labs-nagios-wm>	 RECOVERY Total processes is now: OK on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS OK: 150 processes
[07:04:32] <labs-nagios-wm>	 PROBLEM Total processes is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: PROCS WARNING: 152 processes
[07:25:42] <labs-nagios-wm>	 RECOVERY Current Load is now: OK on parsoid-roundtrip7-8core.pmtpa.wmflabs 10.4.1.26 output: OK - load average: 5.08, 4.90, 4.97
[07:25:52] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on swift-be3.pmtpa.wmflabs 10.4.0.124 output: OK: 20% free memory
[07:33:33] <labs-nagios-wm>	 PROBLEM Current Load is now: WARNING on parsoid-roundtrip7-8core.pmtpa.wmflabs 10.4.1.26 output: WARNING - load average: 5.50, 5.31, 5.12
[07:43:52] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be3.pmtpa.wmflabs 10.4.0.124 output: Warning: 18% free memory
[09:52:33] <labs-nagios-wm>	 PROBLEM Current Load is now: WARNING on parsoid-roundtrip7-8core.pmtpa.wmflabs 10.4.1.26 output: WARNING - load average: 5.72, 5.19, 5.15
[09:53:53] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on swift-be3.pmtpa.wmflabs 10.4.0.124 output: OK: 20% free memory
[10:16:52] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be3.pmtpa.wmflabs 10.4.0.124 output: Warning: 18% free memory
[10:47:33] <labs-nagios-wm>	 RECOVERY Current Load is now: OK on parsoid-roundtrip7-8core.pmtpa.wmflabs 10.4.1.26 output: OK - load average: 4.27, 4.27, 4.98
[11:04:33] <labs-nagios-wm>	 RECOVERY Total processes is now: OK on bots-2.pmtpa.wmflabs 10.4.0.42 output: PROCS OK: 120 processes
[11:06:23] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on bots-2.pmtpa.wmflabs 10.4.0.42 output: OK: 51% free memory
[11:12:32] <labs-nagios-wm>	 PROBLEM Total processes is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: PROCS WARNING: 155 processes
[11:51:53] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Critical: 5% free memory
[12:07:32] <labs-nagios-wm>	 RECOVERY Total processes is now: OK on bots-2.pmtpa.wmflabs 10.4.0.42 output: PROCS OK: 145 processes
[12:11:54] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Warning: 6% free memory
[12:16:54] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Critical: 5% free memory
[13:11:53] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Warning: 6% free memory
[13:14:23] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: Warning: 18% free memory
[14:20:21] <labs-home-wm>	 01/06/2013 - 14:20:20 - Updating keys for hydriz at /export/keys/hydriz
[14:21:52] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Critical: 5% free memory
[15:51:52] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Warning: 7% free memory
[16:24:22] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on bots-2.pmtpa.wmflabs 10.4.0.42 output: Critical: 5% free memory
[16:46:53] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Critical: 5% free memory
[17:44:22] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: Warning: 6% free memory
[17:54:24] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on bots-2.pmtpa.wmflabs 10.4.0.42 output: Critical: 5% free memory
[18:51:53] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Warning: 7% free memory
[19:14:46] <addshore>	 !logs
[19:14:46] <wm-bot>	 logs http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-labs
[19:15:27] <addshore>	 !msys
[19:15:28] <wm-bot>	 MSYS is a collection of GNU utilities such as bash, make, gawk and grep for windows http://www.mingw.org/wiki/MSYS
[19:15:30] <addshore>	 mysys
[19:15:32] <addshore>	 :D
[19:15:50] <Damianz>	 eww
[19:18:00] <addshore>	 why ewwww
[19:18:01] <addshore>	 ? :<
[19:18:59] <Damianz>	 windows
[19:19:04] <addshore>	 oh shh
[19:19:06] <addshore>	 i know
[19:21:52] <labs-nagios-wm>	 PROBLEM Free ram is now: CRITICAL on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Critical: 5% free memory
[19:31:21] <petan>	 Damianz all my games don't work in linux :(
[19:31:29] <addshore>	 petan!
[19:31:35] <petan>	 I use windows as well for entertainment!
[19:31:36] <addshore>	 i have msys but no ssh command?
[19:31:37] <petan>	 :D
[19:31:37] <Damianz>	 I don't play games :D
[19:31:44] <petan>	 Damianz shame on you
[19:31:50] <Damianz>	 Too busy
[19:32:02] <petan>	 addshore mm that's weird, maybe you need to install it separately, but git-bash contains it
[19:32:32] <addshore>	 well, if i have it its not called ssh :P
[19:32:41] <petan>	 !git
[19:32:42] <wm-bot>	 For more information about git on labs see https://labsconsole.wikimedia.org/wiki/Help:Git
[19:33:02] <petan>	 !msys-git is http://msysgit.github.com/
[19:33:02] <wm-bot>	 Key was added
[19:33:45] <addshore>	 ..
[19:35:49] <petan>	 I only play games in windows, sorry :/
[19:35:55] <petan>	 for real work I switch to linux
[19:36:07] <addshore>	 right im so confused now
[19:36:09] <addshore>	 :D
[19:36:13] <petan>	 :D :D
[19:36:23] <petan>	 this git thing contains ssh
[19:36:36] <petan>	 I think you can just download ssh and compile it once you have msys
[19:36:40] <petan>	 ./configure
[19:36:41] <petan>	 make
[19:36:42] <petan>	 etc
[19:36:44] <petan>	 make install
[19:36:45] <petan>	 :D
[19:36:45] <addshore>	 from the repo?
[19:37:00] <petan>	 from its download page :P
[19:37:04] <petan>	 it's gnu I think
[19:37:17] <addshore>	 i found it in https://github.com/msysgit/msysgit/tree/master/bin
[19:37:17] <addshore>	 xD
[19:37:22] <petan>	 aha
[19:37:31] <petan>	 interesting
[19:37:40] <addshore>	 right, im gonna go with the theroy i just stick that in my /bin ;p
[19:37:46] <petan>	 maybe you can download this .zip package and extract just ssh binary
[19:38:23] <addshore>	 only 1 way to find out :P
[19:38:40] <petan>	 https://github.com/msysgit/msysgit/blob/master/bin/ssh.exe
[19:38:44] <petan>	 :D
[19:38:46] <petan>	 here it is
[19:38:49] <addshore>	 ya
[19:38:56] <petan>	 download and copy
[19:38:58] <petan>	 into /bin
[19:38:59] <petan>	 :P
[19:39:07] <petan>	 but you will for sure need openssl libs
[19:39:08] <petan>	 and such
[19:42:21] <addshore>	 yee
[19:42:28] <addshore>	 needed to go find a few dependancys xD
[19:42:31] <addshore>	 but now it works :D
[19:42:33] <addshore>	 !tunel
[19:42:53] <addshore>	 !tunnel
[19:42:53] <wm-bot>	 ssh -f user@bastion.wmflabs.org -L <local>:server:<remote> -N Example for sftp "ssh chewbacca@bastion.wmflabs.org -L 6000:bots-1:22 -N" will open bots-1:22 as localhost:6000
[19:45:19] <addshore>	 haha
[19:45:27] <addshore>	 now to add my keys to ssh :P
[19:46:54] <addshore>	 petan? :P
[19:49:22] * addshore  slaps petan 

[19:55:50] <addshore>	 petan, ? :<
[20:02:19] <addshore>	 PETAN!
[20:06:19] <addshore>	 :<
[20:20:20] <labs-home-wm>	 01/06/2013 - 20:20:20 - Updating keys for seb35 at /export/keys/seb35
[20:25:20] <petan>	 addshore hm?
[20:25:30] <addshore>	 no problem, got it working
[20:25:35] <petan>	 ok
[20:25:36] <addshore>	 was having probs with keys and forwarding :P
[20:25:40] <addshore>	 found an article :)
[21:09:23] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on bots-2.pmtpa.wmflabs 10.4.0.42 output: Warning: 9% free memory
[21:21:53] <labs-nagios-wm>	 PROBLEM Free ram is now: WARNING on swift-be4.pmtpa.wmflabs 10.4.0.127 output: Warning: 6% free memory
[21:24:23] <labs-nagios-wm>	 RECOVERY Free ram is now: OK on bots-2.pmtpa.wmflabs 10.4.0.42 output: OK: 28% free memory
[21:53:53] <labs-nagios-wm>	 PROBLEM dpkg-check is now: CRITICAL on nova-precise2.pmtpa.wmflabs 10.4.1.57 output: DPKG CRITICAL dpkg reports broken packages
[21:58:52] <labs-nagios-wm>	 RECOVERY dpkg-check is now: OK on nova-precise2.pmtpa.wmflabs 10.4.1.57 output: All packages OK
[23:05:22] <addshore>	 does ganglia not work for the individual hosts?
[23:05:56] <Damianz>	 ?
[23:06:32] <Damianz>	 If you mean why are all the graphs blank, it's just broken
[23:06:44] * Damianz  looks at Ryan_Lane who fixed it last

[23:07:00] <Ryan_Lane>	 well, I fixed the interface
[23:07:07] <Damianz>	 *cough*
[23:07:09] <Ryan_Lane>	 I still need to fix that
[23:07:36] <Damianz>	 So on the bright side, it doesn't time out, on the dull side, there's nothing to see? :D
[23:08:10] <Ryan_Lane>	 heh
[23:08:13] <Ryan_Lane>	 indeed
[23:08:59] <addshore>	 :P
[23:09:14] <addshore>	 I remember i looked at it a while ago and it was working :P
[23:09:40] <Ryan_Lane>	 yeah. it may be a simple fix, but I haven't had time to look at it