[00:00:28] 03/17/2012 - 00:00:28 - Creating a project directory for mingle [00:00:28] 03/17/2012 - 00:00:28 - Creating a home directory for jamesur at /export/home/mingle/jamesur [00:00:29] 03/17/2012 - 00:00:28 - Creating a project directory for shop-analytics [00:00:29] 03/17/2012 - 00:00:29 - Creating a home directory for jamesur at /export/home/shop-analytics/jamesur [00:00:48] mingle is an awesome name [00:01:29] 03/17/2012 - 00:01:29 - Updating keys for jamesur [00:01:29] 03/17/2012 - 00:01:29 - Updating keys for jamesur [00:05:26] PROBLEM Disk Space is now: WARNING on nagios 127.0.0.1 output: DISK WARNING - free space: /home/petrb 3533 MB (20% inode=87%): [00:23:55] PROBLEM Current Load is now: CRITICAL on analytics-main analytics-main output: Connection refused by host [00:24:35] PROBLEM Current Users is now: CRITICAL on analytics-main analytics-main output: Connection refused by host [00:25:15] PROBLEM Disk Space is now: CRITICAL on analytics-main analytics-main output: Connection refused by host [00:26:05] PROBLEM Free ram is now: CRITICAL on analytics-main analytics-main output: Connection refused by host [00:27:25] PROBLEM Total Processes is now: CRITICAL on analytics-main analytics-main output: Connection refused by host [00:28:15] PROBLEM dpkg-check is now: CRITICAL on analytics-main analytics-main output: Connection refused by host [02:08:15] 03/17/2012 - 02:08:14 - Creating a home directory for jamesur at /export/home/bastion/jamesur [02:09:13] 03/17/2012 - 02:09:13 - Updating keys for jamesur [02:37:15] RECOVERY Free ram is now: OK on bots-3 bots-3 output: OK: 20% free memory [02:55:15] PROBLEM Free ram is now: WARNING on bots-3 bots-3 output: Warning: 17% free memory [02:57:25] PROBLEM Free ram is now: WARNING on mobile-enwp mobile-enwp output: Warning: 18% free memory [03:55:28] PROBLEM Disk Space is now: CRITICAL on nagios 127.0.0.1 output: DISK CRITICAL - free space: /home/petrb 1862 MB (10% inode=87%): [04:37:28] RECOVERY Free ram is now: OK on mobile-enwp mobile-enwp output: OK: 20% free memory [04:50:28] PROBLEM Free ram is now: WARNING on mobile-enwp mobile-enwp output: Warning: 19% free memory [05:10:28] RECOVERY Free ram is now: OK on mobile-enwp mobile-enwp output: OK: 23% free memory [05:44:18] PROBLEM Disk Space is now: WARNING on labs-nfs1 labs-nfs1 output: DISK WARNING - free space: /export 994 MB (5% inode=87%): /home/SAVE 994 MB (5% inode=87%): [05:49:32] hmm our home directories are filling up fast [05:49:57] Make ryan to up your quota :D [05:52:20] zzz [05:52:28] Its a shared resource [05:52:37] and its not me that is filling up the home directories [05:52:43] I wonder who is doing it [06:29:24] Bot writing to nohup? :D [06:33:35] goddamn 650MB left [06:46:48] PROBLEM Current Load is now: WARNING on bots-sql3 bots-sql3 output: WARNING - load average: 5.46, 6.30, 5.40 [07:09:18] PROBLEM Disk Space is now: CRITICAL on labs-nfs1 labs-nfs1 output: DISK CRITICAL - free space: /export 491 MB (2% inode=87%): /home/SAVE 491 MB (2% inode=87%): [07:11:48] RECOVERY Current Load is now: OK on bots-sql3 bots-sql3 output: OK - load average: 1.49, 2.94, 4.61 [07:13:06] Critical :O [07:19:18] RECOVERY Disk Space is now: OK on labs-nfs1 labs-nfs1 output: DISK OK [07:20:18] RECOVERY Disk Space is now: OK on nagios 127.0.0.1 output: DISK OK [07:21:43] Meh doesn't like me to look at it lol [07:21:52] Probably a lovely bot that's having a party. [07:22:28] lol [07:22:41] freed up 9.2GB [07:22:50] huge space :O [07:22:57] lol [07:23:23] Think Ryan was moving/inprogress of moving/getting around to moving all homedirs then after they are done do project storage. [07:24:13] no. I actually deleted a nohup file of beestra's [07:24:19] :D [07:24:28] and yeah, I'm moving the homedirs to gluster storage at some point [07:24:29] I was thinking more in general about killing nfs with fire [07:24:37] beestra's bots are evil [07:25:05] * Damianz goes back to making snmp love decimals [07:25:10] heh [07:25:19] well, we'll still have nfs [07:25:22] for public datasets [07:25:37] According to this graph this rack server is pushing 160amps :D [07:26:14] since gluster apparently can't do read-only [07:26:41] You could just mount it via gluster/nfs server thing as a ro mount? [07:27:57] nope [07:28:01] doesn't work [07:28:37] That kinda sucks. [07:28:39] you can share it via nfs as read-only [07:28:44] gluster has an nfs server [07:28:48] it's managed the same way [08:11:03] 03/17/2012 - 08:11:03 - Updating keys for wikinaut [08:11:12] 03/17/2012 - 08:11:11 - Updating keys for wikinaut [10:17:56] "iworld is not in the sudoers file. This incident will be reported." [10:18:17] btw. hello! [12:52:29] !accountreq [12:52:29] in case you want to have an account on labs, please contact someone who is in charge of doing that: Ryan.Lane, m.utante or ssmolle.tt [12:53:05] Ryan_Lane: I want the right to create projects/instances [12:55:00] or mutante? ^ [13:12:05] is bots-2 down? [13:12:36] Ryan_Lane? [13:14:30] I get "ssh: connect to host bots-2 port 22: No route to host" [13:14:40] (and my bot is dead ..) [13:50:20] or ssmollett, around? [15:00:14] so is Ryan_Lane around now? [17:13:28] sumanah: around? [17:14:18] hi liangent -- I can talk for 5 min [17:14:45] sumanah: ok I want a project/instance creation right on labs [17:14:59] liangent: I am not the person to talk to for that. [17:15:01] Email labs-l [17:15:20] https://lists.wikimedia.org/mailman/listinfo/labs-l [17:15:20] hmm thanks [17:16:30] sorry [17:17:07] sumanah: https://labsconsole.wikimedia.org/wiki/Help:Access#Access_FAQ says I should ask a person directly? [17:17:28] liangent: that's for a Labsconsole *account* [17:17:29] and "If the user specifically requests Labs access, also go to Special:NovaProject and add the user, by wiki username, to the bastion project. But if they just need an account so they can log into Gerrit etc. then you don't need to do this. " is a part of it [17:18:09] ok, what's your wiki username liangent? [17:18:27] right [17:18:34] is " go to Special:NovaProject and add the user," the right to do that? [17:18:46] never mind, I can't do that [17:18:49] I don't have permissions [17:18:55] email labs-l please [17:18:59] ok [17:19:12] it's confusing, I'm sorry [17:19:22] I don't know, please ask Ryan_Lane to clarify the docs [17:20:50] liangent: do you want access to a project [17:21:00] or the ability to actually create a new project? [17:21:40] Thehelpfulone: the first is enough if someone can create one for me [17:21:45] otherwise the second [17:23:18] Thehelpfulone: are they different (so they have to be given separately)? [17:23:29] they're different [17:23:35] so I believe the latter is sys-admins only [17:23:52] and so if you email the mailing list and request a new project, justify it [17:24:05] then someone will create it for you and give you full access to your new project [17:24:25] you need someone that is a wiki admin [17:24:47] so one of these guys I believe: https://labsconsole.wikimedia.org/wiki/Special:ListUsers/sysop [17:26:33] Thehelpfulone: ok then I'll compose a mail [17:29:24] petan or petan|wk: are you here for chatting? [17:56:06] >.> My sudo isn't working on bots-cb :( [18:08:40] I haven't got access on bots :( [18:26:12] 03/17/2012 - 18:26:12 - Creating a home directory for liangent at /export/home/bastion/liangent [18:26:32] 03/17/2012 - 18:26:31 - Creating a project directory for category-sorting [18:26:32] 03/17/2012 - 18:26:31 - Creating a home directory for liangent at /export/home/category-sorting/liangent [18:27:12] 03/17/2012 - 18:27:12 - Updating keys for liangent [18:27:30] 03/17/2012 - 18:27:30 - Updating keys for liangent [19:58:10] ubuntu vim config is ugly [20:38:35] @help [20:38:35] Type @commands for list of commands. This bot is running http://meta.wikimedia.org/wiki/WM-Bot version wikimedia bot v. 1.1.4 source code licensed under GPL and located in wikimedia svn [20:38:39] ah [20:38:50] wm2012-meetbot: !help [20:38:50] IWorld: Error: "!help" is not a valid command. [20:38:55] !list [20:40:52] wm2012-meetbot: !list [20:40:52] IWorld: Error: "!list" is not a valid command. [20:40:58] wm2012-meetbot: list [20:40:58] IWorld: Admin, Channel, Config, MeetBot, Misc, Owner, and User [20:41:01] aah [20:41:16] wm2012-meetbot: meetbot [20:41:17] IWorld: Error: "meetbot" is not a valid command. [20:41:23] argh [23:22:10] zzz someone really needs to purge memcached for deployment [23:43:21] nagios seems to be down for me, help? [23:54:02] one of the virt hosts died [23:54:10] bringing it back up [23:56:25] looks like its virt3 or something [23:56:40] it is [23:56:47] heh [23:56:56] I'm checking the glusterfs filesystem before I bring the instances back up [23:59:49] cool, there is only one instance that I can login to that resides on virt3