[00:07:42] So Coren, it appears java doesn't like the job scheduling system http://fpaste.org/11748/36840365/ [00:07:57] though when the command is not run with the job sched. system, it runs just fine [00:08:08] JasonDC: It does, the key is "Could not reserve enough space for object heap" [00:08:39] JasonDC: By default, jsub only grants a very conservative 256M of VM [00:09:04] JasonDC: Give it 1G to start with (-mem 1g) and check its actual usage to see if you can scale it back down. [00:09:52] is it 1024mb or 1000mb for the 1g command (some systems love to round) [00:10:26] gridengine does worse. "1g" is 1 billion octets. :-) [00:10:37] fun, seems to be working though :) [00:11:07] I don't see it running? [00:11:11] Short run? [00:12:25] nope, it showed up for about a second then died, same error [00:12:52] i even tried telling the VM to run with a max of 800MB of RAM just to make sure, doesn't seem it likes that [00:13:52] same at 512 [00:18:09] JasonDC: It uses actally mapped memory. Try -mem = 2g [00:18:33] JasonDC: It's a guard against overcommit, not actual resident size usage. :-) [00:19:01] Java probably has a large map [00:19:11] -mem 2g [00:19:13] No = [00:19:29] nope still hates the command :( [00:20:33] o_O [00:20:50] Go all out, try 8g and we'll see what the hell java is smoking. :-) [00:21:20] If it's that bad, I'll make a java-only queue that has different metrics. [00:23:38] i think it wants 4g of ram [00:23:51] stays pretty solid there from the status indicator [00:24:21] idk why it's so fussy though, it has run on 1g laptops [00:24:24] oh well :P [00:25:29] JasonDC: That's the difference between the mapped size and the actual resident set. vmem includes the executable, shared libraries, etc. The program _could_ use all of that memory in certain circumstances, even though it almost certainly never will. [00:26:21] lovely, will note that [00:26:42] i guess i'll let it run as is, or would you like me to restart it with the smaller ram usage? [00:28:23] Java should be reasonably consisten it its memory use. Uf you say 4g you'll be pretty close. [00:38:34] awesome, restarted the task with the right mem usage and setup the cron job to run 2 times a month [00:38:41] thanks again Coren :) [00:39:07] JasonDC: My pleasure. [00:39:14] Oh, wait... [00:39:24] If you set up a cron job, make sure to add the -once option [00:39:37] ah, good point [00:40:20] incidentally, did you intend that bot to be continuously running (e.g. restarted if it happens to crash) [00:40:48] no, the terms of the BRFA were certain amounts per month given the sheer number of edits it makes [00:41:37] Allright, then just -once will do. [03:36:54] Ryan_Lane: notpeter tells me you are the openstack guru. :) [03:37:10] yes, but I'm not really here right now :) [03:37:14] Ryan_Lane: apparently UCB is using it (or will be soon) so I'm reading your docs. [03:37:17] Ryan_Lane: *nod* np [03:37:24] I'm here for about 4 more minutes [03:37:40] I may inquire about how to setup a kvm instance at some point; just getting the ssh keys and shell request stuff filled out now. [03:37:43] no rush. [03:37:51] cool [03:37:52] just wanting to say hello! ^_^ [03:38:00] what's UCB? [03:38:04] UC Berkeley [03:38:18] I'm interviewing for a l4 sysadmin position in their IT/EECS department on Tuesday. [03:38:27] figured it would be nice to get some familiarity with Openstack beforehand. [03:38:36] and using a working instance seems to be a good way to do so. ;0 [03:38:37] Ryan_Lane: Hey there. Odd time to pop up! :-) [03:38:38] :) [03:38:38] ah. cool [03:38:51] Coren: heh. I'm just online checking on for my flight tomorrow [03:39:03] if I stay on too long my GF will be unhappy ;) [03:39:13] Ryan_Lane: http://tools.wmflabs.org/?status <-- fun [03:39:23] hehe, understood. [03:39:31] Good night and good flight! [03:39:34] GF > irc [03:39:43] Ryan_Lane: Next week -> DB sprint. [03:39:48] Coren: [03:39:49] yep [03:40:00] I'm also upgrading openstack [03:43:05] neat. [03:43:28] I was looking to see if I can run it on FreeBSD; appears like the answer is presently no; but it runs kvm... so at least I can run it *on* Openstack... [03:43:53] better than nothing. ;) [03:44:03] heh [03:44:15] I don't see a reason it wouldn't work on it [03:44:24] assuming you can install all of the deps [03:44:46] ok. gotta run [03:44:50] apparently it has some linuxisms, like iptables, and use of brctl etc. [03:44:55] ttyl! :) [03:44:59] see ya :) [04:10:14] Coren: how long does it take for http://tools.wmflabs.org/?list to update? [04:11:09] Did you know that you can disable these tips but doing touch ~/.suppresstips <-- typo, but that is wonderful :D [04:12:43] legoktm: I believe it's taken directly from the file system. [04:13:23] hm [04:13:27] well it just updated now :) [04:13:34] legoktm: You can look at /data/project/.system/public_html/content/list.php. [04:13:45] <-- lazy :P [09:14:30] is there a step by step guide to getting started on labs for people used to toolserver? my searching doesn't turn up anything straightforward enough for my aging brain [09:21:55] hippietrail: https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolserver [09:23:38] aha thanks! [09:24:39] !log wikidata-dev wikidata-dev-9 edited crontab (after demo time) [11:29:33] mhhm, what is the cpu stat on http://tools.wmflabs.org/?status ? [11:35:05] addshore: CPU time, i guess. http://en.wikipedia.org/wiki/Cputime [11:35:40] mhhn [12:14:34] addshore: CPU time [12:48:57] Coren, can you create an "X!s-tools" project for me? [12:49:59] addshore, got your memo. [12:50:38] Not with a bang in its name, I can't. :-) /[a-z0-9-]{3,16}/ [12:51:35] Coren, how about "X!s-super-awesome-highly-depended-on-tools" [12:52:03] :-) *bzzzt* Sorry, try again. :-) [12:53:07] Cyberpower678: Incidentally, there is no reason why you can't just create the tool yourself now. :-) [12:53:13] Coren, "Coren-is-amazing" [12:53:19] oh? [12:56:47] Coren, how do you make a tool? [12:57:30] From https://wikitech.wikimedia.org/wiki/Special:NovaProject, look at 'Add service group' in the tools project actions bar (far right) [12:58:09] Coren: could you add me to the "tools" project? [12:58:28] hippietrail: What's your wikitech username? [12:58:45] hippietrail too i believe [12:59:03] "Successfully added hippietrail to tools." [12:59:03] Cool I can even add users [12:59:29] Cyberpower678: Well yeah, that's how you add maintainers. :-) [12:59:36] thanks. trying to see what labs is all about. i've had stuff on toolserver for years [12:59:42] nothing really big [13:00:51] Coren, Failed to create service group. [13:01:21] Cyberpower678: Like I said: /[a-z0-9-]{3,16}/ [13:01:34] You probably used bad characters. :-) [13:01:46] xtools [13:02:22] That's because someone already /has/ a xtools. :-) [13:03:13] Darn you Jack [13:04:01] Coren, add me to local-admin. [13:04:11] Err, no? [13:04:25] :-) [13:04:27] But whyyyyyyyyyyyy? [13:04:30] :p [13:04:36] Too much paperwork to fill. :-) [13:06:54] Hi, I want to join wikimedia Bot project [13:07:01] what should I do? [13:07:17] I'll add you. [13:07:24] thanks [13:07:26] What's your wikitech username. [13:07:33] Reza [13:07:49] also I have toolserver [13:08:03] Then you can't have an account. :p [13:08:04] account and I used screen and crontab with putty [13:08:06] Just kidding [13:08:20] reza: You want bots (experimental, unstable) or tools (stable) [13:08:39] (experimental, unstable) [13:08:58] reza: Ah, mad science time? :-) [13:09:34] how can i connect to server? [13:09:43] before I used winscp [13:10:04] tools-login.wmflabs.org [13:10:06] via SSH [13:10:30] Coren: is it ok to set up a "test" tool? is that the right way to go when you just want to explore the labs setup rather than start an actual new tool? [13:11:02] i'm following the instructions at https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolserver [13:11:27] hippietrail: Oh, yes. The experimental/not experimental distinction is at the infrastructure level. tools has a well tested infrastructure with supported packages, new experimental versions go to bots first for testing. [13:11:32] Coren, cough, I thought any member of a group can add another member to a group. [13:11:50] Just found out what projectadmins are for. [13:11:55] Cyberpower678: *service* groups. Not projects. :-) [13:12:02] :/ [13:12:18] service group is scary new jargon d-; [13:12:31] * Cyberpower678 feels like an idiot in front of Reedy [13:12:39] Reza: You're on bots now. [13:12:39] *reza [13:12:55] @coren thanks [13:14:05] Coren, where are the bots service groups located. [13:14:28] Cyberpower678: They don't have any, yet. Petan is the one to ask. [13:15:10] So where is cluebot running from ? [13:17:37] hippietrail: service group = MMP [13:17:37] project = cluster [13:17:59] what's an MMP? [13:18:25] hippietrail: TS-speak. "Multi-Maintainer Project" which is the default (and only) option on tools. [13:18:34] Although Multi includes "exactly one" :-) [13:19:02] ah yes i know that. it's a good idea. just a bit odd when you're starting out - who will want to share my messing about? (-: [13:20:14] The point isn't where someone will at any one point in time, but that someone /can/ if it ever becomes necessary. [13:20:24] * Cyberpower678 is hacking labs. [13:20:31] >:D [13:25:47] Hi. I've requested shell access recently https://wikitech.wikimedia.org/wiki/Shell_Request/Ebrahim can I have access to tool labs? I had toolserver account in the past (~ebr) :) [13:25:57] to convert to TS jargon [13:26:34] project admins = roots [13:26:34] what else do we have? :-) [13:28:33] ebraminio: No, you have to ask for that separately, like you just did. You have access to tools now. :-) [13:29:12] ebraminio: Getting started -> https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help [13:31:11] is there any shared place for dumps on any policies about dumps at all? [13:31:43] hippietrail: Dumps are in /public/datasets [13:31:58] Coren: Thanks!!! :) [13:35:33] Coren: do they get put there automatically or can we put the ones there that we need? [13:35:55] hippietrail: All public wikis are dumped there automagically. [13:36:13] cool (-: [13:36:47] 857 of 'em at last count. :-D [13:37:27] very nice [13:39:08] don't type "set" at the shell prompt [13:42:04] Cyberpower678: You've got serious incoming spam from one of your tools. [13:42:40] Cyberpower678: You might want to check your mail. [13:43:37] Coren, why? [13:43:46] and no new messages [13:44:12] -rw------- 1 cyberpower678 wikidev 2380834 May 13 13:43 /var/mail/cyberpower678 [13:44:41] Coren, I have no clue what that means. [13:45:04] Just 'less /var/mail/cyberpower678' for now. :-) [13:45:13] Or use 'mailx' [13:45:35] Huh? I haven't ever received an email from any tool yet. [13:46:03] You're receiving PHP errors from your mediawiki install [13:46:18] Where? [13:46:46] It's working just fine. [13:47:36] ah, i spies a reza [13:47:44] or not [13:49:49] Cyberpower678: Most seem to be 'PHP Fatal error: Uncaught exception 'BadEntryError' with message 'Bad Entry Error: BadConfig (A non-existent configuration file was specified.)' in /data/project/cyberbot/Peachy/Init.php:239' [13:50:13] Ah, not your wiki install [13:50:30] Also 'PHP Warning: file_get_contents(/home/cyberpower678/bots/isUpdating): failed to open stream: Permission denied in /data/project/cyberbot/bots/rfx-tally.php on line 46' [13:52:32] Coren, where are you seeing this? [13:52:41] ... in your mail! [13:52:56] I don't have mail!? [13:53:16] You've got over 2 megs of it! [13:53:30] Direct me to it. [13:54:10] It certainly hasn't been emailing me. [13:54:58] Use 'alpine', 'mailx', or even less like I said above. It's mail *in the project*. We'll get forwarding to external email addresses soon once the domain name has been sorted out. [13:55:25] Coren I would prefer keep it local XD or let the users pick [13:55:37] I would never want to receive this kind of mail to real inbox [13:56:15] petan: Depending on what legal decides, it's either going to be an optional .forward or an obligatory .forward. Not my call. [13:56:27] o.O [13:56:34] who cares about legal! :D [13:56:55] * petan slaps legal with huge torrent [13:56:56] There are legal issues with being an email provider they have not yet settled. [13:57:03] oh... [13:57:09] dear... [13:57:12] god... [13:57:16] O.O [13:57:49] How is my bot even running. [13:57:50] ? [13:58:13] Cyberpower678: Ask its maintainers... oh. m :-) [14:10:09] Coren, I know what's causing the BadEntryError. [14:10:19] Missing config file. [14:45:29] petan: are you around? I got blocked on wikitech [14:45:42] yes [14:45:44] hashar: I am. Wikitech blocks are random [14:45:46] blocked how? [14:45:48] The block was made by Petrb. The reason given is Spamming links to external sites. [14:45:49] Start of block: 10:52, 9 May 2013 || Expiry of block: infinite || Intended blockee: Novaadmin [14:45:54] wtf [14:45:59] can you please remove this one? [14:46:03] https://wikitech.wikimedia.org/wiki/Special:BlockList [14:46:06] sure [14:46:13] but why it blocked you?? [14:46:27] and I get a block ID #21 [14:46:43] petan: Don't block on Wikitech right now. There's something funky going on between LDAP, CanonicalUserName, and autoblocks. [14:46:57] Ryan is aware of the issue [14:47:09] I cloned from mediawiki and keep seeing this: http://tools.wmflabs.org/fatestwiki/w/ Can I have access too apache log for resolving this error? [14:47:15] o.O [14:47:22] Error: Block ID Novaadmin not found. It may have been unblocked already. [14:47:52] and block ID #21 needs to be removed as well :-] [14:47:59] hashar what is your username [14:48:02] hashar [14:48:28] I can't remove it :/ [14:48:37] ebraminio: Does your tool own the PHP script? [14:48:37] Coren can you remove it from db? [14:48:46] I don't have so much powar [14:49:07] petan: I had no trouble removing #21 [14:49:11] hashar: All clear? [14:49:13] where is a list of all root users for bots-gs? [14:49:23] weird [14:49:27] Coren: it is mediawiki code, yes it has php [14:49:29] Coren: yup works for me now thx! [14:49:34] Pyfisch in motd [14:49:43] ebraminio: I mean, the actual PHP file. Is it owned by the tool? [14:50:14] Coren: hmm, so I must chown it to tool group? [14:50:16] Ah, I see that it is not. :-) [14:50:46] ebraminio: You can't chown /to/, but you can 'take files_and_dir' from the tool account. [14:51:02] I.e.: [14:51:07] become fatestwiki [14:51:13] take public_htmll [14:51:17] take public_html * [14:51:28] that'll "Just Work™" [14:51:38] petan: motd? in the message of the day of the irc it is not of course, what do you mean? [14:53:06] I mean message of the day on server [14:53:08] bots-gs [14:53:23] when you login to that server, it is printed out I guess [14:54:06] ebraminio: Literally, type 'become fatestwiki' [14:54:15] ebraminio: That'll switch you to the tool account. :-) [14:54:24] Coren: oh, ok [14:54:37] ebraminio: From there, you can use the take command. [14:55:25] ebrahim@tools-login:/data/project/fatestwiki$ become fatestwiki [14:55:25] sudo: sorry, a password is required to run sudo [14:55:33] o_O [14:55:47] Did you log out and back in since you created the group? [14:56:12] Your group membership is set at the beginning of your session. :-) [14:57:24] ebraminio: There you go. :-) [14:57:40] Coren: Thanks!!!! [15:00:04] Coren: is it possible to have a second group for deving? i.e. addshore-dev? [15:00:12] petan: Hey, where is the build directory for apache2.4 you were working on? [15:00:30] Coren what you mean? I installed it all from apt [15:00:37] addshore: Sure, Just create a new service group. [15:00:45] oh yes, i forgot we can create them now ;p [15:00:46] packages are in /data/project/.shared/ [15:00:55] petan: You made the .deb. Where is the build directory? [15:01:01] petan, Damianz, Ryan Lane, addshore: someone changed a file in my home directory on bots-gs which contained a login password for wikimedia. The file must be changed at maximum two days ago (more likely today), but the timestamp is from 2013-04-28. I can't imagine that one of you do such things, but you are the only ones who can have read and changed with your root rights. I don't find this funny! [15:01:26] Coren nah, I just installed the one from debian together with all dependencies, why should I recreate it? [15:02:13] petan: Because I need a couple more modules, so I was going to rebuild it that way. But if you can find the apache2-ipw for 2.4 (which is part of the same sdeb) that'll do. [15:02:33] petan: I've moved the packages you already did in our project-local apt repo. [15:02:41] you should put modules to external packages [15:03:00] petan: I know, the same sdeb also sources libapache2-ipw [15:04:58] I mean, apache2-mpm-itk [15:05:20] But you just grabbed the debian prebuilts? [15:06:59] petan: Nevermind, then. I just downloaded it. :-) [15:07:13] Coren sorry I am in office and quite busy [15:07:22] Coren yes I grabbed the prebuilds [15:07:25] from sid repo [15:07:25] petan: No worries. RL work takes priority. :-) [15:13:20] petan, Damianz, Ryan Lane, addshore: someone changed a file in my home directory on bots-gs which contained a login password for wikimedia. The file must be changed at maximum two days ago (more likely today), but the timestamp is from 2013-04-28. I can't imagine that one of you do such things, but you are the only ones who can have read and changed with your root rights. I don't find this funny [15:13:48] or is it normal that you look for such information [15:13:50] ? [15:14:02] Hi I have problem with login to server with putty [15:14:14] when i want to login with tools-login.wmflabs.org it shows [15:14:29] Pyfisch hi, why do you think that anyone did it? [15:14:52] are you sure it wasn't changed by some of your tools? [15:14:52] server connection error [15:15:11] reza can you try tools-dev.wmflabs.org [15:15:28] now i try it [15:16:09] it shows the same error [15:16:23] petan: I am 100% sure that my tools did not change it. And please explain me how they should have changed it without changing the Modify time? If it was not my tools and not you who else should have changed it? [15:16:24] Pyfisch if the file has timestamp of 2013-04-28 it likely wasn't changed since then. It's possible to manipulate the timestamps, but it's complicated and I quite fail a reason why anyone would do that [15:17:29] if your file is somewhere in your home or /data/project it could be even some gluster failure (filesystem problem) [15:17:31] Pyfisch: what changed in the file? [15:18:19] addshore: one letter was removed so that my bot can't login anymore and it was disrupted exactly at 13:30 today [15:18:34] petan: my file is in /home/pyfisch [15:19:39] Pyfisch well, that /could/ be some filesystem issue... otherwise I can't think of many other reasons for this :/ [15:21:17] After mediawiki installation I am redirected to http://tools-webserver-01/.... what can I do for it? I must install install mediawiki through a proxy? [15:21:40] I had not such problem on toolserver :( [15:21:45] ebraminio: You need to make certain, when you configure your mediawiki, that the wgServerName is "tools.wmflabs.org" [15:22:09] ebraminio: You're not allowed to install MediaWiki on the toolserver. :-) [15:22:27] ebraminio: Lemme go look at your LocalSettings.php [15:23:04] Coren: I can not even download the localsetting.php :( [15:23:23] ebraminio: It's not there, the config didn't finish. Lemme check. [15:25:00] ebraminio: Did you do the configuration of your wiki yet? [15:25:11] ebraminio: http://tools.wmflabs.org/fatestwiki/w/mw-config/index.php [15:27:20] Coren: With opening of that link, my browser is just redirecting me to invalid place [15:27:34] ebraminio: o_O? It works from here. [15:27:52] Oh, wait, I know what happened. [15:31:42] Cyberpower678: you appear to be running datefix.php on -login :/ [15:32:05] ?? [15:32:14] pid 14685 [15:32:21] ?? [15:32:34] ebraminio: When you configured, you didn't set the server name properly. Lemme see if I can fix that in partial config. [15:33:03] datefix is running from Cyberbot [15:33:12] yes, but it shouldnt be running on -login [15:33:19] it should be submitted to the grid :) [15:33:24] ?? [15:33:25] Cyberpower678: You need to submit it to the grid. [15:33:33] How? [15:34:03] I'm bad with linux remember? [15:34:27] Coren: Oh, I didn't see such thing on the installation [15:34:44] ebraminio: Give me a moment while I try to remember how to fix a partial install. :-) [15:35:04] Coren: Sure. Thanks a lot! [15:35:51] how do I look at a crontab? [15:36:01] crontab -e :) [15:36:03] Cyberpower678: I'll help you in a minute. [15:36:22] Or addshore might be able to guide you a bit in the meantime. [15:37:45] Cyberpower678: rather than running your script directly from cron , your line should look something my my example below [15:37:46] 0 9 * * * jsub -mem 512m -once -N en-usertalksubst php /data/project/addbot/bot/en-usertalksubst.php [15:38:45] and where does that go? [15:39:04] addshore what is irc feed for wikidata [15:39:12] wikidata.wikipedia [15:39:15] k [15:39:53] addshore, Cyberpower678> and where does that go? [15:41:00] On to the execution nodes of the grid :-) [15:41:06] -.- [15:41:17] Laptops just died, brb :/ [15:41:35] That was very helpful. [15:41:47] It doesn't matter where they go as long as it runs right? :p [15:42:00] Is that a question? [15:42:32] Well, since it's running right then it doesn't matter. [15:42:37] Ni think they are tools-exec-01 and 02 and 3 and 4 [15:43:14] You'll have just as much luck talking to me in Chinese. [15:43:34] Oh wait, where does that line go? In your from, or remove the from ttimings from the ffront and just run it as a command on -login [15:44:06] *cron, apparently this thing has auto correct... [15:45:02] In your from, or remove the from ttimings from the ffront and just run it as a command on -login [15:45:04] What? [15:46:34] addshore, you are being very confusing right now. [15:46:37] right, my laptop is back. How are you running your tasks currently? [15:47:00] From a crontab [15:47:17] Like on toolserver [15:47:19] okay, so what does your crontab line say? [15:47:30] what line? [15:47:35] in your crontab [15:48:31] standard stuff. All of the bot scripts. [15:48:39] hashar, is PrivateSettings.php on labs versioned somewhere? [15:48:43] so, something like the below [15:48:57] MaxSem: I don't think it is. it is in .gitignore [15:48:59] 0 0 * * * php /data/project/cyberbot/script.php [15:49:04] ok, thanks [15:49:24] As your crontab is on -login, this means that your script is execusting on the login server [15:49:40] You need to change that line to something like the below [15:50:05] 0 0 * * * jsub -mem 512m -once -N taskname php /data/project/cyberbot/script.php [15:50:06] But I assigned it as cyberbot [15:50:13] !log deployment-prep Created a dummy $wmgRedisPassword = 'not set'; # MaxSem 13.05.1 [15:54:40] addshore, I was just doing what X! was doing. [15:54:47] xcombelle: [15:55:08] Before he retired. [15:55:08] sorry :/ [15:55:15] ebraminio: It's a very good place for it. Unlike the TS, we don't have a rule against running mediawiki. :-) [15:55:46] Coren: great! thanks! [15:55:52] Cyberpower678: X! was using a method that even on the TS was obsolete. [15:55:55] http://pastebin.com/1PZZ18cr [15:56:08] addshore, ^ [15:56:09] give me a few secs [15:56:12] !ping [15:56:12] pong [15:56:13] freaking wm-bot [15:56:13] petan, !pong [15:56:13] don't do that again! :D [15:56:13] you fucking scared me [15:56:33] petan, lol how? [15:56:33] !pong [15:56:33] :< [15:56:34] don't play with me [15:56:34] I thought that was a bot [15:56:39] and that it is fucked more than I thought [15:56:40] @suppresson [15:56:44] How did that go again? [15:56:45]