[00:15:45] Coren: I'm nearly positive labstore3/4 already have NFS rules set up [00:15:49] they were used for gluster [00:16:06] Ryan_Lane: From ae0-105.cr1-sdtpa.wikimedia.org (10.4.16.252) icmp_seq=1 Packet filtered [00:16:33] Can't ping, and tcpdump don't see the packets arriving. [00:16:48] can't ping from in labs? [00:17:02] Nope. [00:17:19] icmp isn't allowed [00:17:20] NFS works [00:17:38] tcpdump calls you mistaken. :-) [00:17:54] from labs: showmount -e labstore1.pmtpa.wmnet [00:18:14] no clue why it isn't working for labstore4 [00:18:22] Perhaps that's just labstore*1*? [00:18:37] nope [00:18:40] works on 2 as well [00:18:46] and the filters are the same for all 4 [00:18:49] is portmap running? [00:18:55] Well, labstore3 times out. [00:19:07] hm [00:19:08] weird [00:19:21] portmap works, actually. [00:19:24] oh [00:19:30] you need to set the ports [00:19:38] Oh! You're not using the defaults! [00:19:42] nfs doesn't work through filters or firewalls by default [00:20:00] It does if you use the standard ports. :-) [00:20:00] you have to set the services to run on specific ports [00:20:05] Which ones do you guys use? [00:20:16] no it doesn't, because portmap will assign most services random ips [00:20:17] err [00:20:18] ports [00:20:41] I know, but there are usual ports to set them to. :-) Nevermind, just tell me the ones you guys use so I can close the ticket. [00:20:53] no clue [00:21:01] check the config of labstore1/2 ;) [00:21:03] it's in puppet [00:21:34] I see 'em. [00:21:48] 32769-32767 [00:21:51] tsk, tsk. [00:22:41] I took this from ubuntu docs, btw [00:22:51] to make sure I was doing it in ubuntu's standard way [00:22:58] there are no "correct" standard set of ports for this [00:23:33] No, but whichever you choose should be >=49152 [00:24:03] Well, in theory anyways. :-) [00:24:23] why? [00:24:31] to be outside of well known ports? [00:25:16] I don't think I've seen any nfs docs that use ports that high :) [00:29:50] Heh. RFC 6056 commands; though I agree I've yet to see anything besides X and IRC obey. :-) [00:30:09] (They used to be much higher than 6k once upon a time) [00:31:05] heh [00:31:17] for the most part well known ports is ignored :) [00:40:28] !log rebooting labstore3 -> silly lockd module [00:40:28] rebooting is not a valid project. [00:45:24] :D [00:53:41] Ryan_Lane: Almost, but not quite. Apparently, even the normally fixed mountd port is changed, and that's not in puppet. [00:53:51] o.O [00:54:01] I think it's gluster doing it, isn't it? [00:54:01] that's due to gluster, then [00:54:17] * Coren tries to find where /that/ is configured [00:54:25] heh [00:54:36] well, ideally we'll set it to that now and change it to the default [00:57:12] Oh, man, you guys keep reusing port 32768 for all sorts of different things! [00:57:54] what other things are we using it for? [00:58:06] nginx, for one [01:02:28] Actually, now that I look at gluster, it uses /its/ defaults, which aren't the same ones you use with your normal nfs servers either. :-) [01:02:43] 100005 3 tcp 38465 mountd [01:02:43] 100005 1 tcp 38466 mountd [01:02:43] 100003 3 tcp 38467 nfs [01:02:43] 100024 1 tcp 55659 status [01:02:43] 100024 1 udp 44153 status [01:02:43] 100021 4 tcp 38468 nlockmgr [01:02:43] 100021 1 udp 750 nlockmgr [01:03:06] nginx uses it? for what? [01:03:22] for sending logs maybe? [01:03:49] Oh, nevermind. I misread: [01:03:50] $nginx_worker_connections = '32768' [01:04:06] 32k connections, not port 32k. :-) [01:17:11] Change on 12mediawiki a page Wikimedia Labs/Account creation improvement project was modified, changed by Ryan lane link https://www.mediawiki.org/w/index.php?diff=672841 edit summary: [+126] /* OpenID as a provider */ [01:18:24] Odd. [01:18:49] Change on 12mediawiki a page Wikimedia Labs/Account creation improvement project was modified, changed by Ryan lane link https://www.mediawiki.org/w/index.php?diff=672842 edit summary: [+60] /* OpenID as a provider */ [01:23:29] * Coren tries to understand why the [bleep] tools-login still tries to mount via port 2049 [01:23:46] Hm. Portmapping cache? [01:35:47] maybe? [01:36:16] where are you trying to mount it? [01:36:37] when you want to override /data/project in your project you can use local autofs overrides [01:36:48] in auto.master/auto.data [01:37:43] we'll need that to be puppetized [01:38:11] then we can do one project at a time by modifying instances' puppet config [01:43:23] Yeah, right now I'm just testing through /mnt [01:43:32] And... I haz a sukses! [01:43:48] labstore3.pmtpa.wmnet:/srv/tools 37T 33M 37T 1% /mnt [01:45:24] Heh. I win. [01:45:31] projectstorage.pmtpa.wmnet:/tools-project 38083885056 10103715072 27980169984 27% /data/project [01:45:31] labstore3.pmtpa.wmnet:/srv/tools 39056055296 33792 39056021504 1% /mnt [01:54:09] how does it have more space than gluster? [02:00:32] Coren: you may want to change the cluster name for this [02:00:36] for labstore3/4 [02:00:51] maybe labsnfs for the cluster, and a new multicast group for it? [02:01:02] it's in the gluster's ganglia cluser now [02:01:06] *cluster [02:01:13] Ah, good point. [02:01:14] err [02:01:22] it's in gluster's ganglia cluster now [02:01:34] * Coren tries to find that. [02:01:44] heh [02:01:58] decent amount of system cpu use: https://ganglia.wikimedia.org/latest/graph_all_periods.php?h=labstore3.pmtpa.wmnet&m=cpu_report&r=hour&s=by%20name&hc=4&mc=2&st=1365818411&g=cpu_report&z=large&c=Glusterfs%20cluster%20pmtpa [02:02:10] Different layout, remember? I don't have an OS disk on the shelves. [02:02:31] It's using CPU because it's rebuilding the raid now. [02:02:35] ahhhh. ok [02:07:16] * Coren starts setting up the timetravel now. [02:07:30] ok. off for the night [02:07:31] * Ryan_Lane waves [02:09:37] * Coren waves. [02:52:02] Coren- ... So if something on tools edits while logged out, it will be coming from an IP address like '10.64.0.126'? [02:57:31] anomie: yes [02:58:06] legoktm- Any idea what the full possible range is? [02:58:26] not sure, all ive seen is https://www.wikidata.org/wiki/Special:Contributions/10.64.0.127 [03:14:16] Ryan_Lane: can has transwiki? or Coren ? [03:14:45] .. transwiki? [03:14:48] 12 18:26:05 < jeremyb_> want to transwiki https://en.wikipedia.org/wiki/Module:String ? [03:15:05] To wikitech? [03:15:10] yes please :) [03:15:18] * Coren tries to figure out the right userright. [03:15:25] sysop should be plenty [03:15:32] [[special:import]] [03:15:38] idk if it has import sources set up [03:16:30] Wha' be yer wikitech username again? [03:17:05] Nevermind, found you. :-) [03:17:22] You should have import now. Dunno about sources. [03:17:25] hah [03:17:36] i thought you'd just do it for me [03:17:38] * jeremyb_ tests [03:17:59] There's a happy fun userright just for that jazz on wikitech: contentadmin. [03:18:09] orly [03:18:43] gah, /me stabs the Echo that don't work with monobook [03:19:01] ~= enwp admin; doesn't have openstack magic or editinterface. [03:19:24] enwiki admin does have editinterface though [03:19:27] afaik [03:19:33] Which is why ~= and not == :-) [03:22:06] heh [03:22:28] ohhh. maybe transwiki can't do contenthandler? [03:23:48] huh. but it saved... [03:23:52] let's see what happens [03:23:55] > Import failed: Can't save non-default content model with $wgContentHandlerUseDB disabled: model is wikitext , default for Module:String/enwiki/Module:String is Scribunto [03:24:18] (that was on mediawikiwiki. which is the only source enabled on wikitech. now on to wikitech) [03:24:48] huh, wikitech did it without error [03:25:13] but probably MW version is wikitech < enwiki < mediawikiwiki [03:26:10] wooot, and it works [03:34:51] hrmmmmm. can't quite figure out how to use replace [03:35:05] anomie seems not too idle [03:35:55] jeremyb_- ? [03:36:19] anomie: how do i use replace from [[Module:String]] ? [03:36:56] anomie: https://wikitech.wikimedia.org/w/index.php?title=Sandbox&oldid=66494 [03:37:41] hrmmm, maybe i need source= instead of s=. but i've no idea why [03:37:56] examples would be lovely :) [03:38:09] Two errors there: the parameter is named "source", not "s". And lua uses % rather than \ for its patterns, so it should be "pattern=%d" not "pattern=\d" [03:38:18] ewwww [03:38:29] Manual: https://www.mediawiki.org/wiki/Extension:Scribunto/Lua_reference_manual [03:38:45] wooo, that looks maybe helpful [03:39:09] i think replace is missing from that last link [03:39:40] or there's multiple things named string [03:40:42] Coren: This is odd. It looks like some packages are only installed on tools-exec-01 and not tools-exec-02. For example, /usr/share/perl5/JSON.pm seems to only be on -01. [03:41:05] That's annoyingly possible. [03:41:06] jeremyb_- Module:String is not the same thing as the string library in Lua. [03:41:13] anomie: The first few installs were manual [03:41:19] * Coren fixes that [03:42:44] woooo, i think that worked. https://wikitech.wikimedia.org/w/index.php?title=Template:InstanceStatus&diff=prev&oldid=66495 [03:43:31] anomie: fix't [03:44:49] anomie: can i make it return the string unchanged if there's no match? [03:45:08] jeremyb_- That should be possible [04:07:01] Coren- I made a table of which packages are installed on one or two but not all of tools-login, tools-exec-01, and tools-exec-02. http://tools.wmflabs.org/anomiebot/packages.html [04:18:00] anomie: You do hard work. :-) The discrepancy between -login and the run environment is normal though. [04:18:14] I tried to create account at https://wikitech.wikimedia.org/w/index.php?title=Special:UserLogin&action=submitlogin&type=signup&returnto=Help%253AGetting+Started but got LOGIN ERROR: There was either an authentication database error or you are not allowed to update your external account. OKAY, a note at page bottom says, 'Note: The error message "There was either an authentication database error or you are not allowed to update y [04:18:14] our external account." generally indicates an invalid shell account name was used (see bug 16524).' SO, are 'roger' 'rogerhc' and 'rogervoy' all invalid shell account names? Where did I go wrong? Thanks! [04:38:46] roger_: Give me a second while I check. [04:40:32] Hi Coren, I have just now also posted this question at http://www.mediawiki.org/wiki/Talk:Wikimedia_Labs#Create_account_26228 but I will wait here to see what you can figure out... [04:42:22] I do not have shell access nor obviously any shell account name. Is that a prerequisite? [04:43:42] No, it works the other way around; you pick the shell account name when you create your wikitech account. [04:45:03] Okay, that's what I was trying to do. I'll wait here to see what you can figure out. [04:52:17] As far as I can tell, there's nothing wrong with any of those account names; I'm afraid I'm not yet familiar enough with OpenStack to figure out what breaks in your case. I'll tell Ryan, who is the guru. Sorry about the trouble. [04:57:15] Coren: Thanks. If convenient, Ryan can email me at rogerchrisman@gmail.com or reply at http://www.mediawiki.org/wiki/Talk:Wikimedia_Labs#Create_account_26228 [04:57:42] He's most likely to contact you through your userpage. [05:03:04] Coren: there's the spam blacklist? or abuse filter or something? [05:04:44] https://wikitech.wikimedia.org/wiki/MediaWiki:Titleblacklist [05:05:12] roger_: which is your first choice of those names? [05:05:17] Yeah, not applicable. [05:05:58] jeremyb_: rogerhc is my first choice [05:06:15] * Coren needs sleep. [05:06:17] * Coren waves [05:06:21] bye :) [05:06:29] i do too though! [05:06:48] wtf, i was granted skipcaptcha i thought? [05:07:28] roger_: and wiki username? also rogerhc ? [05:07:41] yes [05:08:28] Coren|Sleep: could it be opendj is still broke? [05:08:44] Ryan_Lane: ^ [05:09:03] i see no creations in ~11 hours [05:11:29] roger_: good luck [05:11:31] * jeremyb_ sleeps [06:57:41] @seen wm-bot [06:57:41] liangent: I am right here [06:58:11] petan: why doesn't it work in #mwbot ? [07:02:07] @seenrx [Bb]awolff.* [07:02:07] liangent: Last time I saw bawolff they were quitting the network with reason: Quit: g2g at 4/10/2013 11:46:07 PM (2.07:15:59.4378260 ago) (multiple results were found: bawolff_away, bawolff_) [07:20:41] liangent try @seen-on [07:50:34] petan: ah and it seems working in privmsgs too [07:50:59] rx doesnt [07:51:04] just seen works [07:51:10] to prevent some regex ddos :D [08:08:51] petan: you already have some timeouts for regexes right? [08:09:31] yes [08:09:40] 1 second or that [08:09:58] they are being processed in separate thread anyway [08:09:58] but it still eats lot of cpu [08:11:16] anyway someone can invite wm-bot to their own channel, then perform dos there, if it's doable in privmsgs [08:27:38] @notifyrx [08:43:53] liangent ok but that someone would need to have a wikimedia cloak [09:42:46] Hi, everyone [09:43:13] I'm moving my bot to wikimedia labs. [09:44:58] From the documentation, I should place the code at /data/project/"mybotname", but when I mkdir that directory, it shows "Permission denied". What should I do? [09:47:22] hey! [09:47:43] I think it automatically makes the home directories on the hour, so you might need to wait a bit longer [09:48:24] Okay. Thanks for your reply :) [10:24:08] Nullzero no it doesn t [10:24:15] Nullzero make it in userdata folder [10:24:18] !botsdocs [10:24:18] https://wikitech.wikimedia.org/wiki/Nova_Resource:Bots/Documentation [10:24:55] updated [10:25:21] oh? [10:25:35] you need to use /data/project/userdata [10:26:09] that´s a+rwx [10:35:55] Then, what about creatting my folder in public_html [10:45:06] Webserver is availible at /data/project/public_html/username, the path can't change to anywhere else. It's essential to have my folder there. [12:00:35] Nullzero yes I believe these folders are created automatically if not... I can create one for you [12:04:41] Well, then, please do so. Thank you :) [12:12:38] Nullzero done [12:13:47] And for bot, it should be in userdata, right? [12:17:27] your bot can be either in your home or in userdata [12:17:34] that is up to you... [12:17:59] if you want to make it public and accessbile to others then you should use userdata [12:18:32] you can also use tools project as bots project will become a staging area for tools in future [12:18:46] basically, it is a good idea to have accounts on both projects [12:19:16] Got it now. Thanks a lot ;) [12:31:07] [bz] (NEW - created by: Chris McMahon, priority: High - normal) [Bug 46166] Add automated browser tests to beta labs (tracking) - https://bugzilla.wikimedia.org/show_bug.cgi?id=46166 [12:36:14] [bz] (NEW - created by: Željko Filipin, priority: Unprioritized - normal) [Bug 47194] GettingStarted extension not at en.wikipedia.beta.wmflabs.org - https://bugzilla.wikimedia.org/show_bug.cgi?id=47194 [17:22:24] [bz] (NEW - created by: Željko Filipin, priority: Unprioritized - normal) [Bug 47203] Math extension broken at en.wikipedia.beta.wmflabs.org - https://bugzilla.wikimedia.org/show_bug.cgi?id=47203 [17:32:43] [bz] (NEW - created by: Željko Filipin, priority: Unprioritized - normal) [Bug 47205] Sandbox not at en.wikipedia.beta.wmflabs.org - https://bugzilla.wikimedia.org/show_bug.cgi?id=47205 [17:32:44] [bz] (NEW - created by: Željko Filipin, priority: Unprioritized - normal) [Bug 47203] Math extension broken at beta.wmflabs.org - https://bugzilla.wikimedia.org/show_bug.cgi?id=47203 [17:53:25] hmm [17:53:28] I can't create an account [17:53:31] on labs / wikitech [17:53:32] help? [17:56:50] anyone? [17:56:53] I'm getting 'There was either an authentication database error or you are not allowed to update your external account.' [17:57:28] for shell name suchabot [18:07:02] Coren: petan ^ ? [19:00:06] YuviPanda: There's a bug atm in openstack that will probably need Ryan to fix. But a better question is why such an account name? Don't you already have an account? [19:00:23] Coren: mediawiki.org/wiki/User:Yuvipanda/G2G [19:00:30] Coren: I'm working on a GitHub Sync bot [19:00:40] I already have github.com/SuchABot [19:01:02] You /really/ don't want a new wikitech account for this! [19:01:30] okay [19:01:32] what do I want? [19:01:37] That's what tool accounts are for. :-) [19:01:49] hmm, that just whooshed me by :) [19:01:51] explain? [19:02:00] I have not been reading your emails to wikitech-l :( [19:02:05] should I read them now? [19:02:05] !toolsdocs [19:02:05] http://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/Help [19:02:17] Coren: it needs to push to Gerrit, btw. [19:02:21] * YuviPanda reads [19:02:27] Check the "tool account" section. [19:02:36] reading [19:03:27] Coren: sure, but it is a Gerrit bot and needs a Gerrit account [19:03:33] Coren: that page tells me nothing of Gerrit [19:03:49] this is not a wiki bot... [19:04:01] Gerrit credentials, otoh, are a different matter. Your use case is attpical enough (there was no provision for bots pushing to gerrit) that it'll need special magic. [19:04:26] It's the same idea, you have a tool that needs to be maintainable by more than one person. [19:05:09] Coren: sure, I'll move it to labs when it's doing things. But even then it'll need a Gerrit account (+ private keys) [19:05:12] A shared shell account is a big no-no for a number of reasons. That said, we can do it too. :-) [19:05:49] Coren: I'm writing / testing it now, and am forced to test it via my Gerrit account [19:05:53] resulting in things like https://gerrit.wikimedia.org/r/#/c/59058/ [19:05:58] nogood! [19:06:31] Coren: and why was I getting an error for that shell name? [19:06:34] bot is blacklisted? [19:06:51] Yes, the gerrit account will have to be dealt with, but Monday. We need to figure out the right way to do it without having a big hole like a shared wikitech account. [19:07:18] so for now I'll have to do it on mine? [19:07:24] Not as far as I know, there's a problem with openstack right now that fails every name. Unrelated bug. [19:07:29] oh [19:07:36] so nobody can create a gerrit account now? [19:07:37] o_O [19:07:38] O_o [19:08:30] Yeah, major boo boo, but beyond my current meager openstack extension knowledge to fix. [19:08:35] :'( [19:08:36] ok [19:08:40] * Coren is waiting for Ryan. [19:08:43] how long has this been the case, btw? [19:09:05] Last night (my time) is the first I heard of it. [19:09:24] ah [19:09:25] ok [19:09:33] so, for now I"ll just use my Gerrit account itself [19:09:38] and figure this one out later [19:13:12] And, to make matters more fun, I've been banging my head against a problem that, it turns out, cannot be solved. *grumble* [19:16:00] Coren: a problem that cannot be solved by Coren the Amazing? [19:16:02] *GASP* [19:16:13] are you trying to be a travelling salesperson? [19:16:21] Hardware issue. :-) [19:16:25] ah [19:34:40] andrewbogott_afk: I think the uid/gid generator is broken in OSM :( [19:38:08] Ryan_Lane: 'morning. [19:38:16] howdy [19:38:42] Ryan_Lane: productive :) [19:38:57] Ryan_Lane: Except for Gerrit account creation being broken :) [19:39:05] yeah. looking into this [19:39:08] has to do with a code change [19:39:08] Ryan_Lane: wheee :) [19:39:38] I've a somewhat working GitHub -> Gerrit thing that also handles people pushing new commits to the Pull Request [19:39:44] cool [19:39:51] be back in a few. picking up food [19:39:54] :D [20:00:08] ah ha [20:00:10] search limit [20:00:14] it wasn't applied on virt0 [20:00:26] fixed [20:00:42] A semirelated problem of YuviPanda. [20:01:20] He needs gerrit credentials for his tool. I felt that a new wikitech account was... Nit the right way to go about it. [20:01:44] that's the only way to do it [20:01:45] (unmaintainable without shared creds when maintainers change, etc) [20:02:00] Is it? Eew! [20:02:03] how else? [20:02:09] gerrit uses ldap auth [20:02:28] and we have a shared system [20:02:34] how is it unmaintainable? [20:02:58] and yeah, it needs shared creds [20:03:03] Well, it means you're stuck sharing password for one, and won't really work with two factor. [20:03:04] so do bot accounts on wikipedia ;) [20:03:16] we don't expect everyone to use two factor [20:03:21] only people who want to and admins [20:03:30] admins are required [20:03:38] Hm. Still uglyhackyuk territory. [20:04:39] But unless you have a brilliant idea for gerrit-only accounts, we be stuck with it. [20:05:10] And yeah, bot accounts on wikipedia are also teh suck. [20:05:24] For much the same reasons, really. [20:07:07] this is a common authentication problem [20:07:14] and there's not a really great solution [20:07:48] oauth usually helps, but you also don't want things tied to a real user either [20:08:28] * YuviPanda looks [20:08:55] Ryan_Lane: Coren so, I should just register the Gerrit account? [20:09:00] once it is back, that is [20:09:20] Yeah, it's ugly but it'll work. [20:09:46] * Ryan_Lane doesn't think it's ugly [20:10:00] But please write the tool with a real tool account and from your own rather than actually use that one for anything but the gerrit creds. [20:10:07] it would be nice if the auth had roles and we could limit its access [20:10:12] whee created! [20:10:36] Coren: sure, I don't expect to login to that at all [20:10:44] if we could disable it we should [20:11:02] you need to occasionally [20:11:11] to change ssh keys or change the email address [20:11:12] etc [20:12:03] yeah, monthly ssh keys cycling or something [20:12:27] no need for that :D [20:12:31] only if the key is compromised [20:12:58] Coren: this tool requires a webserver, so I think I'll get it's own instance / project? [20:13:04] whee! https://gerrit.wikimedia.org/r/#/c/59058/ :) [20:13:15] hmm, ANSI color codes [20:13:16] * YuviPanda fixes [20:13:30] Ryan_Lane: should this go into a new project? or is there a project for Gerrit stuff already? [20:13:43] eh? [20:13:54] oh you mean in gerrit [20:13:54] no ide [20:13:54] *idea [20:13:54] oh no [20:13:55] not gerrit [20:13:57] I mean in Labs [20:14:18] it can probably exist in tool labs [20:14:23] this requires a web server so that GitHub can notify me whenever a new pull request is there. [20:14:29] * YuviPanda reads Tool labs things again [20:16:52] Coren: what is 'A web URI mapped to its ~/public_html: http://tools.wmflabs.org/toolname ' running? [20:16:54] apache? [20:17:03] oh [20:17:04] read [20:17:05] nvm [20:17:07] * YuviPanda reas [20:18:52] Coren: this needs a python webserver. Will I *have* to use CGI? [20:28:09] Require a python webserver? You mean you have a python app that listens to port 80? [20:28:48] Coren: WSGI? [20:31:38] Coren: a python webapp, that is [20:33:05] Huh. Interesting. Well, I haven't yet looked into that use case since you're the first, but there's no reason we couldn't support WSGI. Not sure if I can use mod_wsgi with the current setup, but worst case is the CGI wrapper middleware. Either way, doesn't change your application. [20:33:40] Coren: hmm, mod_wsgi would be nice, I guess. But I don't know how easy it is to setup in a way that works the same way as PHP [20:33:46] so you can just pop a file in... [20:34:14] Coren: so, i can haz tools lab account? [20:34:37] Coren: shell is yuvipanda [20:34:39] Yep. What name do you want for it? [20:34:42] wiki Yuvipanda [20:34:47] Coren: suchabot? [20:35:02] Coren: or would that be too confusing? [20:35:12] since there's also going to be a ldap user called suchabot? [20:35:18] Isn't that the name you gave the gerrit account? i'm thinking that'd be confusing [20:35:28] yeah [20:35:32] suchabot-serveR? [20:35:34] err [20:35:36] something like that? [20:35:47] Suchaserver? [20:35:48] sure [20:36:11] Gimme a min. [20:36:15] whee [20:39:22] YuviPanda: What's your wikilabs user? [20:39:33] Coren: Yuvipanda [20:40:24] YuviPanda: All done. [20:41:32] whee Coren :) [20:42:40] Coren [20:42:51] Yes? [20:44:30] When execute the bot, in error.log appears outo of memory [20:44:42] *out [20:46:20] The default allowance is fairly conservative. You might want to try it with -mem 1g at first, and use qstat to see what its actual requirement is. [20:47:18] What language is it written in? [20:47:35] perl [20:47:52] It works [20:48:17] Perl is relatively frugal, so unless you're working with huge datasets, you shouldn't be too far off. [20:48:38] Use qstat -j job_number [20:48:48] Look at the actual vmem used near the bottom [20:49:58] Coren: just to confirm, I can put a private key in the toolslab account, right? [20:50:18] YuviPanda: The only roots have an nda [20:50:26] thank you :) [20:50:36] 531.020M [20:52:02] UA31_: Two things, though: you should use your tool account to run the bot once you're done testing, and you probably want to use jstart if your bot is meant to run at all times (jsub will not, by default, restart it if it stops) [20:53:33] UA31_: Which means you can probably use -mem 600m or so. You might want to wait and let it run a bit to see if it grows before you reduce its allocation. [20:53:44] was there an open bug on not being able to create accounts? [20:54:18] Ryan_Lane: Not sure, I didn't see one filed and you got here before I did. [20:54:26] * Ryan_Lane nods [22:04:54] well, the opendj upgrade is helping. no more memory leak on either one, they are using less than 1G of memory each [22:05:00] they are also hardly using the CPU [22:05:19] and the search calls are 4ms faster [22:06:17] once we have NFS rather than gluster login time should be really fast (no need to wait for the damn gluster mount) [22:18:55] Coren: hmm, so I can hold off with just PHP [22:18:56] for now [22:19:16] Coren: but maybe I should write it in PYthon + CGI simply to give me a reason to nag you about WSGI :) [22:19:49] Coren: all it needs to do is to receive a POST request and submit a job to the grid [22:20:05] Coren: I'm going to assume that someone will kick me if I receive a POST request and simply shell out from that itself :) [22:20:38] YuviPanda: Perhaps not a kick; sometimes the judicious application of clue-by-four to the lower abdomen is more indicated. [22:21:01] I'm never really sure if lower abdomen is an euphemism or not :P [22:21:09] but no, it'll also make me feel dirty... so :P [22:21:35] Coren: the CGI will be run as my tool user or as www-data? [22:21:41] As your tool user. [22:21:44] ah, nice [22:21:57] Coren: and I need to shell out to start jobs? [22:22:47] YuviPanda: Yes, although it would be really nice if there was a PHP module equivalent to jsub/job/jstop [22:23:07] Coren: one of the things I *hated* about the toolserver was how hard it was to run non-PHP web stuff [22:23:22] I really hope that isn't going to end up being the case with ToolLabs [22:23:41] YuviPanda: It shouldn't be. If it can run as a CGI, it'll Just Work(tm) [22:23:56] but, CGI feels a bit icky to me, for some reason :) [22:24:32] It's a bit old and clunky, but it provides for process separation. [22:24:39] Which is a Good Thing(tm) [22:24:48] hmm, true [22:24:54] mod_wsgi runs in process? [22:25:22] Coren: If we're going to do Python (which we should, really!) we should ideally do nginx / uwsgi or something like that, rather than apache... [22:27:10] You and me got a different operative definition of "ideal", it seems. :-) More likely with wsgiref.handlers [22:28:02] With some magic so that the WSGI server can actually run on the compute nodes as resources allow. [22:28:17] PHP favoritism! [22:30:12] No, PHP gets the same treatment basically. [22:30:43] php is pretty insecure when run as cgi [22:31:01] lots of things that wouldn't normally be vulnerabilities are [22:31:17] Depends how; in this particular case, suphp does the job quite well. [22:31:34] it's specific to being run as cgi [22:31:59] there was a really horribly nasty one that preilly showed us [22:32:04] now to remember what it was [22:32:18] I'd appreciate if you could so I could see if it's applicable here. :-) [22:53:31] root@labstore3:/srv# ls -l /srv [22:53:31] total 8 [22:53:31] -rw-r--r-- 1 root root 29 Apr 13 22:40 file_one [22:53:31] -rw-r--r-- 1 root root 29 Apr 13 22:53 two [22:53:31] root@labstore3:/srv# ls -l /time/2013-04-13-22-50 [22:53:32] total 4 [22:53:32] -rw-r--r-- 1 root root 29 Apr 13 22:40 file_one [22:53:53] Ryan_Lane: ^^ [22:54:57] ohhhhhh, time machine????!??!?!? [22:55:14] well, when we have unicorns... [22:55:28] i have a unicorn [22:55:38] Coren: ? [22:56:10] Time travel snapshots, yeah. [22:56:15] cool [22:56:18] Coren: http://boingboing.net/2013/04/12/iranian-scientist-invents-mach.html [22:57:19] Coren: \o/ [22:57:51] Ryan_Lane: On a related note, that gives me thinly provisioned filesystems for "free". Do we want to do that instead of one big one? [22:58:12] for multi-tenancy? [22:58:32] quotas? [22:58:35] Yeah; although that'd cost performance. [22:58:37] oh. we're going to need quota support [22:58:42] one big filesystem is better [22:58:48] * Coren nods. [22:58:50] we can do quotas using xfs quotas [22:58:54] those are per directory [22:59:04] Works for me. [22:59:05] orly. good to know [22:59:18] they aren't reported through nfs, though [22:59:34] that will be fun [23:00:05] Ryan_Lane: me. ENOSPC is an unambigous report. :-) [23:00:10] heh [23:00:44] hm. maybe it does work [23:00:49] * Coren tries to figure out a nice way to export filesystems. [23:01:07] historically we've been using daemons that read from ldap [23:01:13] written in python [23:01:43] No, I mean export the timetravel filesystems. I'd like to avoid one export per snapshot, gets unwieldy fast. [23:01:46] ah [23:01:51] indeed [23:02:05] Hm. Perhaps .snapshot a la netapp is feasible with xmounds. [23:02:08] xmounts* [23:04:25] Yeah, I can do it with bind mounts and allowing crossmounting with NFS. [23:05:11] will nfs cross mount boundaries? [23:05:30] If you tell it to with nohide [23:05:43] ah. right [23:06:28] the mtab on that box is going to be fugly though. :-) [23:07:38] * Coren tries it. [23:08:18] use autofs on the server to mount it as its accessed? :) [23:08:34] * Coren ponders. [23:08:38] not sure if that'll work [23:08:43] Can autofs do bind mounts? [23:08:44] but I'd imagine it would [23:08:45] yes [23:09:01] That /should/ work then. [23:09:21] Much better, too, since .snapshot mounts should be relatively rare. [23:09:25] yep [23:09:51] * Coren feels like he's reimplementing a netapp toaster. :-) [23:10:20] basically am [23:10:23] *are [23:34:02] Ryan_Lane: It works, but you don't get to see what snapshots are available. :-( [23:34:40] Manually managed binds it is then.