[08:16:15] !ping [08:16:15] pong [09:01:13] hi! Can I use Labs to create/run Wikepedia bots, or to create http wiki tools ? [09:01:18] yes [09:02:13] nice) [09:02:21] !bots [09:02:22] http://www.mediawiki.org/wiki/Wikimedia_Labs/Create_a_bot_running_infrastructure proposal for bots [09:02:33] !project bots [09:02:33] https://labsconsole.wikimedia.org/wiki/Nova_Resource:bots [09:02:46] thanks [12:36:38] !log wikidata-dev wikidata-dev-3: installed SpamBlacklist extension [12:38:34] !log wikidata-dev wikidata-dev-3: installed SpamBlacklist extension [12:41:56] Why does the above not log any more? :´( [12:48:33] log bot is down... [12:48:57] tbh it's not worth of restart, it will crash anyway in 5 min [12:53:07] ok [13:17:43] Damianz: hi, you want to fix the logbot :)0 [13:18:02] !log is I am broken, and I can't log anything, please fix me [13:18:09] @infobot-ignore- log [13:18:09] Item log was removed from list [13:18:12] !log is I am broken, and I can't log anything, please fix me [13:18:12] Key was added [13:18:15] !log blah [13:18:16] I am broken, and I can't log anything, please fix me [13:18:22] :D [15:02:25] Hi! [15:03:39] hi Silke_WMDE! [15:06:00] A question concerning Apache rewrite rules: For Wikidata we don't have a whole farm on our test system, but one repo installation and one client installation. We would now like to pretend we had separate wikidata instances for the languages. For the rewrite rules, I'm wondering if I can define in apache "these following things are language codes". This could be macros, variables or something like that. Has anybody used anything [15:06:00] similar and could point me to documentation or give me an example how you did it??? [15:09:04] My mind just exploded [15:13:34] Hello all! [15:13:38] hi dschwen [15:13:56] "Please ask a Nova administrator to create credentials for you." [15:14:01] who would that be? [15:14:03] :-) [15:14:25] I cannot sudo in my instance (wikiminiatlas) [15:14:42] the reason might be that I have no clue what my password is [15:14:56] since all outher authentication is key based [15:15:33] Silke_WMDE: you might want to write labs-l [15:15:48] dschwen: wow, there is really 0 docs on labsconsole about that, huh [15:15:59] dschwen: It's what you login to labsconsole with [15:16:06] do I detect a hint of sarcasm? ;-) [15:16:09] and yeah, we really need to fix docs because they're crap [15:16:20] sumanah: Ok, thx! [15:16:37] Especially now account != shell access, which people apparently find confusing heh [15:16:39] Damianz: how would I find out who the Nova administrators are, via labsconsole? [15:17:11] cloudadmin group [15:17:13] though [15:17:16] you shouldn't see that [15:17:26] keystone is using the ldap backend so if you can login you have credentials [15:17:38] might be the stupid stupid but that drops your domain though [15:18:02] Damianz: so, what would we advise Labs users to do, to figure out who Nova admins are and contact them? [15:19:13] dschwen: right now Labs is highly unstable and only suitable for use by people who don't mind the ground shifting under them every 2 weeks :) [15:19:54] 1sec I'm looking at my dev version to see where that's actually thrown [15:19:57] sumnah, yeah, I know [15:20:22] but since the death of the toolserver was basically just announced I'd rather get myself familiar with labs sooner than later [15:20:25] btw dschwen what IRC client do you use? [15:20:31] xchat [15:20:33] why? [15:20:35] Toolserver is shite [15:20:43] do you know about tab-completion, dschwen? in IRC? [15:20:45] 09/25/2012 - 15:20:45 - User dschwen may have been modified in LDAP or locally, updating key in project(s): bastion,maps [15:20:47] http was down for like 10hours the other day [15:20:56] 09/25/2012 - 15:20:56 - Updating keys for dschwen at /export/keys/dschwen [15:20:58] toolserver works for my projects NOW, labs does not work for my projects NOW [15:21:04] that is all i care about NOW [15:21:20] bashing the services of the TS does not help anyone [15:21:34] The only thing labs doesn't work for me currently is sql replicas, the biggest outage to the bots I run is the TS going down heh [15:22:02] ^ regarding toolserver death/blah, please can I get a project created for ACC? I'd like to start thinking about migration at some point in the future. [15:22:02] I'm with dschwen - bashing what TS provides a lot of users isn't helping anything [15:22:04] I need SQL replicals for almest every project I run [15:22:51] I am looking forward to labs. The infrastructure sounds impressive. Unfortunately it is not ready for me. [15:23:38] ok wtf [15:23:40] * Damianz stab [15:23:52] dschwen: If you logout and back in does it work? [15:23:53] This question probably has been asked over and over again: [15:24:00] That error is when it can't get your ldap details -.- [15:24:08] I run 3 bots, and about 5 tools [15:24:29] should I get 8,6,2 or 1 instances? [15:24:51] 1 or use the 'shared' bots setup (which we need to work on a bit) [15:24:57] (one for bots, one for tools; one per tool, one per bot (a common bot instance) etc.) [15:25:04] We have 2 sql servers, 1 web and like 3 tools [15:25:09] Depends how much recourses you need really [15:25:13] sumanah: That's a stupid error message [15:25:15] Damianz: I can log in perfectly fine [15:25:36] I just sudo'ed after resetting my labsconsole password, which I ^H^H^H my browser forgot [15:25:43] Basically if they are not logged in or it can't get their ldap details (silly sessions bug) then that's the error... which is not really useful, it should redirect to login on not logged in. [15:26:01] Damianz: what specifically is the error message that is dumb? "please see a Nova admin"? [15:26:07] yeah [15:26:24] 'to create credentials' might really mean 'login' [15:26:44] I'll look at doing something more logical with this later hmm, or poke Ryan_Lane to see what he thinks/check I'm reading this right. [15:26:51] dschwen: I am (sort of) a nova admin… let me catch up on the backscroll and see what your problem is. [15:27:20] thanks andrewbogott, it seems my problem was only a perceived one [15:27:35] I'm happily updating and istalling packages on my instance now [15:27:39] Really? Are you able to sudo now? [15:27:46] andrewbogott: Do you think the current use of SpecialNova::noCredentials() is silly considering the context is ! $this->userLDAP->exists() or !$this->getUser()->isLoggedIn()? [15:28:26] I am, sudo works. I just did not know which password to use (and had forgotten my labsconsole password anyhow. just resetted it and it worked immediately) [15:28:38] Damianz: Seems like it should direct you to a login/create account page [15:28:42] yeah [15:28:57] dschwen: OK, great. [15:29:32] sumanah, yes, tabcompletion works fine, didn't know it existed in xchat. thanks! [15:29:39] dschwen: it's great! [15:29:42] Damianz, what action is landing you on that page? I don't see it much. [15:29:52] Interesting there is noCredentials and notLoggedIn yet the later is called for in some places. [15:30:11] "Please ask a Nova administrator to create credentials for you." [15:30:16] ^ I'm just reading the code atm. [15:30:59] Damianz, one sec [15:31:01] Considering keystone is ldap backed that message is bleh, it should always be re-auth [15:31:22] main page, manage projects [15:31:28] https://labsconsole.wikimedia.org/wiki/Special:NovaProject [15:32:24] https://labsconsole.wikimedia.org/wiki/Help:Access#Rights [15:32:26] See, that should say login required... but I think if the silly I'm going to drop your session key bug is in effect it sees you logged in then fails to get your details to auth against nova or such.. hmm [15:32:34] the link "Manage Sudo policies"! [15:32:50] I get "You must be logged in to perform this action. " So y'all must be falling into a boundary case. [15:32:57] cause, you know, of course I'm searching the help pages first [15:33:00] ;-) [15:33:22] We don't just have a manual to tell you to RTFM :) [15:33:39] Damianz: The "I'm going to drop your session key bug" is believed to be fixed… so if you can make it happen Ryan might be interested. [15:33:45] That or my knowledge is out of date. [15:33:48] Really? [15:33:54] I thought it was mostly fixed [15:34:07] Apparently if you login on another browser mw core can only store 1 token so drops the second machine [15:34:16] Or such.. I'll try when I go over to my desktop later. [15:37:26] ssmollett: I don't think I helped much on the bz ticket, his response makes me more confused than before and provides not much info. [15:42:38] some of my tools need webserver access (i.e. a permanent portforward to bastion, or a public IP (not necessary yet I guess))) [15:42:58] the docs just mention a portforward to my localhost [15:43:09] which of course only lets me access the tool [15:43:18] is there a policy on how to do this? [15:43:47] For bots we have http://bots.wmflabs.org/~ accessible in /mnt/public_html/, or you can request a public ip [15:44:01] oh, neat [15:44:12] how do I get on the bots project? [15:45:44] 09/25/2012 - 15:45:44 - User tanvir may have been modified in LDAP or locally, updating key in project(s): bots,bastion [15:45:49] ... [15:45:55] 09/25/2012 - 15:45:55 - Updating keys for tanvir at /export/keys/tanvir [15:46:00] * Damianz hopes tanvir is dschwen's shell name [15:46:10] * Tanvir is Tanvir. [15:46:14] nope, not that I know [15:46:19] oh [15:46:20] hi Tanvir [15:46:23] I was under the impression it was dschwen [15:46:26] Hello Damianz! [15:46:27] * Damianz waits for the *next* cron run [15:46:37] dschwen: You might have access when labs-home-wm catches up [15:46:38] I am trying to login.. dunno what's happeing. [15:46:41] occam's razor, dude! [15:46:51] I did not login there for a long time. [15:47:14] labs-home-wm is a little buggy, going to DIAF soon though [15:47:49] Okay, I am logged in now! xD [15:48:06] well, great for you ;-) [15:48:07] Hi hashy [15:48:23] Can I access to project database from WMFLabs? [15:48:32] mysql? sure [15:48:36] Aye. [15:48:41] How can I do that? [15:48:57] what username do you want? [15:49:03] is tanvir asking about mysql, or the replicated project dbs? [15:49:15] In TS, I just type sql -r and I am not the respective DB. [15:49:58] If you're after the replicas of prod then we don't have them setup yet, if you just want a mysql db then we can sort that [15:50:05] * Damianz thinks we really need to backup these servers also [15:50:34] Damianz, the mysql db would be fine. [15:50:46] 09/25/2012 - 15:50:45 - Created a home directory for dschwen in project(s): bots [15:52:42] Tanvir: I just sent you some details via pm [15:52:47] bots.wmflabs.org/phpmyadmin exists if you require it [15:53:43] * Damianz needs to go make some phone calls, bbiab [15:53:51] * dschwen cannot log into bots yet. Waiting patiently... [15:55:43] 09/25/2012 - 15:55:43 - User dschwen may have been modified in LDAP or locally, updating key in project(s): bots [16:03:38] bots still denies my login attempts "ssh_exchange_identification: Connection closed by remote host" [16:14:36] https://labsconsole.wikimedia.org/wiki/Help:Access#Accessing_services_using_port_forwarding [16:14:42] this does not work for me [16:15:01] trying to connect to my instance from bastion already fails: [16:15:04] telnet 10.4.0.234 80 [16:15:04] Trying 10.4.0.234... [16:15:04] telnet: Unable to connect to remote host: Connection timed out [16:15:28] on my instance however I see the http server running and listening on 0.0.0.0:80 [16:15:35] (not just on localhost!) [16:16:09] is there a firewall setting I need to change? [16:17:06] haha [16:17:14] "Manage security groups" [16:17:39] dschwen: You can probably do a double ssh tunnel. I.e. ssh tunnel localhost:80 to bastion [16:17:40] there is the "No Nova credentials" thiong again [16:18:09] that is an ugly hack on top of an ugly hack ;-) [16:18:43] It was how I managed to get tile-mil tunneled out. But yes, it isn't directly elegant [16:19:25] actually with the ssh proxy command set up I can just tunnel straight to my instance [16:19:44] I was under the impression, you can only change the firewall settings on instance creation and I didn't want to restart everything again. Not sure if that is (still) true [16:22:15] anyhow, I changed the help page accordingly [16:22:50] Damianz: no luck with logging in to bots yet [16:36:59] i'd like to work on getting replication of prod db to labs, is that possible or has it to be done by a more priviledged person? [16:41:26] why can't I ssh to the toolserver from my labs instance? [16:47:42] What exactly are "Nova credentials" on Labs? [16:47:56] giftpflanze: Needs an ops person to do it as you need access to the prod cluster to setup replication, I believe it's in the pipeline and possible has some servers earmarked it just needs time. Larger priorities like killing nfs before it causes issues currently [16:48:30] dschwen: Security groups possibly [16:48:41] Avic: Nova is the openstack compute service that provides vms. [16:48:57] apmon: You can only change the groups that an instance belongs to on instance creation. I believe you can change the meaning of those groups at any time. [16:49:44] * Damianz hates muscles, stupid things have to hurt [16:49:50] Damianz: Ah. So, how does one get Nova credentials? do I need to have/be a member of a project, I assume? [16:49:52] giftpflanze: We're planning on having the replication db live on native hardware (vs. on a VM like most of labs). So doing that most likely requires root on real hardware. [16:50:14] giftpflanze: Nevertheless, we might be able to make use of your assistance :) Ping Ryan_Lane when he shows up. [16:50:19] Avic: Well it's sorta complicated, because it's not really a credential (it uses ldap auth like the wiki). Either login or logout and login and it should be ok [16:50:19] OK, but that still required foresight to create a security group in the first place which I didn't... [16:50:20] and/or send an email to the labs list. [16:50:38] but for the future that is good to know [16:51:06] andrewbogott: We really need real hardware for user dbs too :P Meh I'd love to get a proxy setup for domain -> internal ip in nova as well... too much fun to have [16:51:18] apmon: Yes, the current setup has poor defaults. I've tried to document it, but documentation is a poor substitute for proper default behavior. [16:51:57] Damianz: loging out/back in fixed it. Weird. Thanks for that tidbit, though! [16:51:58] Damianz: Yep, the IP proxy thing is on our list; that's going to be a topic at the OpenStack conference next month I believe. [16:52:13] Avic: Do you use multiple browsers by any chance? [16:52:20] There's an annoying bug in core that causes us issues [16:52:22] Yep [16:52:40] I go between Opera/Chrome a lot [16:52:43] Basically if you login to another you have to logout and back in one the first because it can only store 1 token. [16:52:56] andrewbogott: :o [16:53:38] Damianz: It takes a LOOONG time for features to make it from OpenStack Design to labs, though, since there's a six month release cycle and then another six month adoption lag. [16:53:49] I saw the python implimentation of managing haproxy that looked cool, would be really awesome as a service managed though the api rather than hacked up with puppet/nginx. Then iterations of ssl support with dedicated ips etc would be nice :D [16:54:09] andrewbogott: It's ok, we're in beta [16:54:09] :P [16:54:25] apmon: are you a net admin on the maps project? [16:54:35] could you allow outgoing ssh connections, please? [16:54:39] I think I might be [16:54:53] 6month release cycle for something the size of openstack is rather quick imo [16:55:07] Considering the amount they could break every time, we just hope they're on the ball. [16:55:21] Damianz: no luck loging into bots. "channel 0: open failed: administratively prohibited: open failed" [16:55:44] Hmm [16:56:15] That's an intesting message, are you just trying to ssh in or access http or ? [16:56:24] ssh [16:56:33] with the proxy-command setup [16:56:47] Damianz: Well, historically, we break every single thing in every release. [16:56:54] Ah proxy explains that message, I swear I saw the bot make you a dir [16:56:55] So it's not /that/ impressive :( [16:56:58] oh, hold on! [16:57:01] andrewbogott: Oh there is that [16:57:10] But that's what minor releases are for! [16:57:18] dschwen@bastion1:~$ ssh bots -> Name or service not found [16:57:22] Bit like ubuntu, wait until x.x.03 and everything is fine [16:57:23] am I retarded?! [16:57:33] dschwen: Ah ok, it's bots-1 bots-2 bots-3, probably use 3 [16:57:40] aHA! [16:58:15] Tools labs hopefully will be easier if we actually impliment a schedular and puppetize the tools. [16:58:16] and bots-4 :p [16:58:33] oh lord, puppetize? [16:58:43] yeat, THAT will be a lot easier [16:59:20] giftpflanze: IT'S THE FUTURE [16:59:42] I can get on bots-1 [17:00:04] ah, -3 works too now [17:00:07] Yes, puppetization is I think (one of) the reason why the wikimedia-osm server isn't yet up and running [17:00:35] aude: btw, do you know what the status on that is? [17:00:36] Heh, beta is up - it's mostly not puppetized (getting there though!) [17:00:48] and again no outbound ssh from bots-3 either [17:01:33] What port? [17:01:38] I can get ssh on port 22 fine [17:01:38] 22 [17:01:44] try telneting it [17:01:46] ssh: connect to host login.toolserver.org port 22: Network is unreachable [17:01:49] might be some weird acl [17:02:01] telnet: Unable to connect to remote host: Network is unreachable [17:02:06] hi apmon [17:02:09] andrewbogott: Is the default security group "per project"? I.e. can I change the firewall settings for the default group? [17:02:22] I'm pretty sure it's per project. [17:02:27] * aude has been busy with wikidata but needs to spend more time on puppet [17:02:34] I bet that when you dig into the config page that will be obvious. [17:02:36] hmm [17:03:05] Weird [17:03:25] aude: I could probably get the puppet setup for mod_tile and renderd if that is a help. But couldn't quite figure out how to puppetize the postgresql / postgis setup [17:03:38] i was running into a problem running puppetd -tv [17:03:56] err: Could not retrieve catalog from remote server: getaddrinfo: Name or service not known [17:03:59] warning: Not using cache on failed catalog [17:04:00] Ah, I think I got further than that... ;-) [17:04:01] err: Could not retrieve catalog; skipping run [17:04:10] and then stuck :( [17:04:32] dunno [17:04:34] not sure if it's a problem with the instance or me doing something wrong? [17:04:39] Seems TS specific, I can login to other servers fine. [17:04:52] * Damianz thinks we should move apache st00f to project data soon [17:04:52] I did manage to install postgresql with puppet. But setting up the correct users and running the scripts to initialise the db and the likes wasn't ideponent [17:04:59] I sense a labs vs. toolserver rivalery ;-) [17:05:21] apmon: if i could get past the error, i can probably figure out something and get it into gerrit [17:05:26] That reminds me [17:05:32] petan: you around, by chance? [17:05:47] rivalery? use what works. toolserver 40% of the time, labs 90% of the time :P [17:06:09] anyone know where on https://gerrit.wikimedia.org/r/#/admin/projects/ we could have a project for osm puppet stuff? [17:06:16] like what we'd call our repository? [17:06:21] aude: I think I had that error as well to start with, but can't quite remember how I solved it. [17:06:32] apmon: ah [17:06:51] It might have involved just creating a new instance and deleting the old one [17:06:53] * aude just needs the time to ask and my time usually doesn't correspond with wmf staff [17:06:59] apmon: probably that's it [17:07:10] i had trouble with the instance to start with [17:07:10] I think you can pretty much name it anything reasonable [17:07:31] like operations/openstreetmap ? or operation/maps [17:07:51] not "osm" because that's confusing with openstackmanager :D [17:07:55] Is it related to operations? [17:08:01] Damianz: yes [17:08:07] then sure [17:08:12] ^demon can always say no [17:08:33] the mobile folks want production map tiles so we don't rely on a third-party [17:10:26] Given the situation on the toolserver, it is probably good to move the production tiles for {de,ru,it,es,...}wiki off of toolserver and onto a WMF server as well [17:11:15] Probably a good idea [17:11:43] and as if I understand it correctly WMF actually already has dedicated hardware for it (at least for mobile), it would be good to be able to use that as soon as possible [17:11:49] Though imo labs is only another excuse for not funding TS more, I remember when they bitched over hardware last time. Dab does have a point though [17:12:11] Also WTF someone is drilling into a wall with a hammer action down stairs, my wall is shaking [17:12:25] apmon: agree [17:13:18] although not sure how much labs can be an exact replacement for toolserver in terms of functionality for maps stuff [17:14:13] The wikimedia mobile server is not part of labs though? [17:14:35] no [17:14:53] but we start with labs, as a beta, testing and experimental place [17:15:24] with the toolserver, people always are experimenting with stuff and that's still important [17:15:59] apmon: https://www.mediawiki.org/wiki/Git/New_repositories/Requests [17:16:04] put a request at the bottom [17:16:27] when i get puppet working, i can put stuff in there [17:16:44] And you have to remember current labs isn't really tool labs, there's sorta plans and goals for that and running 'production' stuff while being able to develop on it in roughly the same but different envs. [17:17:16] Damianz: right [17:17:32] aude: You can use the maps-test2 instance if you want. There puppet works [17:17:41] apmon: i can try it and see if it works [17:17:49] i think i just need to recreate my instance [17:17:51] although I have already created a postgresql.pp and osmtileserver.pp in /etc/puppet/manifest [17:17:55] ok [17:19:00] You probably want to comment those out though and start fresh. [17:19:29] ok [17:19:38] i'll just see if i can run puppetd [17:19:47] Oh, the other thing I was trying that didn't work if I remember correctly was getting the custom debian repository going [17:19:53] ah, yes [17:20:40] As that will be necessary to package osm2pgsql, mod_tile and renderd (and possibly others like mapnik) [17:20:47] yes [17:21:11] and we can do mod_tile and renderd to start [17:21:26] but i'd be interested in trying different rendering tools and see how they do [17:21:31] (if time allows) [17:23:00] Its possible. Although given that I am sort of the upstream maintainer of mod_tile and renderd, those are the ones I am most familiar with and can hack and extend if necessary [17:23:36] ah, okay [17:24:24] although depending on needs, mod_tile together with tirex might be the better option than renderd [17:24:31] Sadtimes, openstreetmap says my house is lidl [17:24:35] simple to start is ok [17:24:42] Damianz: fix it! [17:24:55] aude: We'll it's sorta correct, but lidl is downstairs :P [17:25:02] ah, okay :) [17:25:07] convenient [17:25:12] Indeed [17:25:17] hi Ryan_Lane [17:25:31] * Damianz hands Ryan a cookie [17:25:41] Damianz: we use cron to schedule bots on labs? [17:26:05] dschwen: i think so [17:26:05] dschoon: Yes [17:26:06] Damianz: 3D mapping (i.e. multi story buildings) is not well established yet [17:26:19] unfortunately [17:26:19] buh? [17:26:21] We really need a distributed schedular that uses multiple nodes, but that's a lot of work [17:26:35] * aude tried mapping my local airport, such as the baggage level and departures :) [17:26:46] apmon: I totally should be able to walk up my stairs, we just need to send people around with rPis and webcams :P [17:27:18] howdy [17:27:27] I do rather like http://stations.aeracode.org/ [17:28:05] * Damianz <3 webgl [17:29:30] Are those complete? I always had Oxford Circus more complicated in remembrence. It nearly looks sane on that map... ;-) [17:30:00] alright, i should go home but be back in a bit [17:30:25] Some of them aren't, the notes say what's missing roughly. They're from memory/observion rather than drawings though. [17:30:42] Still pretty cool and would be interesting to have the full set, done 100% to the drawings. [17:32:00] Yes, it is cool. I always wondered what they would look like on a 3D map while walking through those tunnels [17:34:00] Damianz is there a mechanism on bots that allows for multi maintainer projects? [17:35:44] Roughly the idea is to use /data/project which is shared betwean all instances for development, the actual running etc is a little harder due to the lack of schedular/control etc, so depending what user it's running under you have to do a bit of messing about (or be really awesome and make an init script). [17:37:20] could you mkdir /mnt/public_html/dschwen please [17:38:17] and could we get git on bots? [17:39:17] dschwen: can you still not log into bots? [17:39:54] I'm on bots, I hav just had my first successful bot run [17:40:50] great [17:41:16] Damianz: the "No credentials for your account" thing was reintroduced with keystone [17:41:20] on new account creations [17:41:36] dschwen: you said you saw that error? [17:41:46] Ryan_Lane: Just so you know, I hate you. [17:41:47] was it after changing your preferences? [17:41:49] logging out and back in on labsconsole made it hgo away for me [17:41:53] Damianz: heh. why's that? [17:41:53] Ok, now that's out, I love you again. [17:41:58] Stupid bugs [17:42:12] the credentials bug? [17:42:22] well, I know why it occurs on account creation [17:42:24] mhm, really I hate mw core devs but they're grumpy :P [17:42:42] and I also know why it happens on preference chanegs [17:42:44] dschoon: Yes, and yes, I might move the dir as well though so let me finish reading this stupid bug report [17:42:46] actually, let me fix that right now [17:42:54] ... [17:42:59] https://bugs.launchpad.net/ubuntu/+source/gnome-terminal/+bug/1055766 < This is why I hate ubuntu a little [17:43:00] Why what? [17:43:18] I think maybe you are talking to dschwen [17:43:20] heh [17:43:22] Ok, does anyone know why sshing to login.toolserver.org gives a network is unreachable? Other hosts work. [17:43:35] dschoon: Oh damn you starting your nick with dsc :P Yeah I meant dschwen [17:43:41] dschwen: from labs? [17:43:45] yes [17:43:48] I think it's firewall on their side [17:43:51] hehe [17:43:55] from my wikiminiatlas instance in the maps project [17:43:59] I can ssh out to other servers fine and can ping/traceroute it find [17:44:04] Just TS it fails on. [17:44:05] well, it's possible that we block ssh access into it [17:44:08] yeah yeah, i know [17:44:18] the evil toolserver? ;-) [17:44:26] we block ssh access to our production networks [17:44:34] it's possible that TS is also accidentally blocked [17:45:01] ok, it would be convinient to have access so that I can copy my stuff over easily [17:45:08] Let's blame lesslie, she isn't here :P [17:45:27] Yes, damn you, leslie [17:45:30] who is she? [17:45:35] heh [17:46:22] http://meta.wikimedia.org/wiki/User:LeslieCarr < [17:46:41] Still want her hair :( [17:47:21] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Platonides link https://www.mediawiki.org/w/index.php?diff=587101 edit summary: [17:48:08] Platonides++ [17:49:09] oh, right, yeah, her amount of hair would be great on my head [17:49:44] Though I'd probably be in favour of abstracting that hmm, I guess if we used gluster to store the data then scaling out to multiple web nodes later would be easy. [17:49:51] or were you referring to her actual hair, to make a little hair-doll and keep it under your pillow... [17:50:01] ...that went creepy fast [17:50:05] o.0 [17:50:13] * Damianz checks his voodoo dolls are still there [17:50:40] My hopefully new boss seems awesome so I don't think he'd mind me diying my hair pink again [17:51:36] http://www.bbc.co.uk/news/uk-19710183 < In a country that rains this much you'd have thought train companies would have come up with a solution that doesn't involve 'CLOSE ALL THE LINES' [17:51:39] so aaaanyhow... how about that /mnt/public_html/dschwen dir. Or did you already comment on that (dir will be moved) [17:51:49] 1sec [17:52:28] I'm not moving the dir because it's nfsifiedautomounted all over the shot and I'm feeling lazy, I'll bz it and do it when homedirs move [17:52:36] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Platonides link https://www.mediawiki.org/w/index.php?diff=587107 edit summary: [17:52:43] come on apache1... this box is really crappyslow ;( [17:52:51] mostly nfs's fault though [17:53:14] hmm [17:53:16] * Damianz sigh [17:53:34] ok [17:53:42] dschwen: You'll have to give me a min, need to deal with something [17:53:47] sure [17:59:18] Ryan_Lane: You love good news, right? [18:00:42] !log bots apache1 running crazy shit processes under www-data, has at least 3 exploits and 1 udp processes downloaded. [18:00:42] I am broken, and I can't log anything, please fix me [18:00:45] !log bots Disabling phpmyadmin on apache1 due to new exploit [18:00:46] I am broken, and I can't log anything, please fix me [18:00:48] !log bots Copied /var/tmp to /root/var-tmp on apache1 [18:00:48] I am broken, and I can't log anything, please fix me [18:00:50] !log bots Killed 14654 on apache1 [18:00:51] I am broken, and I can't log anything, please fix me [18:00:53] !log bots rm -rf /var/tmp/* on apache1 [18:00:53] I am broken, and I can't log anything, please fix me [18:00:55] !log bots Copied /tmp to /root/tmp on apache1 [18:00:55] I am broken, and I can't log anything, please fix me [18:01:00] !log bots rm -rf /tmp/.s on apache1 [18:01:01] I am broken, and I can't log anything, please fix me [18:01:05] * Damianz frowns at wm-bot [18:01:30] wm-bot: I hate you [18:01:30] Hi Damianz, there is some error, I am a stupid bot and I am not intelligent enough to hold a conversation with you :-) [18:02:05] Anyone a udp script? [18:04:19] dschwen: Done [18:07:22] thanks [18:11:41] Ryan_Lane: Before I descend into OpenStack documentation, is there anything more pressing you'd like me to look at? [18:41:11] haven't done it yet [18:41:21] hm. public datasets don't seem to be accessible [18:41:28] I'll need to fix that [18:41:36] I wish people would report stuff like this. heh [18:41:41] ok, bots is now off nfs for apache... should help a little [18:41:49] <^demon> Damianz: I've done some testing for it. Should be pretty trivial to setup after we upgrade to 2.5 (I've got a patch for the replication plugin I want merged first) [18:42:17] Ah. [18:43:55] <^demon> Also, annoying is that there's no automated way to create repos on github's side when they're created in gerrit. [18:44:00] <^demon> Been working on that yesterday & today. [18:44:14] <^demon> Gonna have to use github's api, prolly. [18:44:55] Via the api should be easy enough, as gerrit uses git for replication and you actually just straight replicate to github? that wouldbe sorta cool [18:45:13] <^demon> Yeah, we'd just replicate straight to github. [18:45:38] <^demon> I'm probably going to just hack up a silly plugin for our usage that does it, so I can Just Do It rather than making it generally useful in the replication plugin. [18:45:51] Still annoying that gerrit rewrites commits and causes a diverge so you can't just use it as a remote to push for review though [18:46:02] <^demon> Ideally, the replication plugin would handle this automatically like it does with local disk and ssh-based replication. [18:47:24] <^demon> Also, I found out that dashboards aren't actually private in 2.5...they're just moved to even *worse* urls than they are now. [18:47:29] <^demon> Completely unguessable. [18:47:32] * ^demon facepalms [18:48:13] <^demon> The user experience with this really needs some work...dashboards are super-useful if they were just exposed in a sane way. [18:49:01] Half the reason people hate gerrit is it's ugly and hard to find stuff, like who the hell puts a project listing under some obscure admin area. That might work for internal we know what exists stuff but sucks for community stuff. [18:49:20] <^demon> Damianz: It's not under admin anymore. [18:49:22] <^demon> I moved it ;-) [18:49:57] looks under admin :P [18:53:52] hmm [18:54:33] * Damianz wonders how much the wmf contributes towards ^demon's dugs, that group interface makes no sense at all [18:55:12] <^demon> https://gerrit-review.googlesource.com/ - It's under Projects -> List now :) [18:55:23] <^demon> And if you've got create rights, there's also Projects -> Create new project [18:55:57] The green hurts my eyes [18:56:20] Though that's much better... though wtf does the url have a # in it [18:58:17] use greasemonkey to change CSS to the way you like it? [18:59:55] <^demon> I like that upstream has a different theme. Easier for me to remember which gerrit I'm using :p [19:00:43] 09/25/2012 - 19:00:43 - Creating a project directory for webtools [19:00:43] 09/25/2012 - 19:00:43 - Created a home directory for platonides in project(s): webtools [19:03:32] mutante: That requires effort, also dotjs > greasemonkey [19:05:43] 09/25/2012 - 19:05:43 - User laner may have been modified in LDAP or locally, updating key in project(s): webtools [19:55:27] Damianz, any reason bots is not running python 2.7? [19:56:10] oh, right ubuntu packages... [19:59:30] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by -jem- link https://www.mediawiki.org/w/index.php?diff=587157 edit summary: Adding dumps, pagecounts and projectcounts [20:01:10] ubuntu [20:01:16] we need to upgrade the instances [20:02:10] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Dschwen link https://www.mediawiki.org/w/index.php?diff=587159 edit summary: [20:02:46] right, my instance has python 2.7 [20:03:04] it is just that pywikipediabot complains about outdated python [20:04:37] is it possible to create and assign unix groups to users in my instance? [20:04:49] yes [20:05:10] dschwen: do you have your own instance? [20:05:16] yes [20:06:12] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Nemo bis link https://www.mediawiki.org/w/index.php?diff=587160 edit summary: +Compatibility for Toolserver tools etc., e.g. same SGE [20:07:31] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Nemo bis link https://www.mediawiki.org/w/index.php?diff=587161 edit summary: bullets [20:12:45] how can I change my primary usergroup and why is it set to 'svn'? [20:13:16] "sudo usermod -g dschwen dschwen" (after adding group dschwen did not work) [20:45:47] 09/25/2012 - 20:45:46 - Created a home directory for scfc in project(s): bastion [20:46:37] dschwen: your user account is in LDAP [20:47:01] so, you can't really change your primary group [20:50:43] 09/25/2012 - 20:50:43 - User scfc may have been modified in LDAP or locally, updating key in project(s): bastion [20:51:09] you should use sgid bits on directories [20:51:14] and project groups [20:51:32] and permissive umasks (002) [20:54:03] ok [20:55:08] btw. on my instance, trying to compile a C++ program I get "virtual memory exhausted: Cannot allocate memory" [20:56:06] any idea why that could be? [20:56:42] ^demon: I am wondering, how do you get the gerrit database setup in labs ? [20:56:48] ^demon: is there a puppet class for that ? [20:58:21] going to need a gerrit instance on the jenkins host :-) https://gerrit.wikimedia.org/r/25173 [21:06:13] <^demon> hashar: No, I just have gerrit-db that I setup a database on. [21:06:24] <^demon> We never did find a way to properly puppetize that process. [21:06:29] just a mysql weigh approriate username ? [21:06:36] gr [21:06:49] so that is just a mysql instance with the appropriate database name and username ? [21:07:01] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Eloquence link https://www.mediawiki.org/w/index.php?diff=587169 edit summary: [21:09:21] ^demon: can you give a pick at ? [21:09:22] https://gerrit.wikimedia.org/r/#/c/25173/ [21:09:26] paravoid: any idea why virt5 is asking me for a partitioning method in the installer? [21:09:33] 'Compatibility for Toolserver tools etc.' 'Support same SGE' that's a serious feature wanted [21:09:46] this is why shit never gets done, let's not improve it lets just re-implimented it somewhere else [21:09:48] virt[5-11]) echo partman/virt-raid10-cisco.cfg ;; \ [21:09:51] * Damianz facepalm headdesk stabs self [21:12:06] Damianz: you will need "Solaris" in the drop-down list when selecting OS image on new instance creation :o [21:12:14] <^demon> hashar: Done [21:12:18] yeah "Support same SGE" looks like a knee-jerk reaction after the toolserver death discussion [21:12:41] mutante: Please god no [21:12:42] ^demon: danke [21:12:46] Bad enough using Suse at work [21:12:46] labs will NEVER be as good as the toolserver unless it is EXACTLY the same as it [21:13:05] ^demon: ho and what is the ssh key for ? [21:13:10] s/good/bad [21:13:27] dschoon: knee-jerk doesn't push things forward and achieve what labs is suppose to be, it just makes a few people happy and everything else goes to shit... but maybe that's just my thoughts. [21:13:40] changing a few qsub/qcronsub commans will be the least of our problems [21:13:42] * Damianz goes to find ice cream [21:13:55] <^demon> hashar: The ssh key stuff is kind of messy right now. The private key is needed if you're doing replication from a master. The public key is needed if you're receiving replication as a slave. [21:14:01] <^demon> The config for that still needs cleaning up. [21:14:03] for the love of god, I'm not dschoon [21:14:14] ^demon: so I guess I can ignore that :) [21:14:29] * Damianz replaces dschoon with dschwen [21:14:50] I'm seriously thinking of suing for trademark infringment ;-) [21:15:17] We should have a rule that the first 3 chars of your nick has to be unique :P [21:15:24] You're like a dyslexic nightmare [21:15:33] ^demon: bah now I will depend on your change https://gerrit.wikimedia.org/r/#/c/24213/ from last week [21:20:29] * dschoon is not pleased [21:20:42] ^demon: did the rebase and changed the files :-) [21:20:49] it's all because o sorts before w :( [21:20:55] someone should sue the alphabet [21:22:39] !g 24934 [21:22:39] https://gerrit.wikimedia.org/r/#q,24934,n,z [21:25:00] ahh my instance is looking silly :-] [21:25:44] git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/34/24934/8 && git checkout -b 24934/8 FETCH_HEAD [21:25:45] git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/73/25173/2 && git checkout -b 25173/2 FETCH_HEAD [21:25:45] git checkout -b zuulintegration && git merge 24934/8 && git merge 25173/2 && puppetd -tv [21:25:53] *yawn* [21:26:15] !log integration jenkins: git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/34/24934/8 && git checkout -b 24934/8 FETCH_HEAD [21:26:15] There are multiple keys, refine your input: logbot, logs, [21:26:19] pff [21:26:27] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Ryan lane link https://www.mediawiki.org/w/index.php?diff=587171 edit summary: [21:26:28] I give up logging anything [21:27:00] !logs del [21:27:00] Successfully removed logs [21:27:03] !logbot [21:27:03] https://labsconsole.wikimedia.org/wiki/Nova_Resource:Bots/Documentation#Troubleshooting [21:27:07] !logbot del [21:27:07] Successfully removed logbot [21:27:12] now wtf is adminbot [21:28:20] Damianz: labs-morebots = adminbot [21:28:29] (There is an init script for it called "adminbot". (/etc/init.d/adminbot). ) [21:28:51] * Damianz looks at mutante [21:29:18] !log integration jenkins: git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/34/24934/8 && git checkout -b 24934/8 FETCH_HEAD [21:29:23] Damianz: "It needs /var/run/adminbot as a cache directory and permissions to write to it." [21:29:43] if it dies on !log command it is usually the cache dir [21:29:47] !log integration jenkins: git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/34/24934/8 && git checkout -b 24934/8 FETCH_HEAD [21:30:03] mhm [21:30:12] I'm just freaking patching it because it sucks [21:30:53] !log integration jenkins: git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/34/24934/8 && git checkout -b 24934/8 FETCH_HEAD [21:30:58] oh diaf [21:31:00] so.. Ryan_Lane there is a project for webtools? [21:31:13] * Damianz goes to figure out why the hell it can't write to the wiki [21:31:30] I have the ZoomViewer project which does not need DB access, but uses VIPS to scale superlarge images for fluid zoomin on commons [21:31:43] would that be a fit for the webtools project? [21:32:20] VIPS can be demanding memory-wise, but only for short amouts of time (20s every 30min) [21:32:30] !log integration -jenkins, applying role::gerrit:labs::jenkins [21:32:57] !log integration -jenkins created a new branch for puppet master self and merged in 24934/8 and 25173/2 [21:33:00] dschwen: well, probably. I think Platonides meant it to be a project where root access wasn't required [21:33:05] wm-bot: you really suck [21:33:05] Hi hashar, there is some error, I am a stupid bot and I am not intelligent enough to hold a conversation with you :-) [21:33:14] which means someone would need to build the environment from scratch [21:33:23] Damianz: see, it does not want me to log anything [21:33:29] is the log bot dead again? [21:33:35] hashar: that's a different bot [21:33:35] I just killed it [21:33:41] it's throwing an excepting trying to edit the wiki [21:33:49] what exception? [21:33:58] mwclient.errors.EditError [21:34:04] any idea why? [21:34:11] trying to figure out why [21:34:15] oh [21:34:18] !log test [21:34:18] Message missing. Nothing logged. [21:34:19] I wonder if the password changed [21:34:24] !log test SCREW YOU [21:34:24] test is not a valid project. [21:34:26] yeah, I would not need root access, just someone with root access installing some pacages [21:34:29] heh [21:34:31] !lot bots MEH [21:34:33] !log bots foobar [21:34:34] Logged the message, Master [21:34:38] WTF [21:34:40] * Damianz rawr [21:34:46] Yeah this bot sucks [21:34:47] heh [21:34:55] well, it was working without issue previously [21:34:59] wait [21:35:00] Now I'll restart it under init... [21:35:03] where are you running the bot? [21:35:08] bots-labs [21:35:11] ok [21:35:12] just making sure [21:35:15] it used to be on bots-2 [21:35:19] it's still on 2 though [21:35:22] dunno if people start it there [21:35:25] it needs to be purged from 2 [21:35:30] Platonides, are you around? is the web tools project ready for having me added as a user? [21:35:35] needs to be puppetized and shiz [21:35:39] !log bots TEST [21:35:41] Logged the message, Master [21:35:44] ok.. [21:35:47] I give it 10min [21:35:47] ugh.. lets fix the docs [21:35:53] dschwen: well, it has nothing in it right now [21:35:57] does it have the cache directory? [21:35:58] dschwen: it's an empty project [21:36:07] I'll fill it up! [21:36:12] it needs to have instanced added, etc [21:36:17] oh [21:36:23] Ryan_Lane: Just wondering, I see it logs in every time *but* is it possible that it may be killed the session token randomly and the lib is trying to re-use it? [21:36:31] I'd hope .login actually does a re-login but who knows [21:36:38] hm [21:36:43] I wonder if it stores a token [21:36:46] that would be problematic [21:36:48] be nice if the traceback actually had the headers... or you know, logged anywhere [21:36:54] because the same user is being used in multiple places [21:36:55] because running it to see stdout sucks [21:36:55] oh, Platonides was busy writing an email... [21:37:43] Is adminbot in git? [21:37:52] I might add some logging stuff because it annoys me [21:38:52] Damianz: operations/debs/adminbot [21:39:02] but i did not check if there is anything in the project yet [21:39:03] mutante: <3 [21:39:16] feels like it should be in puppet also [21:39:24] considering it's rather central to labs [21:39:33] yep https://gerrit.wikimedia.org/r/#/q/project:operations/debs/adminbot,n,z [22:01:35] Ryan_Lane mentioned /data/project directories on the wiki. I dont see those from my instance (wikiminiatlas). My /data only contains: [22:01:41] d????????? ? ? ? ? ? home [22:01:46] yikes! [22:02:01] it's an automount [22:02:01] Ryan_Lane: no... [22:02:04] just try to access it [22:02:09] ls /data/project [22:02:12] it'll mount itself [22:02:16] ls: cannot access /data/home: No such file or directory [22:02:19] (responding to your question from an hour ago) [22:02:22] paravoid: ok [22:02:28] dschwen: which instance is this? [22:02:40] wikiminiatlas in project maps [22:03:00] is it safe to reboot instances? it should be, right? [22:03:01] it has an old version of gluster installed [22:03:08] oh [22:03:11] fixing that [22:03:53] thanks! [22:03:59] works now [22:04:13] yeah [22:04:18] so who can write there? [22:04:33] well, by default root [22:04:43] you should avoid using /data/home for now [22:04:48] oh, that's me ;-) [22:04:54] we'll be switching to that for home directories soon [22:04:58] ok, is that on the storage cluster? [22:05:01] yep [22:05:08] it's accessible from all instances in the project [22:05:11] what does that mean? [22:05:21] no more per user home dirs?! [22:05:30] it'll still be per-user [22:05:52] the home directories are in a location that isn't terribly reliable right now [22:05:53] but it would be useful for collaboration within a project? [22:05:58] ok [22:06:04] I kinda wish we had a 'global' folder [22:06:12] across projects? [22:06:13] no. home directories are very non-collaborative [22:06:15] Be nice to fix stupid vim configs on all projects [22:06:19] /data/project is collaborative [22:06:26] Damianz: heh. that's too dangerous [22:06:35] Ryan_Lane, yes, I meant /data to be for collaboration [22:06:39] it's just begging for projects to own other projects [22:06:49] Ryan_Lane: Oh just because I'd put sudo unicorn_porn in your bashrc :P [22:06:51] dschwen: /data/project specifically, yeah [22:07:13] only read-only things can be global [22:07:27] hm [22:07:47] we could have a global share that has useful environment [22:07:50] that's read-only [22:08:00] we could manage it via gerrit [22:08:54] /public/environment [22:09:25] or something like that [22:10:05] I'm waiting for the day gluster screws up it's acls and someone totalls everyones ssh keys but there again I'm mostly an ass [22:17:54] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Ryan lane link https://www.mediawiki.org/w/index.php?diff=587180 edit summary: [22:18:44] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Ryan lane link https://www.mediawiki.org/w/index.php?diff=587182 edit summary: [22:25:06] Change on 12mediawiki a page Wikimedia Labs/Toolserver features wanted in Tool Labs was modified, changed by Ryan lane link https://www.mediawiki.org/w/index.php?diff=587187 edit summary: [22:46:09] Ryan_Lane, I'm still getting a "virtual memory exhausted: Cannot allocate memory" error trying to compile a code on my wikiminiatlas instance (with both g++-4.6 and g++-4.4, but it compiles fine on my ubuntu precise machine at home). Any ideas? [22:46:34] what size instance are you using? [22:46:47] ugh, one sec [22:47:11] total mem 1G, free 740M [22:47:42] at home I have 8G in my VM [22:48:00] hm [22:48:02] this is a small instance [22:48:05] it should be fine [22:48:10] but it is a fairly small program (uses mapnik, boost etc.) [22:48:32] where are you running it? [22:48:56] running what? [22:49:05] the compile job? on the wikiminiatlas instance [22:49:11] in which directory>? [22:49:12] in wma/programs/mapniktile [22:49:25] build4.sh [22:49:29] which top level directory? [22:49:36] /home/dschwen [22:49:53] use /mnt or /data/project [22:50:13] there is no /data/project [22:50:17] just /data.home [22:50:19] yes. yes there is [22:50:24] it's an automount [22:50:24] it's automount [22:50:27] it auto mounted [22:50:30] just do ls /data/project [22:50:31] s/it/ie/ [22:50:48] oh, you wizzards! [22:52:02] what other secret directories are there? [22:52:09] same story [22:52:41] in 'top' I can see the cc1plus process memory shooting up [22:52:55] why would that change in another dir? [22:53:49] at home virtual memory requirement tops out at 500M [22:54:26] that;s with g++ 4.7. I'm trying to upgrade my instance [22:55:27] darn, I forgot I'm already running quantal in my VM at home [22:59:49] any way the RAM of an instance can be increased? [23:01:06] dschwen: create a new instance with more RAM [23:01:42] ugh and move all my stuff to the new instance? [23:01:49] or you can create a cluster on top of several instances. So you will just have to throw new instances in your cluster. [23:02:08] I understand that resizing disk is hard, but adding ram should be easy after a reboot [23:02:10] dschwen: just apply the same puppet classes [23:02:18] aaahhhrrrgggghhhh [23:02:29] there are no puppet classes for my stuff [23:02:49] and last time I created an instance selecting puppet classes caused it to fail on creation [23:03:14] puppet is like chinese to me at this stage [23:03:19] dschwen: it usually works better to first finish creation, like until you can SSH to it.. and then apply classes [23:03:53] one major advantage is just this..do work once and be able to apply it to multiple instances as much as you like [23:04:26] yeah, but it seems to be a ton of work now for creating 1 or 2 instances [23:04:35] I'm debating if puppet is awesome or not currently [23:04:44] there will be more in the future,right [23:04:48] Looking at Chef as an alternative but I'm not liking so much [23:05:21] any configuration management will be more work to setup the very first instance [23:05:26] I'd rather spend my time developing my tools... [23:05:28] but a huge advantage after that is done [23:05:37] I need one instance [23:05:38] sigh... [23:05:46] yes, "sigh" [23:05:47] I shall not enter this conversation due to my huge bias [23:05:49] others could also use your classes and see what is actaully going on [23:06:20] yes, I'm sure there are plenty of benefits [23:06:44] dschwen: how about just usign rsync then from old to new instance? [23:07:07] i mean ..how many pathes do you possibly have to copy to get your stuff moved over [23:07:14] /home and /etc ? [23:07:33] well.. /etc minus the hostname and stuff [23:08:01] yeah yeah. [23:08:11] renameing instance is not possible, is it? [23:08:12] dschwen: also.. if you use the shared storage.. it is already shared between instances in a project [23:08:22] so dont even need to copy [23:08:42] it is more about user config (ssh stuff) and installed packages and PPAs [23:09:35] dschwen: installed packages like this: on old host: dpkg --get-selections > somefile [23:09:56] dschwen: on new instance: dpkg --set-selections < somefile [23:11:18] I know [23:11:27] but thanks [23:11:40] so, renaming instance is not possible, right? [23:11:54] i dont think it is, nope [23:12:06] well, you can change your hostname [23:12:17] but not changing the type of instance to have more RAM [23:12:55] nahrgh. failed to create instance [23:13:31] I think I have f(*&^*ed up majorly now [23:13:43] or let's say I'm testing the labsconsole thoroughly [23:13:56] I created an instance with the same name as an existing instance [23:14:04] and deleted it while it was stiull building [23:14:21] ldap error: cannot create hostname (of course because it exists already) [23:14:35] on deleting: cannot remove DNS entry (I hope so) [23:14:42] now it is still in the instance list [23:14:56] state: building [23:15:33] huh, now suddenly creating a new instance doesn't fail [23:15:37] this is odd [23:16:29] nope, staus: error [23:16:47] you killed it :)) [23:16:59] :-( [23:16:59] so.... [23:17:04] you deleted? [23:17:11] not the new instance [23:17:15] the old one? [23:17:20] the one with the same name I deleted [23:17:25] and it is gone from the list [23:17:28] It can't have the same name [23:17:31] yeah [23:17:38] I have an open bug on it allowing the creation to occur [23:17:58] it needs to have a different name, due to dns [23:18:05] so, a resize is possible [23:18:08] technically [23:18:10] I haven't tried it [23:18:13] since the upgrade [23:18:16] wikiminiatlas-2 instance was created but has status error and no console output [23:18:16] Urgh why is it only 7am, damn timezones... what's with this sillyness of working with teams around the world. [23:18:28] status error? [23:18:34] sec [23:18:34] yeah [23:18:58] Ryan_Lane: I keep meaning to ask - is it possible to make the console output render quicker? It seems somewhat delayed from an instance being built [23:19:01] hm. you must have hit the race condition [23:19:06] yay [23:19:14] delete that instance and re-create it [23:19:17] ok [23:19:20] it's pretty rare to hit [23:19:25] I will click slowly this time ;-) [23:19:35] well, it's a race condition with openstack nova [23:19:49] the interface isn't triggering it [23:20:07] it is still in the list [23:20:17] should I wait before creating the new instance? [23:20:21] I'd love if you found a timing bug, else clicking slowly isn't going to help [23:21:06] dschwen: no need to wait [23:21:14] where do I add puppet classes? I have zero clue about this. [23:21:19] configure instance page [23:21:34] * Damianz thinks we need screenshots in docs [23:21:37] actually [23:21:50] no, I mean where do I add my own classes? [23:21:54] Ryan_Lane: Didn't you do a talk on the basics on using labs? could be intesting for docs [23:21:58] puppet groups [23:22:02] like in create a puppet class for my stuff [23:22:05] then they'll appear in the configure instance page [23:22:18] you'll need to get it merged into ops/puppet repo first though [23:22:23] or use puppetmaster::self [23:22:26] which is semi-awesome [23:22:29] I think I saw that talk and still have no clue ;-) [23:22:59] i can see the words you typed in the chat but I don't know what they mean [23:23:04] (to paraphrase) [23:23:05] Basically, if you want to develop anything puppet tick the puppetmaster::self box and read https://labsconsole.wikimedia.org/wiki/Help:Self-hosted_puppetmaster [23:23:19] ok [23:23:25] That's the sorta route to making stuff that's mergable into the ops/puppet repo [23:24:13] assuming you don't know wtf puppet is, read http://docs.puppetlabs.com/guides/language_guide.html and look at the existing stuff for examples [23:24:59] ok, thanks for the pointer [23:25:25] Ryan_Lane, I've got another state: error on the new instance [23:30:45] ugh [23:31:14] I'm looking at the logs [23:32:51] Are the trees pretty this time of year? [23:33:54] with a supermarket on the ground floor I guess you have zero reason to go 'outside' [23:34:18] You have to go outside to get to the supermarket [23:34:26] And to the office [23:34:51] hatch in the floor -> supermarket. telecommute -> office [23:35:23] wow. I just got "no credentials" [23:35:30] :-D [23:35:47] have a taste of your own medicine ;-) [23:36:07] Ryan_Lane: That combined with the log randomly not being able to make edits makes me think there's a larger issue with labsconsole but meh [23:36:15] s/log/bot/ [23:38:16] dschwen: which project is this? [23:38:19] maps [23:38:26] instance wikiminiatlas-2 [23:42:40] sorry for breaking your stuff :-( [23:44:05] * Damianz hands dschwen the naughty step [23:44:11] well, we've introduced some changes lately [23:44:22] I'm going to try to create it and watch the logs [23:44:26] ok [23:44:35] it deleted for me [23:44:39] which threw an error on you [23:45:14] ok, deletion only threw an error for the second wikiminiatlas instance which was also called wikiminiatlas [23:46:00] seems to be working this time [23:46:11] active [23:46:11] yeah. it did [23:46:14] so... [23:46:21] now you need to wait for puppet to finish running [23:46:31] it takes about 5 minutes [23:46:35] which type did you make it? [23:46:41] m1.medium [23:46:47] ok, thanks [23:46:50] yw [23:46:55] sorry you hit errors [23:47:03] no big deal [23:47:17] you're the lucky winner today ;) [23:47:28] I haven't been hearing many complaints of breakage lately [23:47:37] could my prize be membership in the webtools project? [23:48:00] My complaining doesn't count :P [23:50:53] Ryan_Lane, it couldn't be that the instance IP ended in .255 when I tried to create it, right? [23:51:30] just struck me as suspicious [23:51:36] wait [23:51:37] it did? [23:51:42] that would definitely be a problm, yes [23:51:43] yeah, it did [23:51:56] that should be a reserved IP [23:52:04] I thought so [23:52:23] well, that should be an easy fix [23:53:32] it is .4.252 now [23:53:38] and indeed it isn't marked as reserved in the database [23:53:49] aha [23:54:07] * Damianz noms [23:55:37] well, fixed that