[00:10:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [00:11:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [00:32:59] New review: Johnduhart; "No, still links to a new bug form (for the subversion component, wtf)" [operations/puppet] (test) C: -1; - https://gerrit.wikimedia.org/r/2012 [00:40:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [00:41:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [00:42:13] mdale: what did you need? [00:42:41] hi [00:43:02] need to update TimedMediaHandler [00:43:03] hey [00:43:05] k [00:43:15] would like to know where it lives on the server... [00:43:37] 1s while I verify [00:44:29] mdale: /usr/local/apache/common/live/extensions/TimedMediaHandler [00:44:41] thanks ;) [00:44:47] mdale: I'll run svn up now [00:45:00] great thanks ( for TimedMediaHandler and MwEmbedSupport ) [00:47:49] running [00:51:11] !log deployment-prep svn up * updatedata [00:51:12] Logged the message, Master [00:54:04] PROBLEM Current Load is now: CRITICAL on labs-realserver labs-realserver output: Connection refused by host [00:54:34] PROBLEM Current Users is now: CRITICAL on labs-realserver labs-realserver output: Connection refused by host [00:55:04] PROBLEM Disk Space is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [00:55:44] PROBLEM Free ram is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [00:57:04] PROBLEM Total Processes is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [00:57:44] PROBLEM dpkg-check is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [01:01:56] johnduhart: BTW, what's the URL to your Phabricator setup? [01:05:25] RoanKattouw: phabricator.wmflabs.org [01:07:13] I need to look at that in more detail [01:07:19] Judge how much I like the interface [01:08:11] I'll be working on more lint tests and getting unit test working soon, aswell as a script to set up arcanist on the client [01:08:25] Once we're sure this is what we're want I'll add LDAP integration [01:08:28] Ryan and I were talking about maybe putting the Phabricator front-end on the Gerrit backend [01:08:35] I have lint tests working in my Gerrit+Jenkins VMs [01:08:47] RoanKattouw: What do your lints look like? [01:08:58] Lemme grab this [01:09:03] My VMs don't have public IPs [01:09:09] sure [01:09:44] It's in Jenkins, using the Gerrit Trigger Plugin (the OpenStack fork of it, specifically) [01:09:58] Output looks like http://jenkins2.pmtpa.wmflabs:8080/job/lint/22/ [01:10:25] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [01:10:47] The lint job is really just a shell job that does this https://gist.github.com/1667120 [01:11:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [01:14:48] RoanKattouw: Besides besides syntax arcanist can do more advanced linting [01:15:13] Ryan_Lane, RoanKattouw what does Phabricator do that gerrit would be good for as a backend [01:15:14] Cool [01:15:28] Having a non-terrible UI? xD [01:15:39] How does their submission backend work, compared to Gerrit, anyways? [01:15:57] I really like the way gerrit stages revisions and automatically merges them when submitted [01:16:06] can phabricator do gated trunk? [01:16:15] <^demon> +1. And unless something else can do that too it's a non-starter. [01:16:23] RoanKattouw: There's a client that shovels diffs to the server for review [01:16:28] <^demon> I'm totally for friendlier UIs, but that backend functionality must be there. [01:16:47] Oh, just diffs? [01:16:57] So they're not actually git revision objects? [01:17:01] RoanKattouw: no [01:17:05] OK [01:17:11] And when you approve something, does it merge it for you? [01:18:24] RoanKattouw: Not really, well sort of. [01:18:42] Also, I was thinking, it seems that Gerrit's UI is mostly just AJAX to their API, so it might be reasonably feasible to write a standalone JS/AJAX UI that uses their API [01:18:44] Like, not from the UI. You need to have a trusted user merge it in [01:18:52] OK, so that's manual intervention [01:19:45] <^demon> RoanKattouw: To their "API" [01:19:50] Well yeah [01:19:56] <^demon> It's not an external API, so we'd likely get bit on every upgrade. [01:20:20] Well, it's JSON and will probably remain reasonably backwards compatible [01:20:26] (I hope) [01:20:28] <^demon> Pfft, no it doesn't. [01:20:32] Oh, I had no idea [01:20:44] <^demon> They're completely willing to change it since it's not a public API. [01:20:52] So I guess that's why the OpenStack guys said that Gerrit should have a public API [01:20:58] <^demon> Yes ;-) [01:21:02] <^demon> There are bugs for this. [01:21:48] http://phabricator.wmflabs.org/xhpast/view/1/ [01:22:05] hah, nice [01:22:16] <^demon> I also want http://code.google.com/p/gerrit/issues/detail?id=861 really really bad. [01:22:31] ^demon: Hey, now that you're here [01:22:39] <^demon> Hmm? [01:22:55] I have a project and I was hoping I could get you to help me with it, or take it over from me. Because the time in which I'd do it is like doubly hypothetical [01:23:06] I set up a Gerrit VM and a Jenkins VM [01:23:26] I installed the OpenStack fork of the Gerrit Trigger Plugin in the Jenkins VM, and got it to work [01:23:39] Currently the only job that's in there is a recursive php -l check [01:24:01] But ideally we'd port our entire hook infrastructure (IRC logging, RT ticket updates, future BZ integration) to Jenkins [01:24:39] Oh and of course refactor the Gerrit puppet class so it's not WMF-specific (you can't even use it on labs right now), and figure out wtf is going on with the two Jenkins puppetizations [01:24:57] (one of which is broken for new installs, because the installing-the-package statement is commented out) [01:27:45] <^demon> (sorry, kind of urgent wm2012 registration issue came up, and since we're live...) [01:30:00] Why are we so set on this gated trunk model that other options aren't considered? Have I just missed the boat on that discussion? [01:35:18] Specifically, Gerrit has automated gated trunk [01:35:41] I personally like that a lot, Ryan and Chad do too I think, and all of the OpenStack projects use it [01:36:03] <^demon> And android, obviously ;-) [01:36:36] Well yeah [01:37:04] But since they wrote Gerrit and its UI, I don't fully believe that's the best example ever :P [01:38:14] <^demon> But google's hardly the first people to pioneer "push-for-review" CR [01:39:11] Sure [01:39:15] I like that model [01:39:34] It's really unfortunate that Gerrit's model is awesome, its backend is good, and its frontend is garbage [01:41:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [01:41:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [01:41:38] <^demon> Put the usability initiative on them ;-) [01:42:02] What usability initiative? [01:42:19] <^demon> Re-form it :p [01:42:30] You mean half the VisualEditor team, one analytics guy, and two people that left? :D [01:42:45] <^demon> Yep, that's the team [01:44:29] Alright, dinner [02:11:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [02:11:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [02:41:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [02:41:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [02:43:52] New review: Jeremyb; "I only repushed commit this so that the other change dzahn approved (I75990998fe540b19301d91d9025ee1..." [operations/puppet] (test) C: 0; - https://gerrit.wikimedia.org/r/2012 [03:11:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [03:11:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [03:41:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [03:41:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [04:11:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [04:11:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [04:41:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [04:41:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [05:11:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [05:11:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [05:41:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [05:41:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [06:09:45] PROBLEM host: deployment-sql is DOWN address: deployment-sql CRITICAL - Host Unreachable (deployment-sql) [06:11:05] RECOVERY host: deployment-sql is UP address: deployment-sql PING OK - Packet loss = 0%, RTA = 0.89 ms [06:11:05] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [06:11:15] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [06:24:53] PROBLEM Total Processes is now: CRITICAL on deployment-transcoding deployment-transcoding output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:25:23] PROBLEM dpkg-check is now: CRITICAL on deployment-transcoding deployment-transcoding output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:26:53] PROBLEM Current Load is now: CRITICAL on deployment-transcoding deployment-transcoding output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:27:33] PROBLEM Current Users is now: CRITICAL on deployment-transcoding deployment-transcoding output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:28:13] PROBLEM Disk Space is now: CRITICAL on deployment-transcoding deployment-transcoding output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:28:33] PROBLEM Free ram is now: CRITICAL on deployment-transcoding deployment-transcoding output: CHECK_NRPE: Error - Could not complete SSL handshake. [06:41:13] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [06:41:43] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [06:49:51] Ohai Ryan_Lane [06:50:05] howdy [06:50:07] Are you free to set me up a Labs account? [06:50:11] sure [06:50:18] !account-questions | Hydriz [06:50:18] Hydriz: I need the following info from you: 1. Your preferred wiki user name. This will also be your git username, so if you'd prefer this to be your real name, then provide your real name. 2. Your SVN account name, or your preferred shell account name, if you do not have SVN access. 3. Your preferred email address. [06:50:29] ok... lets me type them [06:50:45] 1. Hydriz 2. hydriz 3. admin@wikisorg.tk [06:50:56] yeah, I don't have SVN [06:51:03] * Ryan_Lane nods [06:51:11] but I am working on an extension that would be merged later [06:51:16] into WikimediaIncubator [06:51:21] ah. cool. which extension? [06:51:25] PrefixExport [06:51:28] very very beta [06:51:30] heh [06:51:33] so no docs yet [06:51:33] what's it do? [06:51:34] :P [06:51:46] Give it a prefix and it will export the pages in it [06:51:51] very useful for the Incubator [06:52:04] so that we can export created test wikis and import them to the new wiki [06:52:05] what should i write here for something to show up at https://labsconsole.wikimedia.org/wiki/Nova_Resource:Deployment-prep [06:52:27] in the SAL? [06:53:08] just a question: Is the shell account something like the Toolserver's shell? [06:53:18] it's a little different [06:53:28] but we can still use it to do things to our instances? [06:53:34] !initial-login | Hydriz [06:53:34] Hydriz: https://labsconsole.wikimedia.org/wiki/Access#Initial_log_in [06:53:40] * Hydriz feels dumb not knowing these [06:53:57] use toolserver? [06:54:03] yeah [06:54:08] I'm not sure if toolserver is accessible from labs [06:54:14] I'll have to look into that [06:54:17] wait, did you just set up an account for me? [06:54:20] yep [06:54:27] those directions let you know how to log in [06:54:38] oh thanks! :) [06:54:42] yw [06:54:48] * Hydriz hugs Ryan_Lane [06:55:36] have you used Amazon EC2 or Rackspace Cloud? [06:55:58] labs works in some ways like those services [06:56:12] 01/24/2012 - 06:56:12 - Creating a home directory for hydriz at /export/home/bastion/hydriz [06:56:15] oh I see [06:56:25] you can either be added to an existing project, or pick out a project to work on [06:56:28] BTW is the public key publically viewable? [06:56:44] it seems you have one already, so I'll likely create a labs project for you [06:56:49] it can be [06:56:55] oh, I see [06:57:11] 01/24/2012 - 06:57:10 - Updating keys for hydriz [06:57:16] it's possible to get public IPs, but only if something is ready to be demo'd [06:57:37] I see I see [06:57:43] we're working on easier ways to make web services publicly viewable [06:58:01] so, in your project, you can create instances [06:58:08] !terminology [06:58:08] https://labsconsole.wikimedia.org/wiki/Terminology [06:58:10] having an account on gerrit doesn't mean being able to commit to it, right? [06:58:20] you can commit to anything in gerrit [06:58:42] and it would be merged later? [06:58:58] things that go into gerrit get code reviewed [06:59:07] if it passes review it'll be merged in [06:59:09] cool [06:59:57] so, an instance is a virtual machine. you'll need to create one, and install any necessary software on it [07:00:35] unfortunately right now we don't have an automated install of mediawiki [07:00:43] So it says that I need to be part of the sysadmin group? [07:00:56] I haven't created a project for you yet [07:01:06] !access [07:01:06] https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [07:01:33] yeah, I read the documentation, though not much in detail [07:01:33] I added you to the bastion project, but not as a sysadmin or netadmin [07:01:37] it gives you access to an instance called bastion.wmflabs.org [07:02:19] that instance lets you log into instances in any project you are a member of [07:02:52] yes, I am logged in now [07:03:00] should I call your project prefixexport? [07:03:33] hmm [07:03:43] I don't really want to just limit myself to doing that [07:03:46] ok [07:03:59] what do you suggest? [07:04:11] hm. [07:04:26] I would go for something generic, but has something related to me [07:04:36] is it possible to just put my username as the project? [07:05:16] well, we try to name projects after specific things, as it encourages people to work together [07:05:21] ok then [07:05:26] lets just name it incubator [07:05:27] we can make other projects as well [07:05:33] incubator is a good idea [07:05:44] then other people working on incubator projects can join in [07:05:55] yeah [07:05:58] we have a team :P [07:06:21] but I can't create projects myself right? [07:06:26] right [07:06:37] only wikimedia ops team can create projects [07:06:45] oh, I see [07:06:49] but, you can add anyone you want to this project [07:06:50] thats good too, to prevent spam [07:06:57] and anyone in any other project can add you to them [07:07:07] seems quite social [07:07:18] 01/24/2012 - 07:07:18 - Creating a project directory for incubator [07:07:18] 01/24/2012 - 07:07:18 - Creating a home directory for hydriz at /export/home/incubator/hydriz [07:07:18] 01/24/2012 - 07:07:18 - Creating a home directory for laner at /export/home/incubator/laner [07:07:19] yeah, it's intended to be fairly open [07:07:51] we're automating as much as possible, as well [07:08:00] so, some things about being in a project [07:08:13] * Hydriz listens [07:08:21] 01/24/2012 - 07:08:21 - Updating keys for hydriz [07:08:21] 01/24/2012 - 07:08:21 - Updating keys for laner [07:08:24] we follow the procedures of the operations team and the development team [07:08:46] which means projects should log their actions, and document their project [07:08:56] code should ideally go in code review [07:09:09] projects should puppetize their instances [07:09:27] we can help you out with this stuff [07:09:37] okay :) [07:09:44] we'll be happy to walk you through it a couple times [07:10:03] !logging [07:10:03] To log a message, use the following format: !log [07:10:08] this is how you log [07:10:16] yep, its on the main page [07:10:26] !log incubator created the incubator project. Added Hydriz as project owner. [07:10:26] Logged the message, Master [07:10:38] !project incubator [07:10:38] https://labsconsole.wikimedia.org/wiki/Nova_Resource:incubator [07:10:45] logging keeps the project activity open I suppose? [07:10:55] your project page shows the log [07:11:13] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [07:11:14] nope, it just lets others know what you are up to [07:11:28] it makes it easier to work together on a project [07:11:35] yep [07:11:43] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [07:12:06] there's also a combined server admin log [07:12:07] !SAL [07:12:08] https://labsconsole.wikimedia.org/wiki/Server_Admin_Log [07:13:06] yeah, just looks weird that my project is at the top [07:13:25] it's sorted by the most recently changed SAL [07:14:04] the recent changes also show what's going on [07:14:35] as it'll show project creations, instance creations, instance deletions, etc [07:15:00] so, how do we create an instance? [07:15:15] on the sidebar are all of the available actions [07:15:21] "Manage instances" [07:15:25] yes [07:15:33] in that, add an instance in your project [07:15:49] generally don't choose tiny instances [07:15:57] m1.small is usually a good first instance [07:16:15] what is the difference between m1 and s1? [07:16:29] s1 instances are storage [07:17:34] then for the instance name, is there any method in naming the instance? [07:17:47] try to be descriptive [07:17:53] instance names must be unique [07:18:17] okay, so what about the availability zone? [07:18:37] either is fine [07:18:43] it's a bug [07:18:48] I see [07:18:56] so I can just leave the rest as it is? [07:19:11] ah. crap. forgot to mention something [07:19:16] !security-groups [07:19:16] https://labsconsole.wikimedia.org/wiki/Security_Groups [07:19:26] security groups are firewall rules [07:19:42] you must create security groups before instances [07:20:17] but there seems to be a default enabled [07:20:21] so, if your instance is going to need web access, you'll need to create a security group [07:20:47] there's a default, but if you create other instances that shouldn't have web open, it would be a problem [07:20:55] default is applied to all instances in a project [07:21:32] once you create an instance, you can never change its security groups [07:22:04] this is a limitation with the api we're using :( [07:22:42] But for now I can use the default group for allowing the public to view? [07:22:58] no. you should create a security group [07:23:58] is there any docs about the rules? [07:24:02] yep [07:24:05] !security-groups [07:24:05] https://labsconsole.wikimedia.org/wiki/Security_Groups [07:24:18] in general you want to ignore the group rules [07:24:29] you want to use an individual rule [07:25:12] so, to open port 80, you should use from 80 to 80 protocol tcp CIDR ranges 0.0.0.0/0 [07:25:41] so these ports is the port we access from the browser? [07:25:52] yep [07:25:55] 80 and 443, generally [07:26:16] so, what is the protocol about? [07:26:33] you'll need to use a socks-proxy to get to your web server, until you get a public ip [07:27:04] the protocol is tcp, udp or icmp. different applications use different protocols [07:27:13] http uses tcp [07:27:22] dns uses tcp and udp [07:27:44] I see [07:27:53] so, I can just have one rule? [07:28:01] yeah [07:28:08] unless you want https, then you need for 443 [07:28:26] * Hydriz apologises for asking questions that is too simple for everyone [07:28:31] it's ok [07:29:45] !log incubator Created new security group for web access to port 80 [07:29:46] Logged the message, Master [07:30:21] great. have fun. let me know if you have any questions [07:30:30] :) [07:30:34] note that when you create an instance, it takes a little while to build [07:30:42] So, how do we access the new instance from the web? [07:31:10] if you look at the console log you'll see it's done by waiting fo "finished puppet run" [07:31:12] *for [07:31:24] it takes 3-5 minutes [07:31:31] !socks-proxy [07:31:31] see https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances ssh @bastion.wmflabs.org -D [07:31:47] you can access it via a socks proxy [07:31:53] okie [07:32:17] !socks-proxy del [07:32:17] Successfully removed socks-proxy [07:32:52] !socks-proxy is https://labsconsole.wikimedia.org/wiki/Access#Accessing_web_services_using_a_SOCKS_proxy ssh @bastion.http://wmflabs.org/ -D [07:32:52] Key was added! [07:32:52] * Ryan_Lane pokes wm-bot [07:35:24] PROBLEM host: prefixexport is DOWN address: prefixexport PING CRITICAL - Packet loss = 100% [07:35:25] oh and BTW is there any limitation about what we can do on the shell server? [07:36:07] no copyrighted material, no hacking, no warez, etc. [07:36:15] yes bots? [07:36:29] if you want to run a bot, you should join the bot project [07:36:40] Oh I see [07:36:48] Just finding out how it works haha :P [07:36:55] volunteers are building a bot infrastructure there [07:37:03] ? [07:37:09] a place to run bots [07:37:21] that has a lot of things already done for you [07:37:48] so it is possible for me to join it? [07:38:01] yes, talk to some of the project members. [07:38:07] ah I see [07:38:13] thanks for all your help! [07:38:25] the project's talk page may be a good place for it [07:38:31] or the labs-l list [07:38:40] yw [07:40:20] labs-l? [07:40:27] the mailing list [07:40:28] ah Isee [07:40:31] *I see [07:40:44] https://lists.wikimedia.org/mailman/listinfo/labs-l [07:41:14] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [07:41:44] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [07:41:56] but I thought the -l suffix is going away? [07:42:09] is it? [07:42:17] I hadn't heard of this [07:42:20] I heard from someplace [07:42:31] that they are trying to avoid using it [07:42:34] not sure [07:42:34] I rarely make lists, so I hadn't heard [07:43:26] Isee [07:59:10] connection refused, FARK [08:03:41] !log incubator Created new instance prefixexport [08:03:42] Logged the message, Master [08:06:04] PROBLEM host: prefixexport is DOWN address: prefixexport PING CRITICAL - Packet loss = 100% [08:11:14] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [08:11:44] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [08:36:04] PROBLEM host: prefixexport is DOWN address: prefixexport PING CRITICAL - Packet loss = 100% [08:41:14] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [08:41:44] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [08:55:53] !log incubator Deleted instance prefixexport, recreating to fix security groups issue [08:55:54] Logged the message, Master [09:02:31] !log incubator Created new instance i-00000107 for testing PrefixExport extension [09:02:32] Logged the message, Master [09:06:33] PROBLEM host: prefixexport is DOWN address: prefixexport CRITICAL - Host Unreachable (prefixexport) [09:11:43] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [09:12:03] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [09:26:22] !log incubator Installed PHP5, Apache2 and Mysql to prefixexport [09:26:23] Logged the message, Master [09:26:24] !log incubator SVN Checked-Out 1.18wmf1 into /var/www/w [09:26:25] Logged the message, Master [09:26:48] !sal [09:26:48] https://labsconsole.wikimedia.org/wiki/Server_Admin_Log see it and you will know all you need [09:29:52] god damn it how do you make your instance publicly available [09:33:30] RECOVERY host: prefixexport is UP address: prefixexport PING OK - Packet loss = 0%, RTA = 3.12 ms [09:37:00] PROBLEM Total Processes is now: CRITICAL on prefixexport prefixexport output: CHECK_NRPE: Error - Could not complete SSL handshake. [09:37:40] PROBLEM dpkg-check is now: CRITICAL on prefixexport prefixexport output: CHECK_NRPE: Error - Could not complete SSL handshake. [09:39:00] PROBLEM Current Load is now: CRITICAL on prefixexport prefixexport output: CHECK_NRPE: Error - Could not complete SSL handshake. [09:39:50] PROBLEM Current Users is now: CRITICAL on prefixexport prefixexport output: CHECK_NRPE: Error - Could not complete SSL handshake. [09:40:10] PROBLEM Disk Space is now: CRITICAL on prefixexport prefixexport output: CHECK_NRPE: Error - Could not complete SSL handshake. [09:40:40] PROBLEM Free ram is now: CRITICAL on prefixexport prefixexport output: CHECK_NRPE: Error - Could not complete SSL handshake. [09:42:00] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [09:42:20] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [10:12:00] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [10:12:20] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [10:24:58] Hydriz, what do you want? [10:25:07] ? [10:26:20] you had problems? [10:26:29] yeah [10:26:38] but somewhat resolved [10:27:02] but I was wondering how we can access the instance from "around the world" [10:33:47] get it an external IP and DNS record [10:34:15] hmm [10:34:38] I am googling around to find out how to connect the IP and the virtual host [10:34:59] but isn't the public ip available? [10:42:00] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [10:42:20] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [11:12:00] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [11:12:20] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [11:24:44] PROBLEM Current Load is now: CRITICAL on deployment deployment output: Connection refused by host [11:25:04] PROBLEM Current Users is now: CRITICAL on deployment deployment output: Connection refused by host [11:25:44] PROBLEM Disk Space is now: CRITICAL on deployment deployment output: Connection refused by host [11:26:24] PROBLEM Free ram is now: CRITICAL on deployment deployment output: Connection refused by host [11:27:44] PROBLEM Total Processes is now: CRITICAL on deployment deployment output: Connection refused by host [11:28:24] PROBLEM dpkg-check is now: CRITICAL on deployment deployment output: Connection refused by host [11:42:04] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [11:42:34] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [11:53:52] !access [11:53:53] https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [12:12:04] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [12:12:34] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [12:29:49] ACKNOWLEDGEMENT Current Load is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:30:33] ACKNOWLEDGEMENT Current Users is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:34:13] PROBLEM Current Users is now: CRITICAL on miniswarm miniswarm output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:35:03] PROBLEM Free ram is now: CRITICAL on miniswarm miniswarm output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:35:13] PROBLEM Disk Space is now: CRITICAL on miniswarm miniswarm output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:36:23] PROBLEM Total Processes is now: CRITICAL on miniswarm miniswarm output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:37:43] PROBLEM dpkg-check is now: CRITICAL on miniswarm miniswarm output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:38:23] PROBLEM Current Load is now: CRITICAL on miniswarm miniswarm output: CHECK_NRPE: Error - Could not complete SSL handshake. [12:42:13] PROBLEM host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [12:43:13] PROBLEM host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [12:44:33] ACKNOWLEDGEMENT host: patchtest is DOWN address: patchtest CRITICAL - Host Unreachable (patchtest) [12:44:48] ACKNOWLEDGEMENT host: patchtest2 is DOWN address: patchtest2 CRITICAL - Host Unreachable (patchtest2) [12:54:53] RECOVERY Total Processes is now: OK on deployment-transcoding deployment-transcoding output: PROCS OK: 81 processes [12:55:23] RECOVERY dpkg-check is now: OK on deployment-transcoding deployment-transcoding output: All packages OK [12:56:53] RECOVERY Current Load is now: OK on deployment-transcoding deployment-transcoding output: OK - load average: 0.03, 0.07, 0.05 [12:57:33] RECOVERY Current Users is now: OK on deployment-transcoding deployment-transcoding output: USERS OK - 1 users currently logged in [12:58:33] RECOVERY Free ram is now: OK on deployment-transcoding deployment-transcoding output: OK: 72% free memory [13:08:53] PROBLEM Disk Space is now: WARNING on deployment-transcoding deployment-transcoding output: DISK WARNING - free space: / 51 MB (3% inode=55%): [13:13:33] ACKNOWLEDGEMENT Disk Space is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:14:03] ACKNOWLEDGEMENT Current Load is now: CRITICAL on prefixexport prefixexport output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:14:18] ACKNOWLEDGEMENT Free ram is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:14:33] ACKNOWLEDGEMENT Free ram is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:14:48] ACKNOWLEDGEMENT Current Load is now: CRITICAL on labs-relay labs-relay output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:15:03] ACKNOWLEDGEMENT Current Users is now: CRITICAL on labs-relay labs-relay output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:15:03] ACKNOWLEDGEMENT Disk Space is now: CRITICAL on labs-relay labs-relay output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:15:18] ACKNOWLEDGEMENT Total Processes is now: CRITICAL on labs-realserver labs-realserver output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:15:33] ACKNOWLEDGEMENT Total Processes is now: CRITICAL on labs-relay labs-relay output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:46:09] !access [13:46:09] https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [13:53:14] !log deployment-prep reconfigured new instance and fixed some issues on puppet, no logs in sal regarding it [13:53:15] Logged the message, Master [13:55:20] hello [13:55:27] hi [14:21:02] eh [14:21:23] what is patchtest? [14:21:25] what do you guys usually talk in here? [14:21:36] moz`deph: labs? :) [14:21:46] it is my first time joining here on freenode network [14:21:51] ah [14:21:54] and wikimedia-labs [14:22:00] this is channel for labs [14:22:07] we usually don't talk about anything else [14:22:23] labs? [14:22:26] if you need to help with wikipedia you should visit #wikipedia [14:22:36] moz`deph: wikimedia labs [14:22:48] what is it about? [14:22:52] *curious* [14:22:54] !labs [14:22:54] https://labsconsole.wikimedia.org/wiki/$1 [14:22:56] :p [14:22:59] ah, bad one [14:23:02] see topic [14:23:05] there is a link [14:23:24] oh ok [14:23:27] @search labs [14:23:27] Results (found 30): instance, git, nagios, labs, monitor, alert, instancelist, instance-json, amend, sal, security, sudo, access, keys, stucked, group, pathconflict, terminology, nova-resource, bastion, ryanland, initial-login, SAL, manage-projects, rights, new-labsuser, cs, puppet, projects, socks-proxy, [14:23:49] @regsearch labs [14:23:49] Results (found 30): instance, git, nagios, labs, monitor, alert, instancelist, instance-json, amend, sal, security, sudo, access, keys, stucked, group, pathconflict, terminology, nova-resource, bastion, ryanland, initial-login, SAL, manage-projects, rights, new-labsuser, cs, puppet, projects, socks-proxy, [14:23:54] @regsearch [Ll]abs [14:23:54] Results (found 33): instance, git, nagios, labs, monitor, alert, instancelist, instance-json, amend, sal, info, security, sudo, access, keys, bots, stucked, group, pathconflict, terminology, nova-resource, bastion, ryanland, initial-login, SAL, manage-projects, rights, new-labsuser, cs, puppet, projects, labs-project, socks-proxy, [14:24:07] can't find a link there [14:24:10] it's in topic [14:24:28] !wl is https://www.mediawiki.org/wiki/WMF_Projects/Wikimedia_Labs [14:24:28] Key was added! [14:24:37] !about alias wl [14:24:37] Successfully created [14:24:44] !about [14:24:44] https://www.mediawiki.org/wiki/WMF_Projects/Wikimedia_Labs [14:24:46] here [14:24:55] !wl del [14:24:55] Successfully removed wl [14:25:12] !wl is https://www.mediawiki.org/wiki/Wikimedia_Labs here you can find more [14:25:12] Key was added! [14:25:49] moz`deph: maybe if you tell me what you need I can help you [14:37:20] !sal [14:37:20] https://labsconsole.wikimedia.org/wiki/Server_Admin_Log see it and you will know all you need [14:37:23] !SAL [14:37:23] https://labsconsole.wikimedia.org/wiki/Server_Admin_Log [14:37:25] :o [14:37:31] !SAL del [14:37:31] Successfully removed SAL [14:38:40] hi Sumana [14:38:47] hi petan|wk [14:39:19] petan|wk: have you ever heard of Doodle? http://www.doodle.com/ [14:39:37] you might find it useful for scheduling meetings [14:39:53] ah [14:39:54] cool [14:40:35] yeah, it's a big innovation, compared to manually asking people what times are ok for them [14:41:22] so people can vote for a time? or how [14:41:29] I need to find out how it work [14:41:31] yeah [14:41:34] it's a poll [14:41:44] you tell it the times/dates that you want people to choose from [14:41:52] and then people checkbox the form [14:42:43] my email was rather a proposal to create a meeting, I think I should disucuss it with Ryan... before using doodle [14:43:02] ok, just wanted you to know about the tool [14:43:03] for eventual use [14:43:17] ok [14:44:06] I need to know if he is available during weekend too [14:44:18] but I would prefer work day [14:44:33] best of luck [14:44:37] :) [14:59:47] !log incubator Created new instance "deployment" to test out beta code before final push to the Wikimedia Incubator [14:59:48] Logged the message, Master [15:04:23] PROBLEM Current Load is now: CRITICAL on deployment deployment output: CHECK_NRPE: Error - Could not complete SSL handshake. [15:05:03] PROBLEM Disk Space is now: CRITICAL on deployment deployment output: CHECK_NRPE: Error - Could not complete SSL handshake. [15:05:13] PROBLEM Current Users is now: CRITICAL on deployment deployment output: CHECK_NRPE: Error - Could not complete SSL handshake. [15:05:43] PROBLEM Free ram is now: CRITICAL on deployment deployment output: CHECK_NRPE: Error - Could not complete SSL handshake. [15:07:03] PROBLEM Total Processes is now: CRITICAL on deployment deployment output: CHECK_NRPE: Error - Could not complete SSL handshake. [15:07:33] PROBLEM dpkg-check is now: CRITICAL on deployment deployment output: CHECK_NRPE: Error - Could not complete SSL handshake. [15:11:07] hi Hydriz [15:11:17] hello [15:11:33] are you pinging me for the above? [15:11:55] right, I can help you with this :) [15:12:06] you need to do puppetd -tv on all instances you created [15:12:10] * run [15:12:13] okie [15:12:16] will do so now [15:12:19] that will insert them to nagios properly [15:12:44] hi hexmode [15:12:50] received your mail [15:12:58] petan|wk: ! [15:13:11] hexmode: someone created a new instance [15:13:18] and BTW petan|wk: how do you create instances that can actually show up in say, bots.wmflabs.org? [15:13:20] this someone want to know we have more bots [15:13:21] :D [15:13:33] Hydriz: you need to allocate public ip [15:13:34] petan|wk: do you have time in about 45 min to talk about status and such? [15:13:40] yes [15:13:44] yeah, I don't see any docs about it [15:13:49] maybe with small interrupts but sure [15:13:55] * Hydriz is a newbie in this, sorry! [15:13:58] Hydriz: I will make some [15:14:06] :) [15:14:15] we don't have docs yet [15:14:21] petan|wk: mdale and others in WMF want to use beta to develop some new features [15:14:26] ok [15:14:30] This is what I call a success :) [15:14:33] that's what I thought beta is for :) [15:14:51] I was actually about to deploy all stuff waiting in deployment queue there [15:15:04] before john duhart slapped me for that :o [15:15:18] I tried to do allocate ip, but it always fails [15:15:33] "Failed to allocate new public IP address." [15:15:44] Hydriz: because you don't have any ip assigned for your project [15:15:53] well, true [15:15:53] Hydriz: you need to request it [15:16:01] how? [15:16:10] Hydriz: maybe, state a reason for that? you can request it in this channel [15:16:25] is this project supposed to be accessed from outside? [15:16:28] well, this project is for the incubator [15:16:33] petan|wk: right, but first we need to get 1.19 out ... I think johnduhart whas right on this. [15:16:38] because if it's only for devs or people from labs, they all can access it [15:16:50] yes, indeed [15:16:51] getting 1.19 out is first priority [15:16:54] ok [15:16:57] no, it would be better if we can access it from outside [15:16:59] so WMF people should wait? [15:17:04] Hydriz: ok [15:17:07] (I think) [15:17:18] mutante: can you give him ip? ^ [15:17:31] but this public ip is only given to one instance... [15:17:36] yes [15:17:55] so thats quite troublesome [15:18:02] Hydriz: huh? [15:18:11] why you need to use more ip's? [15:18:12] petan|wk: yes, mdale is just staging some stuff, it shouldn't interfere with 1.19 [15:18:17] like, we have to consistently trouble you guys to set up ips [15:18:17] ah ok [15:18:26] lol no [15:18:59] if possible, just assign me an ip for the instance "deployment" of the Incubator project [15:19:20] hexmode: I don't really care :) it's no problem to deploy it, it would be more fun [15:19:25] to have more testing sw there [15:19:40] btw regarding wikidiff [15:19:46] I already compiled latest version [15:19:55] hexmode: ^ you might overwrite it using aptitude [15:19:55] I would be away for now, have to take a bath [15:20:02] ok [15:20:11] petan|wk: Right but aptitude had a later version [15:20:31] petan|wk: and Ryan helped me figure out how to fix the diffs [15:20:58] petan|wk: also, I didn't realize that it was in aptitude [15:21:28] in time I was compiling it there was still old one [15:21:37] so probably it was packaged now [15:21:54] I comitted some changes to that extension so now it should be very easy to build it [15:22:05] oh, one more thing petan|wk: did you get the mw gadgets installed for that wiki I sent you an email about this w/e? [15:22:26] oh I will check it, but probably yes [15:22:43] hexmode: so wikidiff is fixed? [15:22:54] I really miss stuff in sal :) [15:22:56] yep :) [15:23:00] sal? [15:23:02] there is a new instance but no line about it [15:23:03] !sal [15:23:03] https://labsconsole.wikimedia.org/wiki/Server_Admin_Log see it and you will know all you need [15:23:13] whoever created it, didn't log it [15:23:32] ah, yes I did log the apt update, IIRC, though [15:23:43] yes, I know [15:23:50] that's how I know that [15:23:54] I actually read it :) [15:23:56] :) [15:24:08] but I don't know what is new instance I know it from nagios that it exist :D [15:24:25] yeah, that should be logged [15:24:49] ok, no problem once I find out who created I will hopefully find out what is it for :) [15:25:00] I mounted nfs there for now [15:25:06] ping mdale when he comes on [15:25:09] ok [15:25:14] or I'll get him to [15:26:56] :) [15:27:25] problem with that instance is that it only has 1 gb hdd [15:27:32] I thought there was a redlink to the documentation somewhere on labsconsole [15:27:34] so it's full now even when nothing is there [15:27:49] hm, I don't like to have it on labs console, but ok... [15:28:10] if we made it on mww people could improve it, even those without access to labs [15:28:11] oh, you were going to use labs.beta? [15:28:16] correct typos etc [15:28:18] bonus! [15:28:28] no I mean, mediawiki.org [15:28:38] ah, yeah [15:28:39] I think it would be best to have all documentation on one wiki [15:28:39] k [15:28:44] back [15:28:46] sure [15:28:47] unfortunatelly no idea which one [15:28:49] any updates? [15:29:07] I moved some documentations I made from enwiki to mediawiki.org because I was told it's a best one for that [15:29:26] labs.beta makes sense. But if we aim to make this a long running project, then mw.o is good [15:29:32] better even [15:29:41] depends, I wanted to make documentation even for labsconsole [15:29:49] because new people look confused [15:29:51] petan|wk: talk more in 30 min... got a meeting [15:29:54] ok [16:10:19] Hydriz: I started Help:Contents [16:10:28] hmm [16:10:30] check it [16:10:54] yeah, good start to documentation :P [16:11:00] heh [16:11:05] but it will take a while though [16:11:13] yes it will [16:11:25] I am still trying to get myself familiar with things [16:11:43] * Hydriz loves to write documentation [16:11:46] :P [16:12:44] the access page itself is horrible [16:12:51] like, some steps I tried doesn't work [16:12:52] feel free to improve it [16:13:00] yes because it changes by time [16:13:09] it's obsolete [16:13:45] true that [16:14:03] but still I am trying things out [16:14:17] like, how to connect a domain to the server via the public IP [16:14:19] :P [16:14:40] !help [16:14:40] want docs? ask for "!wm-bot". all keywords? try "@regsearch .*" [16:15:04] bot knows a lot [16:15:14] just do @regsearch [16:15:24] @regsearch dns [16:15:24] No results found! :| [16:15:27] @regsearch domain [16:15:27] No results found! :| [16:15:29] bah [16:15:32] @regsearch ip [16:15:32] Results (found 4): wiki, epad, bastion, projects, [16:15:51] lol [16:16:14] @regsearch .* [16:16:14] Results (found 81): instance, morebots, git, bang, nagios, bot, labs-home-wm, labs-nagios-wm, labs-morebots, gerrit-wm, wiki, labs, extension, wm-bot, putty, gerrit, change, revision, monitor, alert, password, unicorn, help, bz, os-change, instancelist, instance-json, leslie's-reset, damianz's-reset, amend, credentials, queue, sal, info, security, logging, ask, sudo, access, $realm, keys, $site, bug, pageant, blueprint-dns, bots, stucked, rt, pxe, ghsh, group, pathconflict, terminology, etherpad, epad, nova-resource, pastebin, newgrp, osm-bug, Ryan, bastion, ryanland, afk, test, initial-login, account-questions, manage-projects, rights, new-labsuser, cs, puppet, new-ldapuser, projects, quilt, labs-project, openstack-manager, wikitech, load, load-all, socks-proxy, wl, [16:16:44] !bang [16:16:44] Bang!! [16:16:48] ... [16:16:56] !domain is in case you want to assign a domain to your ip, you can use manage addresses to do that [16:16:56] Key was added! [16:17:39] @trusted [16:17:39] I trust: petan!.*@wikimedia/Petrb (2admin), .*@wikimedia/.* (2trusted), .*@mediawiki/.* (2trusted), .*@wikimedia/Ryan-lane (2admin), .*@wikipedia/.* (2trusted), .*@nightshade.toolserver.org (2trusted), [16:17:57] cool, so I can also contribute [16:17:59] :) [16:18:06] yup [16:18:59] ;) [16:19:02] ah, I just remembered about the bots project [16:19:11] can I be part of it? :P [16:19:56] yes [16:20:04] !log bots Adding hydriz to project [16:20:05] Logged the message, Master [16:20:09] petan|wk: back [16:20:14] heh :P [16:20:21] thanks :) [16:20:35] hexmode: cool [16:21:08] 01/24/2012 - 16:21:08 - Creating a home directory for hydriz at /export/home/bots/hydriz [16:22:09] 01/24/2012 - 16:22:09 - Updating keys for hydriz [16:22:16] ok [16:22:18] Hydriz: done [16:22:21] ah great [16:22:31] I see some ideas about linking the instances to domains [16:22:37] so, there are 4 application servers and 3 sql servers now [16:22:53] you are supposed to use bots-4 application server because that one is empty now [16:23:05] yeah [16:23:05] you can create unlimited number of databases on any db server [16:23:10] I see lots of instances [16:23:16] keep in mind this is a testing project only and not very stable [16:23:16] * Hydriz is dazzled [16:23:31] you should log everything you change [16:23:59] in case you want to install any package etc [16:24:15] nah, not really [16:24:19] feel free to ping me for any help [16:24:22] ok [16:24:26] I try to, as much as possible, have bots on the Toolserver [16:24:32] right [16:24:33] though I actually don't run bots [16:25:26] hexmode: wikidiff is ok now? [16:25:39] hexmode: anything I can help with :o [16:25:59] http://labs.wikimedia.beta.wmflabs.org/w/index.php?title=Test_cases&diff=prev&oldid=112 [16:26:15] petan|wk: ^^ was worse before [16:26:29] looks ok [16:26:32] right [16:26:34] now only highlights the chars that are different [16:26:46] before was words or lines [16:27:08] ah [16:28:08] k [16:28:42] ok, see you guys another day [16:28:45] got to sleep now [16:30:43] petan|wk: could you import the gadgets from https://te.wikipedia.org/wiki/Special:Gadgets ? [16:30:49] sure [16:31:01] petan|wk: to tewiki.beta, of course [16:31:04] :) [16:35:12] doing [16:42:03] PROBLEM Free ram is now: WARNING on bots-cb bots-cb output: Warning: 12% free memory [16:42:22] Damianz: ^ [16:47:03] RECOVERY Free ram is now: OK on bots-cb bots-cb output: OK: 25% free memory [16:55:10] Ouch [16:56:13] Going to improve a couple of the cbng parts probably tomorrow which should help the random spikes in usage. Kinda busy tonight as I've got maintaince for work planned D: [16:57:01] brb [16:57:05] heading home [16:57:05] :o [16:58:02] :o [17:01:36] Hmm time to go climbing soon :) [17:45:41] ok [18:01:45] petan: could you update http://etherpad.wikimedia.org/DeploymentPrep with the current status and any TODOs that remain? [18:43:06] petan|wk: petan: around? [19:45:46] OrenDsk: around? [20:27:54] hexmode: yes [20:28:12] raindrift: hi [20:28:48] hexmode: only search is missing now [20:34:34] re [20:38:07] petan: just got back \o/ [20:38:28] OrenDsk: any search progress since we last talked? [20:39:10] I've created search3 and put it into continuous integration [20:40:06] hexmode: k [20:40:23] OrenDsk: but nothing on beta? Is search3 the next gen search engine? [20:40:31] yes [20:40:44] petan: could you update the etherpad? [20:40:59] * hexmode goes to check [20:41:50] OrenDsk: ok, is it based on the current code? Is it in svn? or tested on integration.mediawiki.org? [20:43:10] yes, yes and yes [20:44:06] curently no solar specific code [20:44:17] hexmode: dunno [20:44:20] I think all is done [20:44:22] :o [20:45:02] OrenDsk: excellent! [20:45:47] I had a chat with CT of ops [20:46:09] petan: ok could you edit the first part where it talks about project scope by updating/removing bit about squid [20:46:32] OrenDsk: really? nice! did you talk to mark, too? [20:46:59] as I understand it will take 6 weeks to puputize search [20:47:04] nope [20:47:38] hexmode: squid is still not finished [20:47:38] k... so we shouldn't worry about old search on beta, then, right? [20:47:41] I am waiting for Ryan [20:47:45] at least not now? [20:47:54] there is one option [20:48:05] my original idea [20:48:09] Ryan_Lane: when will you be able to help w/ squid? [20:48:18] OrenDsk: remind me what that is? [20:48:52] start with a stand alone installation with a local mediawiki with just 10 pages [20:48:58] uh [20:49:01] get that to work and then change it [20:49:03] we have a bit more than 10 [20:49:08] ok [20:49:19] OrenDsk: how complicated would be to change it [20:49:28] OrenDsk: feel free to set up a temporary wiki then [20:49:47] I will create a separate db account ok [20:49:47] ok [20:50:21] changing the config would not be so hard [20:50:36] the local config would be the same [20:51:18] the globalconfig would be based on the one from production [20:51:35] just that there would be initaly one search group [20:52:03] i.e. indexes should be able to live on one machine [20:52:17] once that works [20:52:42] !log deployment-prep created db user oren and new database for temporary wiki [20:52:43] Logged the message, Master [20:52:51] it would be possible to copy just the search deamon + internationlization files + index to a searcher [20:53:03] OrenDsk: want me to set it up? [20:53:09] so long as it has java [20:53:16] hm [20:54:18] the other thing is that the indexer needs to be able to see the other wikis to run maintenece scripts on them [20:55:28] that is no problem [20:55:33] ok [20:55:40] but this temporary acc is for temporary wiki :) [20:55:49] once you get it working ok we move to cluster [20:56:05] sure [20:56:35] once it works you might be able to paputise search by yourself [20:56:48] hm... dunno [20:56:54] I am definitely new to puppet [20:57:04] I was happy that I puppetized mono so far :) [20:57:24] so I think that puppetizing rest of bots is a best I can do now [20:57:58] well no point in counting our eggs before they hatch [21:17:23] petan: make the local wiki a standalone one with a localsettings not a globalsettings [21:18:13] 01/24/2012 - 21:18:13 - Creating a home directory for sara at /export/home/ganglia/sara [21:19:13] 01/24/2012 - 21:19:13 - Updating keys for sara [21:19:41] I love horace silver ... [21:22:02] Ryan_Lane: what's the status on labs account for the pediapress team? [21:22:18] working on that today [21:22:46] ok [21:23:07] Ryan_Lane: Can I watch when you set that up? (If it's interesting?) [21:25:20] andrewbogott: sure [21:43:53] PROBLEM Current Load is now: CRITICAL on aggregator1 aggregator1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [21:45:03] PROBLEM Disk Space is now: CRITICAL on aggregator1 aggregator1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [21:45:03] PROBLEM Current Users is now: CRITICAL on aggregator1 aggregator1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [21:45:16] andrewbogott: ok, want to set up some accounts? [21:45:36] Um... shortly. Trying to get a patch in for e3 [21:45:42] ah. ok [21:45:43] PROBLEM Free ram is now: CRITICAL on aggregator1 aggregator1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [21:47:23] PROBLEM Total Processes is now: CRITICAL on aggregator1 aggregator1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [21:47:33] PROBLEM dpkg-check is now: CRITICAL on aggregator1 aggregator1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [22:50:01] New patchset: Lcarr; "Adding in the sw repo as well as making it available via fileserver.conf" [operations/puppet] (test) - https://gerrit.wikimedia.org/r/2070 [22:50:59] New review: Lcarr; "cherry-picked from production" [operations/puppet] (test); V: 0 C: 2; - https://gerrit.wikimedia.org/r/2070 [22:50:59] Change merged: Lcarr; [operations/puppet] (test) - https://gerrit.wikimedia.org/r/2070 [23:40:53] Ryan_Lane: Is labs using XEN or KVM? Or something else? [23:41:03] we're using KVM [23:41:05] via libvirt [23:41:18] ok, thanks. [23:42:01] KVM ftw [23:43:27] I think KVM doesn't support file injection... writing to the mailing list to see if there are plans to add it. [23:43:38] um... openstack on KVM that is. [23:56:58] oh. crap [23:56:59] really? [23:57:05] well, that would be annoying [23:57:33] I wonder if its something we could stick into the instance's metadata [23:58:24] Yeah, metadata is plan B. [23:58:40] It'll require a custom daemon on the instance, most likely. But that won't be all that hard. [23:58:45] yeah [23:58:53] and instances have cloudinit already [23:59:57] and cloudinit has puppet support in some basic way