[00:01:40] all the IO from the mysql stuff is really making virt2 unhappy [00:23:38] Give it a cookie [07:13:27] petan: good morning [10:23:31] hey [10:23:37] jeremyb [10:23:42] morning :D [18:01:08] And yet another wikimedia channel :-) I guess here it is more on topic: Any ipv6 on labs? [18:08:01] multichill: i think not [18:08:01] inet6 fe80::16:3eff:fe7b:a5cd/64 scope link [18:09:10] petan: hey [18:16:09] hi jeremyb [18:16:20] wanna help with set up of deployment site? [18:16:31] http://deployment.wmflabs.org/auth/wiki/ [18:16:36] you mean hexmode's thing? [18:16:37] I created a central login there [18:16:39] yes [18:16:42] sure [18:16:50] it should be ready by tommorow I guess [18:16:55] I need to clone commons now [18:17:11] create a global account, if it works :D [18:17:13] heh. what's central login? centralauth? [18:17:31] petan: I'm updating your global.php .. there were some notices when I ran update.php [18:17:34] SUL [18:17:40] anything I should be aware of? [18:17:46] oh I know [18:17:47] I will fix it [18:17:56] k [18:17:58] hexmode: I created SUL so you can register one account for all sites [18:18:10] * hexmode takes his hands off [18:18:16] that saved me a lot of time [18:18:19] yes, saw that [18:18:37] because I don't have to register everywhere + we get more production like settings [18:18:51] this is awesome, btw: you will really help us make labs viable :) [18:18:58] I hope [18:19:07] is this a replacement for prototype? [18:19:12] yes [18:19:16] I filled a bug to bz and I used prototype as category [18:19:21] prototype sucks [18:19:23] maybe you should update it [18:19:25] in bz [18:19:36] wmflabs has a product... bug #? [18:19:39] #? [18:19:47] https://bugzilla.wikimedia.org/show_bug.cgi?id=33578 [18:20:07] first people from community complained heh [18:20:16] user scripts don't work [18:20:37] can we make the fp of development point to the SUL wiki? [18:20:59] just redirect there? [18:21:07] sure [18:21:10] fp? [18:21:13] k, I'll do igt [18:21:14] front page [18:21:54] btw I am installing commons atm, hexmode what was the RTL wiki [18:21:59] which language it was [18:22:04] arwiki [18:22:06] ok [18:22:22] I think we could make a configuration clone for all wikis we have [18:22:27] only problem is with content pages [18:23:28] btw hexmode [18:23:37] http://deployment.wmflabs.org/global/viewfile.php?file=settings.php [18:23:38] :) [18:23:48] it can be used for all files like on noc [18:23:55] :) [18:24:00] so people can see what config we use on labs [18:24:08] and compare it with noc [18:24:44] this is so f'n awesome... I'm telling people we're "soft launch" this on Monday [18:24:47] k? [18:24:54] sure [18:24:56] great [18:25:24] where is wgGlobalConfig_Extension used in prod? [18:26:53] it isn't [18:27:00] I created it in a different way I think this is easier [18:27:22] someone said that production config suck a bit [18:27:41] well, what's the prod counterpart? [18:28:34] they have a variable for each [18:28:38] wait [18:29:22] if ( $wmgContactPageConf ) { [18:29:24] include( $IP . '/extensions/ContactPage/ContactPage.php' ); [18:29:25] etc [18:35:35] > elseif ( $wgDBname == 'arwiki' ) {$wgFlaggedRevsWhitelist = array( '???????_????????' ); [18:35:39] intentionally wrong? [18:35:52] uh [18:35:54] probably not [18:36:00] that's where? [18:37:23] jeremyb: http://deployment.wmflabs.org/global/ [18:37:28] that's a list of config files [18:37:36] that's where i copied from [18:37:44] if you wanted to change anything post it to paste bin and I will change it right now [18:38:06] prod has something like > elseif ( $wgDBname == 'arwiki' ) {$wgFlaggedRevsWhitelist = array( 'الص�?حة_الرئيسية' ); [18:38:06] which file [18:38:26] http://noc.wikimedia.org/conf/flaggedrevs.php.txt http://deployment.wmflabs.org/global/viewfile.php?file=frsetting.php [18:38:37] ok [18:38:39] will change [18:42:51] fixed [18:45:03] still doesn't match prod (wrong encoding?) [19:10:55] petan: when I try to create an account, I get: (Cannot contact the database server: Access denied for user 'mw'@'%' to database 'ar_wikipedia' (deployment-sql)) [19:11:11] um [19:11:16] wait [19:11:19] k [19:12:11] try again [19:12:12] petan: just to verify, you are not Ops in wmf, right? [19:12:18] no [19:12:45] I am just a dev [19:13:21] k just saying "neither one of us is Ops" and wanted to verify [19:13:28] Hmm there's not currently any way to auth people agains't the wikipedia db/grab their access rights (reviewer etc) is there? [19:13:50] no there isn't [19:13:57] oauth would be cool [19:13:58] :( that's that idea out the window then [19:14:03] not access rights but their is a toolserver thing too verify their account [19:15:01] I could use some sort of focing people to put random text in their user namespace then go nom the api to figure out stuff but that sounds painful.... [19:15:24] * Damianz will think about it more as he will need access to the wiki db anyway which labs doesn't have and getting a toolserver account is overcomplicated. [19:17:14] Damianz: http://toolserver.org/~magnus/tusc.php wouldn't do it? [19:19:54] could you create an account? [19:20:18] hexmode: ^ [19:21:04] Oooh that's shiny [19:22:02] Yeah that was the other option but it still means storing user/passwords seperatly+requiring emails etc for resets :( [19:38:15] http://i.imgur.com/Fr6Wp.jpg lol [19:48:21] hexmode: could you create the account there? [19:48:36] I'll try again [19:48:39] ok [19:48:57] it would be embarassing if it didn't work by next week [19:48:58] :D [19:49:15] looks like it worked [19:49:27] lemme try one more time [19:49:47] yes I see it [19:49:50] http://deployment.wmflabs.org/auth/wiki/Special:GlobalUsers [19:50:46] k, worked [19:50:52] again [19:51:05] cool [19:51:49] you should be able to change your permissions now on all project to anything [19:52:26] http://deployment.wmflabs.org/auth/w/index.php?title=Special%3AGlobalGroupMembership&user=MarkAHershberger&wpKnownWiki=en_wikipedia [19:53:07] but I am not really sure it works :D [19:55:36] I gave you steward that's only group which really works ok [19:57:01] hexmode: could someone create a logos for sites? [19:57:03] http://deployment.wmflabs.org/ar_wikipedia/wiki/Europe don't work [19:57:09] I don't know anyone who's good in that [19:57:16] I know [19:57:19] jeremyb: that's in bz [19:57:26] can't you just e.g. use the inverse of the existing logos? [19:57:27] it's a bug [19:57:33] mayve [19:57:37] if someone create it [19:57:38] petan: I can ask someone [19:57:40] cool [19:58:17] btw jeremyb if you create and account I will give you some flags too so that you can import more pages [19:58:36] it's pretty easy now :D [19:58:44] heh [19:58:52] I don't know why I didn't install SUL before [19:59:24] i was led to believe it wasn't easy [19:59:24] do we have a moodbar on commons? [19:59:31] it wasn't [19:59:31] :D [20:00:22] so, what's the deal with all the redlinks? [20:00:24] ),'wmgUseMoodBar' => array('default' => false,'testwiki' => true,'enwiki' => true,'incubatorwiki' => true, // bug 32417'nlwiki' => true, // bug 32202'sewikimedia' => true, // bug 32757 [20:00:34] right [20:00:37] jeremyb: where [20:00:42] [[europe]] [20:00:49] redlinks? [20:00:52] what you mean [20:00:54] [[europe]] [20:00:57] enwiki isn't full db [20:00:59] petan: I'll ask jorm about a logo [20:01:03] only simple wiki [20:01:10] hexmode: nice [20:01:12] so, what? [20:01:22] jeremyb: it's normal it's a redlink [20:01:26] or what you mean [20:01:52] petan: http://simple.wikipedia.org/wiki/Europe has blue links [20:02:07] petan: http://deployment.wmflabs.org/en_wikipedia/wiki/Europe has red links [20:02:08] ah, I think that db import wasn't totaly ok [20:02:19] and what's wrong on that [20:02:24] pages are missing on enwiki [20:02:29] enwiki was just a partial import [20:03:03] http://deployment.wmflabs.org/simple_wikipedia/wiki/ don't seem to exist [20:03:18] no it's simplefull [20:03:32] http://deployment.wmflabs.org [20:03:42] there is a list [20:03:58] hi Ryan [20:04:04] howdy [20:04:16] we are filling up last free space on labs :D [20:04:32] how much of this is puppetized? [20:04:39] nothing [20:05:40] 1.2T 408G 678G 38% /var/lib/nova/instances [20:05:55] Oh can I have a 600gig instance? [20:06:13] :o [20:06:17] I want 600 gb too [20:06:23] heh [20:06:27] I could make a full clone of enwiki [20:07:35] * Ryan_Lane will stab [20:07:35] :) [20:08:42] We just need some inifiniband and a few HPCs [20:08:50] * Ryan_Lane pukes [20:08:59] no infiniband :D [20:09:06] infiniband needs some love [20:09:22] ethernet is going to be faster than infiniband soon anyway [20:09:46] but is it currently faster? [20:09:47] I've got a nice infiniband raid card sat on my desk at work that cost like 700� + 400� for the cables and it doesn't work. So got replaced with a nice storage cluster connected together with cat7 PD [20:09:51] :D* [20:10:04] heh [20:10:04] 40gig fibre > inifiband, slightly cheaper too IIRC [20:12:41] The funny thing is, the once hpc I sometimes use actually just mounts nfs shares over some funky tcp over infiniband setup so the nfs servers are just meh. [20:13:55] petan: http://prototype.wikimedia.org/release-en/Main_Page use this logo? [20:14:02] np [20:14:19] that's only en [20:15:47] done [20:16:38] hexmode: if we could have similar logo for other wikis it would be cool :) [20:16:48] I don't think it's hard to create it but I am really not good in that [20:17:07] eh [20:17:10] heh [20:17:21] will, I'll ask Jorm if he can do it [20:17:33] languages? [20:18:10] ar and ... ? [20:18:12] ar, hi commons and a logo for auth [20:18:20] auth could probably use enwiki [20:18:21] too [20:18:26] it doesn't really matter [20:18:29] http://prototype.wikimedia.org/rc-en/wiki.png [20:18:35] that's what I used [20:18:37] so, ok, will ask [20:18:39] on en_wikipedia [20:31:22] petan: maybe hewiki too? [20:31:32] np [20:31:45] jeremyb: if someone help me with imports of db [20:32:01] easiest is to use Special:Import [20:32:07] well what's TODO atm? [20:32:21] if I create he_wiki than todo is MW: space [20:32:24] than randomly content [20:32:36] ohh, you're not using dumps? [20:32:36] * then [20:32:41] no [20:32:45] we don't have a space for that [20:33:02] I always use Special:Export to extract MW space and then import it [20:33:08] dumps are too big [20:33:19] I did for simplewiki [20:33:24] but that took 40 gb [20:33:26] in sql [20:33:34] no history? [20:33:39] full history [20:33:47] simple wiki is full clone [20:33:58] yeah, so what about others with no history? [20:34:05] that's still too big [20:34:12] enwiki no history has more than 40gb [20:34:21] I don't know about hewiki [20:34:31] yeah, i'm not talking about enwiki :P [20:34:56] lol [20:34:58] enwiki with history but not counting the ES is ~700GB? per ganglia [20:35:00] hewiki is bigger than simple [20:35:16] I can't use dump for hewiki [20:35:19] there isn't space [20:36:06] we could have a script that spiders doing [[special:exports]] [20:36:10] export* [20:36:17] I was thinking of a bot [20:36:23] actually export is no problem [20:36:24] same thing [20:36:30] you can export unlimited number of pages [20:36:38] I always generate list of all MW: [20:36:42] then import then [20:36:45] * them [20:36:47] well same process would do import. and choose the pages [20:37:06] jeremyb: if you create a list of all mw space pages for hewiki I will import it [20:37:20] create a list and copy it to paste bin ok? [20:37:25] sure [20:41:17] petan: also, another option is to use a smaller project like wikisource. plus we have to test sister projects anyway [20:41:30] we can [20:41:41] there is [[Global Requests]] page [20:42:00] please insert a list of all wikis we have to import [20:42:04] so we have a todo [20:42:19] it's on auth [20:42:23] see? [21:01:41] jeremyb: done [21:01:56] done? [21:02:00] he wiki [21:02:06] configuration [21:02:09] now we need db :D [21:02:11] ugh, i just made a list :P [21:02:24] ok [21:02:26] paste it [21:02:27] for import. or you did something different? [21:02:30] oh, ok [21:02:35] I did configuration only [21:02:38] no pages [21:03:06] jeremyb: you definitely want to make an account :D [21:06:22] petan: https://pad.riseup.net/p/hewiki-ns8-titles/export/txt [21:06:58] make sure you set encoding to utf8 in your browser [21:09:16] jeremyb: did you create an account? [21:09:37] creating [21:09:42] right [21:11:10] 180 days?! [21:11:21] that's only on auth I think, or not? [21:11:30] anyway I will fix it [21:11:47] did you create it? [21:11:54] yes [21:11:57] where :D [21:12:02] simplefull [21:12:17] do you want a group? [21:12:18] global [21:12:28] sure, whatever [21:12:31] ok [21:12:40] merge your account on he [21:12:44] just login :D [21:14:17] you should have some now [21:14:26] i do [21:14:28] feel free to import anything you want [21:14:43] heh [21:16:46] jeremyb: all mw pages imported [21:17:16] lol [21:17:19] they are not :D [21:17:29] I forgot to set up names :D [21:17:41] so that space in he language wasn't MW: [21:17:59] well, just run namespacedupes then [21:18:14] but really we should just copy this stuff from wikimedia directly [21:18:57] (i mean prod) [21:19:04] wholesale not as needed [21:19:18] hm... [21:19:28] that's not so easy [21:19:35] no one knows how their config works [21:19:42] we need to convert the rest of configuration there [21:19:48] i know some of it... [21:19:57] i've spent enough hours reading it [21:19:57] some... [21:20:00] yes [21:20:04] I do know some too [21:20:11] but it's not enough to make it working properly [21:20:37] i think i could get it working. but part of the problem is there's parts that are unpublished [21:25:33] jeremyb: it still doesn't work properly, I rename NS [21:25:41] but for some reason it doesn't seem to work [21:25:49] petan: you have to run namespacedupes.php [21:26:00] done [21:27:11] http://deployment.wmflabs.org/he_wikipedia/wiki/MediaWiki:Commons.js [21:27:21] http://deployment.wmflabs.org/he_wikipedia/wiki/%D7%9E%D7%93%D7%99%D7%94_%D7%95%D7%99%D7%A7%D7%99:Sidebar [21:27:29] why it doesn't change? [21:28:14] I still see old sidebar and can't change it [21:31:58] sure looks like it changed, http://deployment.wmflabs.org/he_wikipedia/wiki/%D7%9E%D7%93%D7%99%D7%94_%D7%95%D7%99%D7%A7%D7%99:Common.js [21:34:39] petan: can i log in to dig about the sidebar? [21:34:48] sure [21:35:41] don't break it [21:35:47] it's on deployment-web [21:36:03] we have no backups :D [21:36:07] so be carefull please [21:36:12] 01/07/2012 - 21:36:12 - Creating a home directory for jeremyb at /export/home/deployment-prep/jeremyb [21:36:27] * jeremyb chuckles :) [21:36:35] sounds like you're asking for puppetizing [21:36:50] petan: since I don't want to have 50 different places where people are reporting bugs, I'd like to point them to http://deployment.wmflabs.org/auth/wiki/Problem_reports [21:37:11] 01/07/2012 - 21:37:11 - Updating keys for jeremyb [21:37:15] ok [21:37:24] hexmode: that's a good idea [21:38:14] * hexmode tries to edit it but fails [21:38:23] it's ok now, sorry :) [21:38:28] :) [21:38:34] I uploaded noc confs, but it broke all sites :D [21:38:35] ITS BETA! [21:38:44] it's fixed now [21:44:57] hexmode: can I continue now? [21:45:12] wikis may be not available for 2 min [21:45:28] petan: I didn't know you were doing something that interuppted work [21:45:34] petan: or that I was [21:45:39] ok [21:45:44] I can come back when you say [21:45:57] I mean I want to change config so it probably crash again :D [21:46:00] don't edit now [21:46:11] k [21:46:21] jeremyb: can you close global for a while? [21:46:22] :D [21:46:34] I need to change it [21:46:52] petan: you mean vim? [21:46:54] i'm out [21:46:57] ok [21:50:00] lol [21:50:49] it's completely broken now [21:52:40] petan: i can't remember, CET or EET? [21:52:47] ? [21:52:51] GMT [21:53:01] petan: for you not the wiki :) [21:53:05] aha [21:53:06] I don't know [21:53:21] you mean mine? [21:53:25] I am in GMT+1 [21:53:52] ok, so CET. i.e. not the time of greece or israel [21:56:03] so, idk what you're saying is completely broken [21:56:14] i.e. the pages seem to load for me [21:56:31] took awhile but they seem to work [21:56:50] jeremyb: how long do they take to load for you? [21:58:30] jeremyb: I copied the config from prod. [21:58:32] a while [21:58:40] now I cleaned it up [21:59:01] but it's still very slow [21:59:14] added note to arwiki http://hexm.de/cy [21:59:15] [22:00:06] petan: maybe you need memcache? [22:00:14] it's already there [22:00:26] it's slow because I copied the prod config [22:00:27] hrm... [22:00:33] since then it totally slowed down [22:01:03] so, we can look at profiling and see what is slowing it down [22:01:07] i think generic::webserver::php5-mysql is probably not right [22:01:08] let me see... [22:01:21] jeremyb: yes it's right :) [22:01:31] what's wrong on that [22:02:13] i was thinking applicationserver::homeless or applicationserver::home [22:02:16] have to compare those [22:04:17] Btw did we decide what way we're doing monitoring? Apparently someone was going to merge in my gerrit ticket but while I move it to misc class how it should be I might as well drop out/fix the monitoring stuff to how it's going to work in labs until we have a production cluster. [22:06:05] big problem is that file is so large [22:06:18] it have to parse such a huge file in every page [22:06:52] right, but it shouldn't have to and surely doesn't in prod [22:07:17] well, idk, maybe i take that back :P [22:07:20] I don't know but in my work production servers are incomparably faster than any other [22:07:49] you think it's just a matter of throwing cycles at it? [22:08:38]