[00:29:49] Warning: There is 1 user waiting for shell: Hersfold (waiting 0 minutes) [00:32:06] :D [00:34:26] I thought he retired [00:40:46] andrewbogott_afk: https://gerrit.wikimedia.org/r/#/c/63807/ <— mind a quick review? :) [00:43:24] Warning: There is 1 user waiting for shell: Hersfold (waiting 13 minutes) [00:49:57] andrewbogott_afk: note that I'm removing the toc stuff, but may add it back in with JS later [01:46:01] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/TODO was modified, changed by MPelletier (WMF) link https://www.mediawiki.org/w/index.php?diff=690300 edit summary: [+55] One more done [01:47:12] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/TODO was modified, changed by MPelletier (WMF) link https://www.mediawiki.org/w/index.php?diff=690303 edit summary: [-92] Also one. [02:56:10] Warning: There is 1 user waiting for shell: Mdgilbert (waiting 0 minutes) [04:01:15] Change on 12mediawiki a page Wikimedia Labs was modified, changed by Superm401 link https://www.mediawiki.org/w/index.php?diff=690324 edit summary: [+230] /* Background */ not just for practicing for production (staff) [08:17:51] @notify Ryan_lane [08:17:51] I will notify you, when I see Ryan_lane around here [09:51:57] Can somebody tell me who is the contact for project "etherpad" [09:58:07] petan: yes [10:00:37] Jan_Luca I don't remember but I am member of that project [10:00:48] @labs-project-members etherpad [10:00:52] @labs-on [10:00:53] Labs utilities are already enabled [10:00:54] @labs-project-members etherpad [10:01:05] @labs-project-users etherpad [10:01:05] Following users are in this project (showing all 11 members): Ryan Lane, Wikinaut, Johnduhart, Abartov, Dzahn, MarkTraceur, Novaadmin, SuchetaG, Yuvipanda, Demon, Petrb, [10:01:26] petan: I want to try using Moodle and Etherpad together [10:02:04] so I have installed the etherpad-plugin for our Moodle installation http://sandbox.wikiversity.wmflabs.org/moodle [10:02:53] but I need the api-url and api key [10:03:03] to use the labs-etherpad installation [10:04:27] petan:Do you know where I could get this infos? [10:05:53] petan: Maybe you could add me to the project so I can look for the infos myself? [10:20:51] sure [10:22:00] Jan_Luca I don't have permissions :( you need to ask someone else [10:24:40] petan: OK, I will ask Ryan_Lane or ^demon [10:43:17] Change on 12mediawiki a page Wikimedia Labs/Migration of Toolserver tools was modified, changed by Silke WMDE link https://www.mediawiki.org/w/index.php?diff=690421 edit summary: [+114] /* Road map for upcoming features */ added link to roadmap [10:45:09] Change on 12mediawiki a page Wikimedia Labs/Migration of Toolserver tools was modified, changed by Silke WMDE link https://www.mediawiki.org/w/index.php?diff=690422 edit summary: [+109] /* Where is documentation? */ added link to Magnus' page [10:48:19] Change on 12mediawiki a page Wikimedia Labs/Migration of Toolserver tools was modified, changed by Silke WMDE link https://www.mediawiki.org/w/index.php?diff=690424 edit summary: [-1647] /* List of available features */ updated Ryan's list with Coren's one [10:48:56] Change on 12mediawiki a page Wikimedia Labs/Migration of Toolserver tools was modified, changed by Silke WMDE link https://www.mediawiki.org/w/index.php?diff=690425 edit summary: [-3] /* List of available features */ corrected wrong formatting [10:49:59] Change on 12mediawiki a page Wikimedia Labs/Migration of Toolserver tools was modified, changed by Silke WMDE link https://www.mediawiki.org/w/index.php?diff=690426 edit summary: [-384] /* Schedule for the transition */ deleted old stuff [11:35:03] peeps? [11:35:07] CHEMOBOT 000000 returned error: Closing Link: 208.80.153.192 (Too many user connections (global)) [11:35:07] CHEMOBOT 000000 IRC Got disconnected call from irc.freenode.net, attempt to reconnect @ 1368617657 [11:35:33] Is that a problem of my IRC accounts, or of labs? [11:57:51] Beetstra: there are too many connections from labs to freenode [11:58:03] @notify Coren [11:58:03] I will notify you, when I see Coren around here [11:58:23] as Far as I know this is something that is being sorted out with freenode [11:59:34] sorting it out now in #freenode :P [12:04:51] Beetstra you will need to wait :( freenode is busy with hackers now [12:05:06] petan, please ask again in a few days after we have dealt with the attacks and have our support email back and time to deal with other things than the attack [12:05:20] you can use some alternative network, like mine :> [12:05:24] irc.tm-irc.org [12:05:25] :P [12:05:47] 200 users per ip is default limit and it can be raised [12:47:49] anyone around for help? it is not possible to create new instances within our wikidata-dev project [12:48:05] error: "Failed to create instance." [13:29:40] Change on 12mediawiki a page Wikimedia Labs/Migration of Toolserver tools was modified, changed by Silke WMDE link https://www.mediawiki.org/w/index.php?diff=690453 edit summary: [+368] /* Do I have to migrate my software to Labs? */ [13:32:44] Abraham_WMDE: You probably reached the default instance quota. Increasing it will take an intervention from Ryan_Lane or andrewbogott_afk [13:34:44] Coren: we definitely need the limit increased [13:34:45] okay, wait. We now have three projects that sort-of half-resemble the toolserver? [13:34:58] Bots, Webtools and Tools? [13:36:11] Abraham_WMDE: Here is the ticket I created for better feedback when instances failed to create. Could be worth to follow up on this. https://bugzilla.wikimedia.org/show_bug.cgi?id=45483 [13:37:41] valhallasw: three? [13:38:01] Silke_WMDE_: thanks [13:38:37] Silke_WMDE_: https://wikitech.wikimedia.org/wiki/Nova_Resource:Bots / https://wikitech.wikimedia.org/wiki/Nova_Resource:Webtools / https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools [13:38:41] valhallasw: There is "Tools" for stable tools and "Bots" for experimental tools (the naming being as "historical accident" as Coren puts it.) [13:39:38] as I have understood it, Bots is the experimental environment (it's the environment that's experimental, not the tools on it), while Tools is the stable environment [13:39:52] valhallasw: You've got it right. [13:40:24] Coren is the Webtools project still active and should it be used? [13:40:46] valhallasw: Bots can have unstable/untested versions of packages and libraries, for instance, and is more flexible about changing the environment and giving root; tools is more static and restrictive, but because of that will be the stable home. [13:41:35] Silke_WMDE_: webtools is... I don't really know. It looks like it has been mostly abandonned and will definitely be deprecated once petan installs web support on bots. [13:42:18] That's confusing. petan can you tell us more? [13:42:20] Silke_WMDE_: It's not part of "Tool Labs" in any meaningful way, I suppose. It's "one lab project" where people have been running web tools before Tool Labs, and won't get the same support. [13:42:24] I think we should make it possible to rename projects in future [13:42:28] brb (telephone) [13:42:35] Silke_WMDE_ sorry I didn't follow, what you need to know? [13:42:38] in office now... [13:42:51] I don't know anything about webtools project [13:43:31] valhallasw: So yeah; your understanding is right and webtools should be considered deprecated/obsolete. [13:44:10] OK, so we should just communicate Tools as the place to switch to from the TS [13:45:43] re [13:46:32] Is that Webtools project what was merged to "bots" some time ago? [13:48:43] petan Coren: I will add a comment the the Webtools project. [13:53:32] Silke_WMDE_: From my TODO: "The final implementation is to deprecate 'webtools' and use 'bots' as the "experimental/flexible" development project, and "tools" as the official Toolserver-replacement home for stable tools." [13:54:04] hehe ah! [13:54:48] So I think I have to correct something in my roadmap: I thought that in Bots also the tools are experimental not only the environment. [13:56:49] Silke_WMDE_: Ah! No, you can run experimental bots in both, but bots is the right place when you want to experiment with changes in the environment. Once things are complete, normally, any major shift in the environment would be tested on bots first. [13:57:12] ok [14:00:04] Jan_Luca: Hey, did you get what you needed for EPL? [14:00:37] EtherPad Lite* [14:00:43] marktraceur: No [14:00:55] OK [14:01:02] If you could help me it would be nice :-) [14:01:17] Coren, do you know if Phillipe is on IRC? [14:01:19] Jan_Luca: The idea is you need an API access key, so that Moodle can contact our EPL server? [14:01:52] marktraceur: Yes, I need the api-url and a api key to access your server [14:02:32] Is the url "http://etherpad.wmflabs.org/pad/" the right one or do I have to use a other port? [14:03:32] Cyberpower678: Doesn't look like. It's very early still west coast, so he might not even be up yet. [14:03:51] Coren, Well, what's his nick? [14:04:12] marktraceur: I'm away for around a half hour now. Can we talk then? [14:04:38] Cyberpower678: Philippe [14:05:00] Ah. [14:05:03] Thanks. [14:06:12] Jan_Luca_away: Sure [14:06:45] Jan_Luca_away: I'm sort of blinking out of sleep right now, and commuting in about an hour, but I'll be around sometime [14:13:04] petan, addshore - thanks! I thought that was the problem (and that is why one of my other bots is sometimes losing contact then as well). I know it can be raised, I'll wait for that (it is now weekend anyway!!), see you guys on Saturday [14:20:26] marktraceur: I'm back [14:33:50] Jan_Luca: So am I! [14:34:11] Jan_Luca: OK, so...the API should probably be accessible via the normal URL, but I haven't tried it [14:34:41] marktraceur: I will try this but I need the key first [14:35:32] Jan_Luca: Also, the pads on the server are sometimes expected to be there...it's labs, but people have gotten comfortable with using this instance as a reasonably-stable solution and I don't want to inordinately affect that if we don't have to :) [14:35:51] Jan_Luca: Mostly "be smart", I guess. [14:36:55] marktraceur: I think the Moodle plugin should be stable: https://moodle.org/plugins/view.php?plugin=mod_etherpad [14:37:08] *nod* sounds OK to me [14:37:21] Jan_Luca: Also, will you need anything on the EPL side to make this work? [14:38:12] marktraceur: I think no. The plugin seems only to include the standard editor into moodle [14:38:16] *nod* K [14:38:30] Jan_Luca: In case you haven't seen, the key is in your PM now [14:39:49] marktraceur: A moment, I try it [14:44:37] petan or Coren: Could you add me to the Tools project before the Hackathon? I've got a Labs account already. [14:44:43] sure [14:44:50] cool, thanks [14:44:59] petan: Will you be at the Hackathon, too? [14:45:03] yes [14:45:08] Silke_WMDE_ can you tell me your name? [14:45:08] great [14:45:10] marktraceur: Could it be that there is a server on port 9001 for API requests? [14:45:17] petan: Silke Meyer [14:45:21] ok [14:45:36] or smeyer might have been the login [14:45:39] done [14:45:46] thanks, that was fast! :) [14:45:50] :) [14:47:22] Jan_Luca: There is, but it's reverse proxied to port 80 under the pad/ subdir [14:49:12] marktraceur: Mhh... because I get the error "Empty or No Response from the server" [14:49:27] I have to look what the plugin does internally [14:52:00] Coren I recommend you to remove .suppresstips now since the tips were replaced with sysadmin motd now :P [14:52:39] Sysadmin motd: [14:52:39] TO-DO: fix motd on this box [14:52:45] :-) [15:10:04] petan, I discussed with Coren the possibility of having a 'I would like a Tools Labs account'-page, with a shell-access like wm-bot warning. Would that be hard to implement? [15:10:36] what do you mean [15:10:46] ah [15:10:48] we can then have a simple three-step plan on the Tools web page which does not include 'go to IRC to poke people, or mail people' [15:10:54] no it wouldn't be hard [15:10:56] 21:40 <+wm-bot> Warning: There is 1 user waiting for shell: Basti2342 (waiting 54 minutes) [15:11:03] ^ that kind of warning [15:11:12] I'd really want it analogous to shell requests with the SMW cat and all [15:13:57] Coren: I <3 forms. [15:14:35] other than that some SMW classes that have to be made... ok [15:16:09] valhallasw bot is parsing the pages from a category and reading their content, once you set up similar system I will be able to implement this [15:16:21] petan: ok [15:27:10] ^^ good idea [15:31:09] https://wikitech.wikimedia.org/w/index.php?title=Special%3AAsk&q=%5B%5BCategory%3ATools+Access+Requests%5D%5D%5B%5BIs+Completed%3A%3ANo%5D%5D&po=%3FTools+Justification%0D%0A%3FModification+date&eq=yes&p%5Bformat%5D=broadtable&sort%5B0%5D=Modification+date&order%5B0%5D=ASC&sort_num=&order_num=ASC&p%5Blimit%5D=&p%5Boffset%5D=&p%5Blink%5D=all&p%5Bsort%5D=Modification+date&p%5Border%5D%5Basc%5D=1&p%5Bheaders%5D=show&p%5Bmainlabel%5D=& [15:31:21] the edit form doesn't work yet, though [15:33:59] and now it works! whoo! [15:34:28] petan: https://wikitech.wikimedia.org/wiki/Category:Tools_Access_Requests [15:35:46] petan: you can also use the SMW search thing, that might be more efficient [15:36:01] not really [15:43:15] petan: ok, whatever is easier for you :-) [15:56:54] Warning: There is 1 user waiting for shell: Alan (waiting 0 minutes) [16:03:48] petan: not sure if you got my message but yes I am still on bsql01 [16:06:37] Coren: is it possible to use ftp (filezilla) on tools? [16:07:26] Alchimista: God forbid! :-) There are a number of Windows clients for file transfers that use SFTP which will do the right thing for you, though. [16:08:29] Alchimista: I'm no Windows expert, though, but I'm pretty sure a google of "Windows SFTP" would find something easily enough; or others Window users on the channel can point you at what they use. [16:10:08] Coren: Is ~/.forward supposed to be working already? [16:10:23] Warning: There is 1 user waiting for shell: Alan (waiting 13 minutes) [16:10:53] scfc_de: It does for me. :-) [16:11:03] Coren: I'll try again. [16:14:09] Coren: God forbids windows :P I sayd filezilla because it's what i use more. If sftp is possible, how do i conect to my tools? sftp tools-login.wmflabs.org is as far as i can go in terminal :S [16:15:02] You don't connect to your tool's account, but to your own; it has permission to write in your tool's directories by default (/data/project/theproject) [16:19:54] wow, awsome [16:38:41] Coren: do you know if labs has na ipexept from freenode? [16:40:29] Betacommand: It's in the pipeline, but the freenode staff has told us "go away, we're in full panic mode" and it may need a couple more days to put in place. Running into a bot connection issue? [16:55:23] Coren: is it safe to put passwords regarding the bot/tool on the tool space, right? Only project members or sys admins have access to it by default? [16:55:44] Alchimista: If you check your permissions that they do not include world read, yes. [17:05:51] Ryan_Lane: I'm here (in respond to your email ;-)) [17:05:56] \o/ [17:06:01] so, I increased your quota [17:06:09] it should work now [17:06:23] if not, I'll change some more settings [17:06:40] alas, in this version of openstack it's not very easy to tell which quota is being reached [17:06:49] works! [17:06:52] great [17:07:04] \o/ thx [17:07:07] yw [17:08:28] robla uses znc? hipster [17:13:35] andrewbogott: yep, we can change submit to something like "apply filter" [17:13:46] that'd be way better! [17:13:47] just need to add a message and apply it to the form [17:13:56] let me do that really quick [17:17:04] Coren: yeah, I took down one of my minor bots to free up a slot for another [17:17:44] mhhm Coren PHP Fatal error: Out of memory (allocated 822870016) (tried to allocate 79 bytes) in /data/project/addbot/classes/database.php on line 104 ;p [17:18:21] addshore: That seems straighforward enough. Allocate moar memories! :-) [17:18:27] yus :P [17:19:17] the question is how much.. [17:19:26] This one process may be rather hungry :/ [17:21:08] addshore: Well, a good idea is to try to increase it by a large margin, possibly reducing it later when you have a better idea. [17:21:18] Like, add an extra gig [17:22:00] I'm upping it to 4G ;p Everything else is happy with under 500m, This one run on de might take a bit more :) [17:22:41] Coren: I'll be upgrading openstack in 2.5 hours [17:23:00] Should have no effect on the actual cluser, should it? [17:23:12] I.e.: VMs will keep on runnin'? [17:23:12] Coren: there's a possibility it'll cause issues. the worst I really expect is needing to reboot some instances [17:23:14] but I doubt that [17:23:15] looks like 1.5GB would have been enough [17:23:20] everything should just keep working [17:23:29] * Coren knocks on wood. [17:23:40] I tested in labs and in eqiad [17:27:02] Steinsplitter: Please ask in this channel, you question seems relevant here. [17:27:41] i hav asked a other user now :P [17:27:55] petan: whats bots-dev1 for? :) [17:28:14] Im guessing the same as -dev? ;p [17:29:46] Hey could anyone ask puppet to install 'python3' in the bots project? tools has it already [17:34:28] sitic: The best person to ask that is petan. Normally, I'd offer to help but I'm in rather a crunch at this time until Amsterdam. :-) [17:35:32] Coren: ok, thanks :-) [17:40:38] andrewbogott: https://gerrit.wikimedia.org/r/#/c/63884 [17:45:19] andrewbogott: so, i'm trying to be good and use the release model used in production on virt0 [17:45:32] addshore: If the code (g.php?) is still the same as some weeks ago, IIRC you had the strange habit of loading the complete database table into memory and then iterating over it. I'm sure PHP has some better way to loop over queries. [17:45:37] we use the wmf release branch, and our extensions are in it [17:45:52] scfc_de: yes, but then I kill the DB ;p [17:46:06] so, we merge changes into our extensions, then we go to the release branch, update the submodule and commit the change into the release branch [17:46:11] then push that in for review and self-merge [17:46:16] too many reads and writes and updates for the current infrastructure [17:46:29] I should be able to change this when we get the new tools dbs [17:46:39] I believe it's documented on the "how to deploy code" page [17:47:07] then on virt0, in the slot: git pull; git submodule update --init [17:47:20] this will ensure we keep up with updates in the branch as well, which is a good idea [17:47:29] Ryan [17:47:46] Ryan_Lane: Sure, that makes sense. I'll try to remember to do that too :) [17:47:49] addshore: No, the reads stay the same?! [17:48:06] andrewbogott: for emergency fixes, it's fine to cherry-pick a change directly into the extension [17:48:32] scfc_de: ? [17:49:12] * andrewbogott nods [17:53:13] New patchset: Andrew Bogott; "Make the fake racktables login/password resemble production." [labs/private] (master) - https://gerrit.wikimedia.org/r/63888 [17:55:24] addshore: At the moment, you're reading the whole table into memory. That causes a huge memory consumption. If you're properly iterating over the query results instead, the amount of data read stays the same, but the memory consumption reduces to that of one row instead of all of them. [17:55:43] "$list = Database::mysql2array($result);" [17:56:14] scfc_de: I meant it kills the database server :) [17:56:38] Change merged: Andrew Bogott; [labs/private] (master) - https://gerrit.wikimedia.org/r/63888 [17:57:02] addshore: For the database server, there is absolutely no difference whatsoever. [17:59:19] heh, thats what I thought [17:59:28] but for some reason it just couldnt cope [18:00:04] not sure if it is because of the virtualised environment :/ or just the pure quantity [18:00:28] Im rewriting it before or during the hackathon (or thats my plan) [18:01:55] scfc_de: if you feel like it make a branch on git? ;p [18:02:24] addshore: I don't know either. At the moment, I think using 4 GBytes (or whatever) is totally acceptable. If it runs 24/7 for eternity, it would of course be nice to reduce its memory footprint. [18:02:58] hehe, its actually only using 1.2GB :) Also give it a day and this will hopfully be reduced to under 700 [18:03:24] de is just the bane of my life ;p [18:03:58] :-) [18:07:27] scfc_de: you a php person? :P [18:07:46] * anomie does not understand how AnomieBOT can show a peak memory usage of 132M but reliably fail when allocated 256m. [18:08:27] I dont understand why only come tasks show a peak.. [18:08:42] * anomie also does not understand how AnomieBOT is shown as using only 1.69s, which is the same as it showed just a few minutes after being started 16 hours ago [18:09:37] (a) Peaks are only shown when they are > current, and (b) peak is only sampled at interval (I think 1s) so very brief bursts can be missed (possibly during startup) [18:10:04] addshore: Not more than I need to read MediaWiki code :-). Others can probably be more helpful. [18:10:21] Ryan_Lane: https://wikitech.wikimedia.org/wiki/Special:NovaProject // it's not really clear how you should add a project in that list [18:10:37] addshore: As for CPU time, that's what the kernel reports. Most programs actually spend most of their time waiting for I/O or something, and that doesn't grow CPU time. [18:10:39] Ryan_Lane: maybe there could be a [+] in the white space? [18:10:48] valhallasw: hm. likely a good idea... [18:11:16] Ryan_Lane: <3 the new project filter! [18:11:17] Ryan_Lane: and an auto-refresh would increase the awesomeness [18:11:25] Coren: I'm sure AnomieBOT has used more than .01s of real CPU time in 16 hours, though [18:11:26] valhallasw: that's in the plans :) [18:11:29] cool :-) [18:11:44] Ryan_Lane looks awesome [18:11:47] anomie: Lemme go see. What node is it running on atm? [18:11:54] so, gmail has an interface similar to this for adding recipients in the new composer view [18:11:56] tools-exec-04 [18:12:05] addshore: He /does/ have an awesome beard does he not? [18:12:21] I guess folks are more used to typing addresses into a to line, though [18:12:30] so they don't need a + or anything like that [18:13:07] anomie: Aha! Anomiebot forks(); POSIX semantic says CPU time of children is only added to the parent's when reaped. [18:13:17] Ryan_Lane: are there plans to improve the add-a-member-to-a-service-group interface? if not, how easy is it to get a basic local development for Nova? [18:13:42] Ryan_Lane: the jquery.chosen interface would be a great fit there, too [18:13:47] valhallasw: yes. the project management interface has always been my least favorite [18:13:50] Coren: So is it ignoring the memory usage until reaped too? [18:14:14] I decided to tackle something simpler for the first place I'd add ajax elements :) [18:14:30] anomie: It might be underreporting it, yes. Darn, that complicates finding the right value. [18:14:34] (and then, since the children no longer exist to use memory, continues to ignore it) [18:14:47] valhallasw: we have a shared development instance right now [18:14:51] which isn't ideal [18:15:03] but I think andrewbogott is working on making it easy to add addtional ones [18:15:08] anomie: The actual limit is a hard rlimit, though, so it's still checked. [18:15:24] addshore: glad you like it :) [18:15:36] I added chosen to core for this :) [18:15:47] it can be used anywhere an HTMLForm multiselect is used [18:16:17] Ryan_Lane: Suggestion (if possible): have the 'set filter action' performed on losing focus. [18:17:03] Coren: well, I'd like it to occur as the filter changes [18:17:17] but I need an api action for the project list for that [18:17:22] err [18:17:33] for the 'list' action on every interface [18:17:36] that's a tough change [18:18:05] but that's my end-goal [18:18:36] at that point there won't even be a 'Set filter' button [18:19:16] I also want to make the project name a hyperlink, so if you click it, it brings you to an anchor on the page [18:19:32] to restore andrewbogott's TOC in the project filter behavior [18:33:57] Am I the only Chrome user for whom the empty text box is too narrow? [18:34:27] scfc_de: the chosen one for the project filter? [18:35:25] ugh. it is for me too [18:35:42] Ryan_Lane: Yep. "bots" fits in it (and the dropdown list), but the rest is cut off. [18:35:52] I wonder if there's a core change missing for this [18:36:32] there should be a minimum width set [18:37:00] ah. yep. that's it [18:37:23] I guess that change isn't going in till next wmf branch release. let me cherry-pick it in [18:38:07] Hello. Is Coren about? [18:39:51] ah. crap. heh. that change isn't merged in yet [18:40:22] well, I'll put in the temporary fix till I fix this in chosen upstream [18:40:31] Ryan_Lane: np [18:40:48] scfc_de: force refresh the page [18:40:51] it's fixed now [18:41:08] JohnMarkOckerblo: Coren seems to take a minute to come back. What do you need? [18:41:10] Ryan_Lane: Perfect. [18:43:40] wanted to ask about the status of a request for IP address access that he'd forwarded for me. [18:44:06] JohnMarkOckerblo: Okay, then you'll have to wait :-). [18:44:21] No problem. [18:45:16] But maybe you could answer a general question. I've uploaded the new version of my code to tools.wmflabs.org and put it in the CGI directory and it seems to be working in my tests. [18:45:44] JohnMarkOckerblo: you needed a public IP for a project? [18:45:56] or was this specific to tool labs/ [18:46:34] ...well, I'm actually looking for the user's IP address. But if this is going to take a while, I might decide to redirect my templates to point to the Wikimedia Labs instance of my code... [18:46:37] ah [18:46:58] I think I'll let Coren handle that :) [18:47:07] ...instead of my local Penn instance (which I'm currently doing). [18:47:12] I'll get more familiar with the tools project at the AMS hackathon :) [18:47:51] Is there anytihng stopping me from chaningg my Wikipedia templates to point there now? Or do I need to get some other sort of approval or test signoff? [18:48:14] to point to labs? [18:48:27] no approval needed from our side [18:48:38] Yes, to point to labs. I want to make sure I'm not committing a faux pas by just turning it on. [18:48:39] note that I'm upgrading the infrastructure in about an hour [18:48:44] so you may want to wait a bit :) [18:48:48] just in case [18:49:32] okay, I can do it later today (will head home and check here later. [18:49:51] if you enable it and we can't handle the load, we'll add resources [18:50:06] JohnMarkOckerblo: I am. [18:50:08] Load shouldn't be very large at this point, based on my own logs. [18:50:17] ok. cool [18:51:02] Thx! Was just talking with Ryan_Lane abt flipping the switch on the tools instance of my lirbaries forwarder, without waiting for IP access to be approved. [18:51:18] JohnMarkOckerblo: Your request is at Legal, but I know they are very busy right now so it may take a few extra weeks still. If you can live without the "your library" feature, you can point here whenever you feel you ar ready [18:51:38] (But if that's close to resolution, I can hold off. I'll lose a little functionality w/o IP access, but only a few campuses are using that particular feature, and they're on break now :-) [18:52:16] I would expect that they'll have a response after they are done with the Labs privacy policy and TOS which is high on their TODO list. [18:52:30] The "your library" feature will still work; users will just have to set a preference explicitly instead of relying on the service to figure it out on its own from their IP address. [18:53:10] JohnMarkOckerblo: Right. Like I told you, your request seems reasonable enough that I expect they'll find a way to make it work that satisfies them. [18:53:58] Okay. Well, if I flip it over now it sounds like I can put the "box" template back to looking like a box, and a few people have been asking about that. [18:54:20] So I'll try it later today, after Ryan_Lane does the upgrade. [18:54:50] JohnMarkOckerblo: If you want it to be extra nice, it'd reassure the enwp community if you also had a second maintainer. [18:55:14] As in a seconf maintainer of the code? Or of the local installation? [18:55:26] JohnMarkOckerblo: Incidentally, thanks for the hard work. That is a valuable resource. [18:55:51] Happy to train someone up on either one. The main module's on github and I've been starting to field pull requests (only on the data so far, not yet the code) [18:56:04] JohnMarkOckerblo: Of the local installation, mainly. It's "hit-by-a-truck" insurance to make certain that if you can't or won't maintain the service, someone else can assure continuity. [18:56:53] Okay. Do you have someone in mind? I can also look for someone in the community, and write up docs. [18:57:39] Thank you for all your help again, btw! [19:00:24] Anyway, I have to go now to do school pickups. Let me know if there's someone you recommend as a second maintainer. [19:16:36] How long does it take for a sudo rule to update? [19:16:53] I made myself a sudoer on deployment-parsoid1 and deployment-parsoidcache1 a while ago but it's still not letting me sudo [19:18:52] RoanKattouw: it's immediate [19:19:11] let me look at the policy [19:19:37] RoanKattouw: I don't see one [19:19:52] oh [19:19:53] sorry [19:19:55] wrong project [19:20:06] deployment-prep has a sudo policy called deployment-parsoid [19:20:25] hm. that should owrk... [19:20:28] *work [19:21:51] ugh [19:21:58] seems we still haven't fixed this issue [19:22:51] new instances have their hostnames set to the instance name, not the ID [19:23:01] but we stick the instanceid based hostname into ldap for sudo [19:23:11] RoanKattouw: for now just add yourself to the admin policy [19:23:56] * Ryan_Lane checks to see if we have an open bug for this [19:24:12] OK thanks [19:24:15] yw [19:24:16] So per-host sudoers are broken [19:24:25] we need to rename all old instances [19:24:56] https://bugzilla.wikimedia.org/show_bug.cgi?id=40945 [19:24:57] bleh [19:28:46] Change on 12mediawiki a page Wikimedia Labs/Stability improvement project was modified, changed by Ryan lane link https://www.mediawiki.org/w/index.php?diff=690580 edit summary: [+136] [19:46:20] Does instance-proxy work for non-port-80 services? [19:46:35] http://deployment-parsoidcache1.instance-proxy.wmflabs.org:8000/ doesn't seem to work offhand [19:46:57] I don't think so sadly [19:47:33] I forwarded port 80 to 8000 temporarily and it seems to be trying to do something now [19:47:58] Hmph using the right instance name also helps [19:52:24] <^demon|lunch> RoanKattouw: Yes! [19:52:44] Reboot ALL the instances! [19:52:46] ;-) [19:53:10] <^demon|lunch> RoanKattouw: http://8080.deployment-parsoidcache1.instance-proxy.wmflabs.org/ [20:01:30] I'm starting the openstack upgrade now [20:01:49] Break a leg. After you're done. :-) [20:01:52] heh [20:04:44] Should we place bets on how long before labs goes down totally? :P [20:06:26] ^demon|lunch: Ooooooh [20:06:28] valhallasw: Is there a link to make a new request for access to tools I can stuff on the landing page? [20:06:44] Sadly I have to rebuild the instance as I forgot to open up that port in the security group :( [20:07:03] And I can't rebuild the instance because OpenStackManager is dow [20:07:05] n [20:10:41] RoanKattouw: I should be done soonish [20:10:46] OK cool [20:10:59] Wasn't complaining at you specifically, just at the universe for inconvenient timing :) [20:16:51] andrewbogott: heh [20:17:04] andrewbogott: err: Failed to apply catalog: Could not find dependency Class[Generic::Mysql::Packages::Client] for Exec[create_puppet_db] at /var/lib/git/operations/puppet/manifests/openstack.pp:374 [20:17:05] :( [20:17:18] dammit [20:18:51] Why can't it find it? Shouldn't it be loaded from mysql.pp? [20:20:29] Oh… probably I only tested on systems that also had a mysql-server installed [20:20:31] is that class being called somewhere? [20:21:08] I guess I should have made sure puppet was running properly before I upgraded [20:21:30] This is in openstack::database-server? [20:21:32] yeah [20:21:36] let's revert that change for now [20:21:40] ok [20:22:17] There's probably a quick fix, but... [20:22:21] well, no need to mess with it. [20:22:32] Want me to do the revert? [20:22:32] yes, please [20:22:37] we'll push it back in after the upgrade is done [20:22:38] then fix it [20:22:44] after we know everything is working [20:25:03] Coren: yes! [20:25:27] except Nova Resource:Tools has disappeared O_o [20:25:42] there was a link there :-) [20:26:01] valhallasw: I disabled openstackmanager [20:26:09] in the middle of an upgrade [20:26:17] It'll be back soon, valhallasw. :-) [20:26:37] I'll add it to http://tools.wmflabs.org/?list when it comes back. [20:26:51] Coren: https://wikitech.wikimedia.org/wiki/Special:FormEdit/Tools_Access_Request [20:27:15] but I'm not sure what would happen if someone uses it now [20:27:31] because it places the requests under Nova Resource:Tools/... [20:28:14] bad things, likely [20:28:18] :) [20:28:25] it would go into the wrong namespace, I think [20:28:47] probably in ns0 as 'Nova Resource:Tools/...' [20:28:56] yep [20:29:22] and it would also be registered in the category tree and in SMW, so there would be an uncloseable request [20:30:06] although it should be possible to edit it based on article id/revision id instead [20:30:31] I could move it with a maintenance script [20:33:08] Ryan_Lane: hi :) sorry to interrupt, I have posted a message on labs-l to get some new certificates under Labs CA. Haven't found out how to generate new ones. [20:33:16] ah [20:33:28] Ryan_Lane: would be nice if you give a quick "how to" reply maybe dump something on wikitech [20:33:33] that is for beta, we have https now and could use some more certs :D [20:33:44] hm. I'm not really sure the best way to handle this [20:33:53] the certs will obviously be untrusted :) [20:34:04] yup [20:34:17] but at least they will match the domains we are using on beta [20:34:47] I don't think we should order real certs, just let people manually validate them. that will be enough in my opinion [20:34:55] but if you get a better idea go ahead :-] [20:35:50] Ryan_Lane: anyway heading to bed. that let you sometime to figure out a nice reply :D [20:35:51] maybe a single WMLabs SSL cert that people can install, that is used to sign other certificates? [20:35:52] have a good afternoon! [20:36:08] hashar: thanks. sleep well [20:36:15] valhallasw: share your view on labs-l :-] [20:36:17] *wave* [20:37:56] ok. upgrade seems to have went ok [20:38:02] let's see about reenabling openstackmangaer [20:38:56] well, this isn't looking promising [20:39:27] api seems to be timing out on me [20:40:53] Ryan_Lane: https://wikitech.wikimedia.org/w/api.php?action=novaprojects&getall=true works for me [20:41:03] that doesn't hit openstack [20:41:35] hm [20:41:59] You see the CERN openstack figures? or rather what they will be running under it this next year [20:42:30] I've talked with tim bell about it [20:42:33] hm [20:42:40] nova list seems to work for me [20:42:51] I wonder if it's a nova api version issue [20:42:54] hm [20:42:59] no, because I tested this in eqiad [20:43:09] maybe cached tokens? [20:45:28] I hope to bump into him some day, seems a cool guy - watched videos of his talks before [20:46:14] Ryan_Lane: https://wikitech.wikimedia.org/wiki/Special:NovaProject works for m. [20:46:23] Coren: it's not openstack ;) [20:46:29] that hits ldap [20:46:31] glance is the issue [20:46:34] Ah, [20:46:35] glance-api is timing out [20:47:26] I know why [20:47:42] purging and reinstalling it [20:48:34] I normally use nuke-from-orbit... it's a good alias for purge =D [20:49:19] damn ubuntu [20:49:43] glance-registry tries to write to its database on install [20:49:52] how fucking stupid is that? [20:50:12] * Ryan_Lane sighs [20:50:22] this is some bullshit [20:50:33] o.0 [20:50:38] because now the god damn package won't fully install [20:50:44] which means puppet won't add the config files [20:50:56] lol [20:51:26] Ryan_Lane: Could you check the firewall settings of the "etherpad"-project because I have trouble when using curl to access port 80, but when I use the instance name there is no problem [20:51:29] deleted its sqllite [20:51:47] I had issues getting glance working in labs because of it creating the schema on start before now... dislike the way it's got the management commands hooked in [20:51:55] Jan_Luca: was the working before and isn't now? [20:52:02] Warning: There is 1 user waiting for shell: Ricordisamoa (waiting 0 minutes) [20:52:24] Jan_Luca: openstack services are still down somewhat due to upgrade [20:52:36] Damianz: yes, that's the package doing it. it's stupid [20:52:48] Ryan_Lane: I does not know. Accessing it with a web browser shows no problem, only when I try to use curl [20:53:10] you trying to access the external ip or something? [20:53:13] nat borks if you do that [20:53:18] also https://twitter.com/EmergencyPuppy/status/334759375602327552/photo/1 cute mouse [20:54:02] Damianz: I have no problem using a normal web browser [20:54:16] ah. you can't access the public IP from inside of labs, unfortunately [20:54:32] it's an issue with NAT [20:54:51] maybe I can get mark to look at that again one day [20:54:55] I'd like split horizon dns as a workaround for that... but that's not doable without a custom backend :( life sucks [20:55:21] hooray, looks like everything is working again [20:56:28] Yeay! [20:57:14] looks like testing was an accurate depiction of production :) [20:57:37] But but, that's boooring! [20:57:41] :D [20:57:48] * Ryan_Lane likes boring [20:58:26] * Damianz likes sexy and slick [20:59:36] Ryan_Lane, Damianz: Here the output of "curl -v "http://etherpad.wmflabs.org/pad/api/1/createPad?padID=LXbvHWFJ&text=df&apikey=": * About to connect() to etherpad.wmflabs.org port 80 (#0)* Trying 208.80.153.253... Connection timed out [20:59:37] * couldn't connect to host [20:59:37] * Closing connection #0 [20:59:37] curl: (7) couldn't connect to host [20:59:49] Jan_Luca: that's from inside of labs? [20:59:58] Jan_Luca: Yes, the public IPs are broken from inside of labs, that's a known issue [21:00:12] Ryan_Lane: Yes from wikiversity-sandbox-world [21:00:17] If you use the internal ip, or put that in your /etc/hosts file it will work [21:00:19] it sucks, but you need to use the instance's fixed ip [21:00:50] Damianz: I will try the hosts-file solution [21:00:50] curl -H Host: ftw though <3 that flag [21:01:16] Damianz: split horizon dns is an option, but a kind of crappy one [21:01:25] I'd rather access to the public IP just work ;) [21:01:47] maybe alex can take a look at this as well [21:01:55] For our use case, it's a crappy solution.... we use split horizon dns at work and it's kinda awesome, since internal dev stuff just gets fired to pre-release etc [21:03:11] Damianz: Thank you, this works [21:05:31] Warning: There is 1 user waiting for shell: Ricordisamoa (waiting 13 minutes) [21:08:27] hello! [21:10:23] I'd have to say this, openstack upgrades are getting way easier [21:10:40] is the upgrade done now? (I want to make sure it's safe now to switch my template forwarding over to wikimedia tools server) [21:10:45] JohnMarkOckerblo: yep [21:11:07] great. Will change the templates. [21:15:00] None of the tool instances seem to have even noticed the upgrade. [21:15:20] I think none of the instances should have notices [21:15:23] *noticed [21:15:42] I'm pretty happy with this upgrade cycle [21:15:48] I'm writing an email to the devs as usual [21:15:58] and of course there's going to be some complaints :) [21:16:42] They can have cookies the day you can just apt-get update and everything works [21:17:22] that's basically how it worked this time [21:17:28] apt-get install [21:17:33] keystone-manage db_sync [21:17:37] glance-manage db_sync [21:17:41] nova-manage db sync [21:18:11] db sync is fair enough not auto... since you only really want to do it once if running multiple endpoints so cool [21:19:01] yep [21:19:04] Warning: There is 1 user waiting for shell: Ricordisamoa (waiting 27 minutes) [21:19:20] and the way they have things planned it's not necessary to do automatically anyway [21:19:28] they are going to have backwards compatible schema [21:19:43] only way to support not having to take the entire thing down I guess [21:19:51] yay to eventually consistant [21:24:23] yep [21:32:28] Warning: There is 1 user waiting for shell: Ricordisamoa (waiting 40 minutes) [21:33:48] okay; templates now pointing to toolserver (and bug fixed that I inadvertently introduced when adding a server-handoff feature) [21:35:36] I've also put a library index generator onto the toolserver (so if I'm hit by a bus, someone else can download new library and catalof info from github and run the reindexer to make it available in Wikipedia) [21:36:41] JohnMarkOckerblo: Good show. Can you link me to the enwp discussion again? [21:38:11] * Damianz gets the bus out [21:38:44] talk started at http://en.wikipedia.org/wiki/Template_talk:Library_resources_box then there was a TfD on April 30 [21:39:09] There's going to be a CfD called after this is all moved over. Still have to turn the box template back into a box; will do that in a moment. [21:39:23] Damianz: Hey, watch where you're waving that! [21:40:11] Busses make for lame DR testing [21:50:30] okay, box is now a box again. We'll see how long it takes till the debate reopens :-) [21:57:13] JohnMarkOckerblo: If I were you, I'd preemptively start a thread on the VPP; explaining what you did, and what measures you took (i.e., move to labs, make provisions for continuity, looking for added maintainer) and why your box is useful. Get some mindshare behind it. [21:59:06] sounds like a good idea. what's VPP stand for? [21:59:24] JohnMarkOckerblo: Your code doesn't handle the 'no default library set' cleanly. Check your php error logs. [21:59:50] (Also, the templates got put in the WIkiProject LIbraries gorup. Would that be a good place to start discussion or at least have pointer?) [22:00:33] JohnMarkOckerblo: Actually, I wanted to say VPR (which is "Village Pump (proposals)" [22:01:29] JohnMarkOckerblo: [Wed May 15 22:00:58 2013] [error] [client 10.4.1.89] Can't locate object method "new" via package "NetAddr::IP::Lite" (perhaps you forgot to load "NetAddr::IP::Lite"?) at /data/project/ftl/cgi-bin/ftl line 241., referer: http://en.wikipedia.org/wiki/Cognitive_style [22:03:08] ack; will hotfix that in a moment. [22:10:06] ok, should be fixed. NetAddr::IP:::Lite on my server but not on the WMF server. [22:10:33] JohnMarkOckerblo: So it has. [22:11:09] (speaking of which, what's the best way to install CPAN modules like that on the WMF server? I'll eventually need it if IP access is approved.) [22:11:44] (IOn my own machine, I just use the package manager or the usual 4-step CPAN process ending in "make install", but both require superuser access) [22:11:56] JohnMarkOckerblo: Ask one of the sysadmins, or open a bug request (the latter tends to be much faster, the latter more reliable) :-) [22:12:05] In that particular case, I've preemptively installed the module. :-) [22:13:31] thanks; now I don't have to worry about commenting out those lines when I download a new version :-) [22:20:49] thanks for your help. enjoy your dinner!