[01:08:22] how do i set up my tool account so i don't have to unlock my ssh key every time i push/pull from github? [01:10:19] you can setu p an ssh agent [01:10:22] setup( [01:10:24] setup** [01:10:41] http://mah.everybody.org/docs/ssh looks useful [01:10:42] notconfusing: That's not a good thing. If it's a tool acct, anybody who has access to the account could push/pull. In addition to make it auto enable you would have to make the key paswordless. Thrid: To leave the key enabled for all the time is a drain on the process table. [01:11:44] hasteur, yeah i don't mind unlocking it once when i log in [01:11:52] but certainly wouldn't want it unlocked forvere [01:11:57] thanks legoktm [01:12:13] Xia started getting bothered? [01:12:56] Sorry, wrong channel [01:22:14] * CP678|food is away: This is a manual computer virus. Please copy paste me in your away message. I'm not here right now. [01:32:10] how can i get xsltproc installed on my tools account? [01:52:31] what's the precedure for getting new software like xsltproc installed on labs? [01:53:09] file a bugzilla ticket for it [01:53:14] Wikimedia Labs --> tools [02:01:05] legoktm: thanks [03:04:43] 3Wikimedia Labs / 3Infrastructure: latest dump not available again - 10https://bugzilla.wikimedia.org/66362#c8 (10Gerard Meijssen) 5RESO/INV>3REOP Dumps are created outside of labs. They are moved by Labs staff to Labs. This is all I know and it has been done automagically until the move to the new datac... [03:05:01] 3Wikimedia Labs / 3tools: Install xsltproc - 10https://bugzilla.wikimedia.org/66962 (10Max Klein) 3NEW p:3Unprio s:3normal a:3Marc A. Pelletier Requsting installation of xsltproc on tools. Needed to run the [JATS-to-mediawiki library](https://github.com/wpoa/JATS-to-Mediawiki/), which is a requirment... [03:12:09] If I file a request for access to Tool Labs, how long should it take to process? [03:13:16] less than a day usually [03:14:21] Ah, okay, good. [03:14:30] I basically want to set up a fork of an existing thing. [04:01:29] 3Wikimedia Labs / 3Infrastructure: latest dump not available again - 10https://bugzilla.wikimedia.org/66362#c6 (10Gerard Meijssen) I expect that it gets proper attention. For me the processes of getting external data in their Labs place is part of the infrastructure. Thanks, GerardM [04:22:13] 3Wikimedia Labs / 3Infrastructure: latest dump not available again - 10https://bugzilla.wikimedia.org/66362#c7 (10Tim Landscheidt) 5NEW>3RESO/INV Okay, closing then as I can't understand what's wrong. Please see http://www.chiark.greenend.org.uk/~sgtatham/bugs.html for how to report bugs effectively. [04:44:58] 3Wikimedia Labs / 3tools: Cannot submit sge jobs from tools-webgrid-tomcat - 10https://bugzilla.wikimedia.org/66882#c3 (10Peter Schlömer (dapete)) (In reply to Tim Landscheidt from comment #1) > Re Catgraph, are you aware of [[wikitech:Nova Resource:Catgraph]]? Yes, that is the main reason the new tool has... [05:06:14] 3Tool Labs tools / 3[other]: Migrate to Tool Labs: https://toolserver.org/~vvv/adminstats.php - 10https://bugzilla.wikimedia.org/61030#c1 (10Sjoerd de Bruin) Any update on this? It's a pretty useful tool, there is no alternative for it. [06:00:29] 3Wikimedia Labs / 3Infrastructure: latest dump not available again - 10https://bugzilla.wikimedia.org/66362#c9 (10Gerard Meijssen) Learned from Tim that the files are automagically moved to /public/datasets/public/wikidatawiki The question now is why does the script by Magnus not pick it up. Thanks, Ger... [08:33:28] <`fox`> hello, i have an issue with some sql queries that were pretty fast on toolserver but extremely slow on labs. The query is in this form: http://pastebin.com/raw.php?i=YFYc5JHS and is needed for this tool http://wikiwatchdog.com/ [08:33:45] <`fox`> maybe there are some missing indexes [08:34:59] use the revision_userindex table [08:35:21] `fox`: ^ [08:35:25] <`fox`> for instance this query SELECT rev_id, rev_timestamp, rev_user_text, rev_comment FROM revision WHERE rev_user=0 AND rev_user_text="217.77.80.29" ORDER BY rev_timestamp; takes 0.06 sec on toolserver while more than 30 sec on labs [08:35:29] <`fox`> uh ok! [08:35:56] <`fox`> legoktm, is it like that for every table? should I append _index after the table name? [08:36:00] <`fox`> to get indexes? [08:36:03] no [08:36:03] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Tables_for_revision_or_logging_queries_involving_user_names_and_IDs [08:36:06] just revision and logging [08:36:25] <`fox`> oook! perfect :) thank you! [08:36:51] np [08:36:54] <`fox`> but why is that? why aren't there indexes on the "normal" table? [08:38:14] http://lists.wikimedia.org/pipermail/labs-l/2013-September/001607.html explains why [08:55:16] legoktm: `fox` also see https://bugzilla.wikimedia.org/show_bug.cgi?id=66786 [12:41:29] anyone got any ideas as to why I can't access my instance via PuTTY? I've successfully used https://wikitech.wikimedia.org/wiki/Help:Access_to_instances_with_PuTTY_and_WinSCP many months ago and it worked flawlessly back then; now I seem to be able to ssh to bastion, but ssh'ing to my instance always fails with a PuTTY Fatal Error window saying "Disconnected: No supported authentication... [12:41:31] ...methods available (server sent: publickey)" [12:42:19] I have Pageant running in the background and I've also changed pmtpa to eqiad in the URLs...so, I honestly have no clue what's going on [12:44:00] ashley: is agent forwarding turned on? [12:44:21] ashley: https://wikitech.wikimedia.org/w/images/b/b8/20130118-2312-WinSCP_Login.png [12:44:25] under authentication parameters [12:44:49] oh, but the putty examples uses plink [12:44:54] under Connection -> SSH -> Auth "Allow agent forwarding" is checked, yeah [12:45:04] (I couldn't get WinSCP working either, for what it's worth) [12:45:19] I've logged into bastion previously and accepted the certificate, so it's not that either... [12:45:40] does it work if you ssh to bastion, then run 'ssh '? [12:48:55] ashley@bastion1:~$ ssh social-tools1.eqiad.wmflabs [12:48:57] If you are having access problems, please see:https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [12:48:58] Permission denied (publickey). [12:58:09] ashley: echo "$SSH_AUTH_SOCK" [12:58:12] does that return anything? [12:59:08] on bastion? yeah [12:59:11] ok [12:59:14] ssh -v -v social-tools1.eqiad.wmflabs [12:59:37] that should give something like 'debug2: key: ' just before the "If you are having access problems," [13:00:04] then below that 'debug1: Offering RSA public key: ' [13:00:25] if it does that, and still gives debug1: No more authentication methods to try., it suggests you don't have access to that host for some reason [13:01:00] if it doesn't list any keys, that suggests the agent forwarding is broken for some reason [13:04:18] valhallasw: http://pastebin.com/C7kpctMe [13:05:02] ashley: right. That suggest your key is served but not accepted [13:05:57] huh, strange...I'll have to ask legoktm to take a look at it later on, he should have the appropriate access to that instance, given he originally built it for me and all [13:52:14] 3Tool Labs tools / 3Erwin's tools: Migrate https://toolserver.org/~erwin85/randomarticle.php to Tool Labs - 10https://bugzilla.wikimedia.org/60871#c2 (10Nemo) p:5Unprio>3High (In reply to Silke Meyer (WMDE) from comment #1) > Am I right in thinking the tool is working in Tool Labs? No, it's not working,... [14:18:44] When making a new instance, is ubuntu-14.04-trusty ready for use? [14:54:00] Generic web servers for both 12.04 and 14.04 are broken. 12.04 can't find the URL / and 14.04 auto redirects to the /w/ directory. Am I missing something? [14:58:28] slevinski: trusty should be ready. [14:58:40] Can you tell me what you mean by 'generic web servers'? [14:58:54] slevinski: I'm not sure about the current status, but IIRC Puppet cleans up /etc/apache2 quite well, so you might need your own puppetmaster. [14:59:39] thanks. Weird things were happening, but the generic trusty web server is working now. [15:15:44] 3Wikimedia Labs / 3tools: Make exec hosts as submit hosts too - 10https://bugzilla.wikimedia.org/54786#c1 (10Philippe Elie) p:5Unprio>3High Can we have progress on that? Subprocess works poorly for multiple cpu bound sub-task. [15:58:25] Is http://tools.wmflabs.org/ down for everyone or just me? [15:58:57] Hmm, loaded now [16:02:43] 3Wikimedia Labs / 3tools: Update tcl-trf to version 2.1.4-dfsg-3 - 10https://bugzilla.wikimedia.org/62387#c9 (10Andrew Bogott) 5ASSI>3PATC Not having Coren's context for this... I grabbed the trusty package for tcl-trf and added it to reprepro as a precise package. "2.1.4-dfsg2-1" I'm confused about t... [16:30:31] 3Wikimedia Labs / 3tools: struct::stack/TclOO version mismatch - 10https://bugzilla.wikimedia.org/66984 (10Giftpflanze) 3NEW p:3Unprio s:3normal a:3Marc A. Pelletier tools.giftbot@tools-login:~$ tclsh % package require struct::stack version conflict for package "TclOO": have 1.0, need 0.6.1 Suggeste... [16:39:14] 3Wikimedia Labs / 3tools: Update tcl-trf to version 2.1.4-dfsg-3 - 10https://bugzilla.wikimedia.org/62387#c10 (10Giftpflanze) It works. I'm interested in what Coren thinks about it when he comes back. [16:47:44] 3Wikimedia Labs / 3tools: Update tcl-trf to version 2.1.4-dfsg-3 - 10https://bugzilla.wikimedia.org/62387#c11 (10Silke Meyer (WMDE)) 5PATC>3RESO/FIX Nice! Thanks to Andrew and thanks for checking, Giftpflanze. Closing this one. [16:54:14] 3Wikimedia Labs / 3tools: struct::stack/TclOO version mismatch - 10https://bugzilla.wikimedia.org/66984 (10Giftpflanze) [16:54:29] 3Wikimedia Labs / 3tools: Missing Toolserver features in Tools (tracking) - 10https://bugzilla.wikimedia.org/58791 (10Giftpflanze) [18:50:06] !log integration rebased puppetmaster . Among others: that migrates us to puppet 3 [18:50:09] Logged the message, Master [20:05:26] !ping [20:05:26] !pong [20:27:33] bd808, any idea why I cannot login to newly created instances and why their puppet status is "unknown"? Assuming it retrieves the status by running sth. under novaadmin, this obviously also fails [20:28:00] rillke: you need to wait about 5-10min [20:28:12] Hi :) [20:28:15] rillke: also, I saw you recently attempted to apt-get install some package on tools. Usually, filing a bug should get that installed in a day or so [20:28:41] A question, is it possible to remove the "tools." prefix in the identity name of an IRC bot? [20:29:03] YuviPanda, thx. [20:29:20] rillke: yw :) [20:30:36] YuviPanda so it is normal that I get "Permission denied (publickey)." while for the old instance it works wo issues? [20:30:58] rillke: that sometimes happens if the NFS volume with the public keys isn't up yet. [20:31:42] rillke: It looks to me like the initial puppet run may not have worked on mediahandler-tests-test.eqiad.wmflabs; I see puppet related errors on the console. [20:32:40] oh [20:38:31] 3Wikimedia Labs / 3tools: Please install openbabel - 10https://bugzilla.wikimedia.org/66995 (10Rainer Rillke @commons.wikimedia) 3NEW p:3Unprio s:3normal a:3Marc A. Pelletier http://packages.ubuntu.com/precise/science/openbabel I'd like to be able to convert molecules. Thanks in advance. [20:39:06] Just got my Tool Labs account set up. A question, though: is it considered its own small virtual server or is it one account on a shared server (virtual or otherwise), like the Toolserver of old? [20:41:41] rillke: patch submitted, and I hand-installed on tools-dev if you want to experiment. should be on the exec nodes when it gets merged [20:42:36] harej: heya [20:42:43] harej: Tool labs is a shared environment, but a Labs project is separate virtual machines. [20:43:02] harej: YuviPanda can tell you much much more [20:43:08] :D [20:43:23] harej: yeah, it's a shared environment, which should hopefully provide you everything you need. [20:43:30] harej: runs on virtual servers, but we add more when load gets higher [20:43:36] YuviPanda: thx again [20:43:51] rillke: yw. [20:45:21] YuviPanda: where is the dir for the web server? [20:45:52] harej: no web servers for user accounts, only 'tool accounts' [20:46:29] harej: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help is very comprehensive [20:46:35] harej: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Web_services has info on web services [20:47:16] Excellent, thank you. [20:47:31] Unix servers have changed since the days of 2006. [20:47:34] harej: :) yeah [20:47:51] harej: this architecture also allows us to add (in the future) better support for non-php servers (Python, Node, Go, etc) [20:48:53] harej: if you want any new packages installed, opening a bug should get that done in a day or so. Most things people want are already there, though [20:49:22] in a day? may i laugh? [20:49:38] gifti: unless it needs to be specially packaged [20:49:44] which most common packages do not need to be [20:49:54] trivial package additions are usually a day or so [20:50:01] hm, right [20:50:18] gifti: TCL, unfortunately, doesn't have too much representation in debianland :( [21:11:14] YuviPanda: What's the difference between bastion and tools-login? [21:11:26] I thought you needed to log in through bastion to get to your tool, but it appears I can log in through tools-login? [21:11:34] harej: bastion is generally for all labs instances. tools-login is specifically for toollabs only [21:11:44] Okay. [21:12:07] harej: tools-login has some extra nicities, like access to all your tools' /data/project folders, and all the packages that are installed in the exec environment are in tools-login too [21:12:25] harej: there's also tools-dev if you want to do some computationally intensive work (like compiling) that might slow down things for other users [21:12:53] harej: also, tools-login supports mosh (http://mosh.mit.edu/) which is an SSH replacement that is *AWESOME*. Keeps connections open no matter what [21:13:00] !! [21:13:00] hello :) [21:13:11] harej: ? [21:13:27] an always-on ssh sounds cool [21:13:41] harej: by 'no matter what' I mean through hibernate, standby, network drops, etc. also is more reliable/faster than ssh on high latency connections. [21:13:45] harej: yeah. bastion doesn't support that [21:17:06] And am I correct to assume that individual tools have UIDs that are distinct from the UID assigned to me as a human? [21:17:14] hence tools.[toolname]? [21:18:12] harej: yeah [21:18:25] Makes sense. Thank you! [21:19:54] harej: :) [21:30:01] 3Wikimedia Labs / 3tools: Please install openbabel - 10https://bugzilla.wikimedia.org/66995 (10Tim Landscheidt) a:5Marc A. Pelletier>3Yuvi Panda [21:48:14] 3Wikimedia Labs / 3tools: Please install openbabel - 10https://bugzilla.wikimedia.org/66995 (10Sam Reed (reedy)) s:5normal>3enhanc [21:48:14] 3Wikimedia Labs / 3tools: Install xsltproc - 10https://bugzilla.wikimedia.org/66962 (10Sam Reed (reedy)) s:5normal>3enhanc [22:09:15] 3Wikimedia Labs / 3tools: Install xsltproc - 10https://bugzilla.wikimedia.org/66962 (10Tim Landscheidt) 5NEW>3ASSI a:5Marc A. Pelletier>3Tim Landscheidt [22:28:18] 3Wikimedia Labs / 3deployment-prep (beta): Special:UserLogin is busted - 10https://bugzilla.wikimedia.org/67004 (10Chris McMahon) 3NEW p:3Unprio s:3critic a:3None http://en.wikipedia.beta.wmflabs.org/wiki/Special:UserLogin is not seen as a web page, browsers try to download it as a file. [22:31:14] 3Wikimedia Labs / 3deployment-prep (beta): Beta Labs pages are being served as octet-stream - 10https://bugzilla.wikimedia.org/67004#c1 (10Matthew Flaschen) p:5Unprio>3High It's serving it with: Content-Type: application/octet-stream Same for http://en.wikipedia.beta.wmflabs.org/wiki/Main_Page [22:37:29] 3Wikimedia Labs / 3tools: Please install openbabel - 10https://bugzilla.wikimedia.org/66995#c2 (10Daniel Zahn) This package includes the following utilities: * babel: Convert between various chemical file formats * obenergy: Calculate the energy for a molecule * obminimize: Optimize the geometry, mini... [23:14:49] bd808, I still cannot login to mediahandler-tests-small [23:15:01] and it's not listed at https://wikitech.wikimedia.org/wiki/Nova_Resource:Mediahandler-tests [23:16:02] and you are not listed as admin on https://wikitech.wikimedia.org/wiki/Nova_Resource:Mediahandler-tests [23:16:31] rillke: Puppet is blowing up during the provisioning and things are not getting setup correctly with the labs master hosts. [23:17:00] Hmm, is this something I should take care of? [23:17:06] I saw andrewbogott looking into the puppet problem in another channel. [23:17:24] yeah, I'm working on it, will take a few minutes. [23:18:16] andrewbogott: I saw this earlier today and "meant to" track down the problem. Sorry I didn't get it fixed before you had to get involved. [23:18:45] bd808: no problem, so did I :) [23:46:59] 3Wikimedia Labs / 3tools: Please install openbabel - 10https://bugzilla.wikimedia.org/66995 (10Tim Landscheidt) 5PATC>3RESO/FIX