[00:06:02] New patchset: coren; "Bump version for packaging previous fixes" [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67930 [00:06:49] New review: coren; "Version bump" [labs/toollabs] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/67930 [00:06:49] Change merged: coren; [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67930 [01:22:18] Coren or petan: Can I get either libdigest-crc-perl or libstring-crc32-perl installed? [02:05:27] petan: Also, it looks like sharp-memcached is not binary safe. The following command fails when it should succeed: echo -en 'set test 0 0 4\r\nf\xc3\xb3o\r\nget test\r\nquit' | nc tools-mc 11212 [02:05:40] petan: And this succeeds when it should fail: echo -en 'set test 0 0 3\r\nf\xc3\xb3o\r\nget test\r\nquit' | nc tools-mc 11212 [02:11:11] paravoid: http://www.mediawiki.org/w/index.php?title=Git%2FNew_repositories%2FRequests%2FEntries&diff=708815&oldid=708814 [02:11:13] :-P [03:05:41] why am i suddenly getting jsub errors? [03:05:51] Before I could just use jsub python script.py [03:06:12] Now I'm getting http://pastebin.com/eW3pqYBa [03:08:11] petan: ^^ ? [05:35:59] theo|cloud tools or bots [07:06:23] petan: Kk. :) [07:06:34] here [07:16:23] a930913 NotImplementedError: no ability to handle more than one release [07:16:33] pypi can't convert lxml :/ [07:32:17] @notify anomie [07:32:18] I'll let you know when I see anomie around here [07:36:01] !log tools petrb: installed libdigest-crc-perl [07:36:03] Logged the message, Master [07:49:07] petan: :/ [07:49:34] petan: I could do it locally, right? As in virtual environment. [07:49:44] a930913 make a package from it and I will install it [07:50:17] petan: The most I know about packaging is apt-get ;) [07:53:09] the most I know about python is that it's a snake [07:54:02] petan: It's a group of comedians... [07:54:09] * a930913 sighs. :p [07:55:17] http://3.bp.blogspot.com/-Z-587LkJj_0/TqY-rXZCrkI/AAAAAAAAAN8/UxD_JmFRXOg/s1600/python_snake.gif comedian? [07:55:22] !python is http://3.bp.blogspot.com/-Z-587LkJj_0/TqY-rXZCrkI/AAAAAAAAAN8/UxD_JmFRXOg/s1600/python_snake.gif [07:55:23] This key already exist - remove it, if you want to change it [07:55:25] petan: hey there! Have you solved your vimrc configuration on labs ? :D [07:55:30] !python [07:55:30] Damianz [07:55:32] aha [07:55:35] hashar not really [07:55:53] petan: let me know if you need any help / tip [07:55:55] hashar: but I found out it's not in puppet, so you can safely sudo rm the wmf version :3 [07:56:16] which makes it possible for you to use own .vimrc [07:56:16] what is your issue with vim in labs ? [07:56:24] I don't like the colors [07:56:27] it break terminal [07:56:36] and indentation rules suck [07:56:46] I have to switch to paste mode in order to paste :/ [07:56:51] with my own config I do not [07:57:08] kkk [07:57:30] so when you paste, vim will apply the indentation/autoformatting rules to the text being pasted [07:57:36] yes [07:57:39] which totally break it [07:57:41] if you want to disable that, you can: set paste [07:57:49] I know, that is how to switch to paste mode [07:57:53] but I am lazy :D [07:57:58] one problem solved :D [07:57:58] my own config doesn't require that [07:58:08] idk why [07:58:11] for the colors misbehaving in your terminal, I am not sure what you mean [07:58:18] when I turn off vim [07:58:20] a screenshot would be nice [07:58:22] terminal is all green [07:58:39] when I open other application which change colors it get fixed [07:58:42] like htop [07:58:43] even when simply opening and closing it ? [07:58:46] or mcedit [07:58:48] yes [07:58:54] weird [07:58:55] when I open and close vim - terminal is broken [07:59:28] :q -> htop -> q == annoying [07:59:32] maybe something in your environment. You can try: bash --norc --noprofile [07:59:43] but it happens everywhere on labs :/ [07:59:45] that will start a new shell without loading any bashrc / bash profile .profile .. [07:59:47] and only vim does it [08:00:19] so that is most probably your terminal :( [08:01:43] !pythonguy is this guy master python more than you: http://lh5.ggpht.com/-gjDgXLXmWTQ/TsuuOwSKWHI/AAAAAAAAk4w/XJOKxaGti-c/boy%252520python%252520bff%252520snake%2525206_thumb.jpg [08:01:43] Key was added [08:02:44] hashar it happens in lxterm as well as gnome-terminal :/ [08:02:50] I think in putty as well [08:02:50] petan: it must be some escape sequence that your terminal does not recognize or mis interpret [08:04:09] !pythonwalkthrough is http://stream1.gifsoup.com/view2/1903280/ministry-of-silly-walks-o.gif [08:04:10] Key was added [08:10:36] !tooldocs [08:10:36] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help [08:28:23] petan: What's the channel to @join wm-bot again? [08:28:33] #wm-bot [08:28:38] legoktm: Like my walkthrough? :) [08:28:39] it's @add [08:28:42] not @join [08:28:52] a930913: busy sorry [08:28:57] LOL [08:29:26] petan: Ah, that's why :D [08:30:48] petan: It was @relay on wasn't it? [08:31:03] @relay-on [08:31:03] Relay is already enabled [08:31:05] with - [08:31:07] :o [08:31:09] Ah. [08:35:00] petan: I was wondering if it might be worth adding some sort of token authentication to prevent this? [08:35:13] it's already logging it... [08:35:22] but yes that is possible [08:35:23] Logging what? [08:35:26] @verbosity++ [08:35:26] Verbosity: 1 [08:35:28] @verbosity++ [08:35:28] Verbosity: 2 [08:35:34] idk which level it's on [08:35:45] @verbosity+-0 [08:35:46] but on one of the levels it display the source of message in system log [08:35:58] Test? [08:36:32] LOG [6/11/2013 8:35:58 AM]: DEBUG: Relaying message from 10.4.0.220:41103 to #wikimedia-labs:Test? [08:36:48] petan: Source as in user? Surely just IP, in which case... yeah, how does that identify me? [08:36:55] As in, it's from the labs. [08:37:00] That's all you know. [08:37:03] hmm [08:37:22] yes I will implement the tokesn then [08:37:40] @verbosity-- [08:37:40] Verbosity: 1 [08:37:41] @verbosity-- [08:37:41] Verbosity: 0 [08:37:53] the debug log is really huge on verbosity about 8 [08:37:54] :P [08:38:04] petan: :p [08:38:07] like 2000 lines per minute [08:38:10] petan: I know when debuglog gets too big [08:38:11] :o [08:38:15] ツ [08:38:36] AzaToth: You made Twinkle, right? [08:39:07] a930913: aye [08:39:45] petan: a bot I ran on ts before had log eating up quota [08:39:54] heh [08:40:06] AzaToth: Any chance you could help me with WAVE? [08:40:09] gry: so, you said you needed user rights across wikis? [08:40:12] yes I think beetstra or someone was eating whole labs storage in past [08:40:17] yes [08:40:39] gry: user rights to delete stuff and blow up the wikimedia project? :P [08:41:01] gry: I dont know if there is a tool active yet/atm/in planning for displaying such data [08:41:03] let me think... mhm, ok: {{granted}} [08:41:20] petan: thanks, do you know what api query I can use on a username to retrieve their user rights across wikimedia wikis? want to do that to a bunch of users (~40) once only, and I can try to script that [08:41:33] petan: not in the 'grant me al the things' sense, I'm just trying to get a list of such rights for another user's username [08:41:44] that's some nice humour there though, I like it :) [08:41:46] I think there is [08:41:54] I don't think there is api query for that, which is a shame :/ on other hand you can get this from db [08:41:55] but I cant remember which [08:42:03] I think there is a service [08:42:11] yes service [08:42:13] dunno if it's on ts, labs, or wherever [08:42:15] but no query using mediawiki [08:42:36] gry: is it "api" you need, or just a lookup once [08:42:52] just lookup once, it's not a repeat task [08:43:06] what's the username then? :P [08:43:12] * Beetstra munches on the last couple of bytes .. burp [08:43:22] http://toolserver.org/~quentinv57/sulinfo [08:43:27] gry ^ [08:43:28] I have a list of fourty members, http://toolserver.org/~quentinv57/sulinfo/ gives it for one username but not for all [08:43:38] ah ok [08:44:44] Io assume a simple SQL could solve it quicly petan ? [08:44:49] ±spelling [08:44:57] * a930913 nybbles on Beetstra. [08:45:12] a930913: word bro' [08:45:47] AzaToth: ? :/ [08:46:21] OMG [08:46:32] AzaToth you are that guy who moved jsub from /usr/local/bin ? [08:46:48] next time pls create a symlink just to keep backward compatibility [08:47:04] petan: I didn't move them, Coren did the actual work [08:47:09] mhm [08:47:16] and Coren said Coren was going to keep symlinks around [08:47:23] * a930913 sharpens his pitchfork. [08:47:31] he didn't [08:47:32] :P [08:47:40] well, that's his ass thenn [08:48:00] Coren: Coren Coren Coren [08:48:00] which is why all my bots and tools just broke since midnight [08:48:04] hehehe [08:48:06] I fixed it, no problem [08:48:36] AzaToth: Can you help me with WAVE? [08:48:46] a930913: sorry, forgot you asked [08:48:52] whats WAVE? [08:48:54] AzaToth: It's like a huggle/STiki in the browser. [08:49:09] never heard about it [08:49:15] huggle in browser != possible [08:49:40] we were working on it in past, wmf shut it down [08:49:52] haven't used huggle [08:49:55] At the moment, it just hacks into twinkle a bit in order to revert and popup warn. [08:50:04] petan: Why did WMF shut it down? :/ [08:50:09] a930913: linky? [08:50:10] authentication [08:50:33] petan: In terms of it must need rollback? [08:50:41] Or in terms of asking for users passwords? [08:50:48] they said if we were asking user for a SUL password in a browser they would block the access of any such a service to cluster [08:50:58] I'm not asking for passwords. [08:51:05] and, on other hand they provide no other way beside asking for a password [08:51:07] It uses the currently logged in guy. [08:51:09] petan: well oauth is coming too….. [08:51:09] petan: why would you ever ask for password? [08:51:12] which is a deadlock a bit [08:51:19] legoktm this was like years ago... [08:51:25] petan: :D [08:51:27] and oauth is still coming since then [08:51:35] hehe [08:51:46] AzaToth: https://en.wikipedia.org/wiki/User:A930913/wave [08:51:57] AzaToth I would ever ask for a password because oauth is coming for years... [08:52:04] * AzaToth still waits for bug 1 to be fixed [08:52:05] and until it come I will have to [08:53:06] a930913: sorry, need to run, bbl [08:53:49] AzaToth: Kk, ping me. [09:02:34] petan: If they are in their browser, why do you need to authenticate them? ;) [09:13:07] a930913 whart [09:13:17] ah [09:13:22] I will explain to you later [09:13:27] :O [09:15:47] !ping [09:15:47] pong [09:15:52] !log bots petrb: test [09:15:54] Logged the message, Master [09:17:27] Is spawning a number of processes for each enwiki edit a bad idea? [09:22:21] a930913: http://meta.wikimedia.org/wiki/Wm-bot#.40token-on [09:22:37] @token-on [09:22:37] New token was generated for this channel, and it was sent to you in a private message [09:22:49] I don't like that one [09:22:50] @token-on [09:22:50] New token was generated for this channel, and it was sent to you in a private message [09:23:07] petan: tokens are? [09:23:12] oh [09:23:14] authentication [09:23:26] yes [09:23:51] petan: Can you recollect token? (Assume has to be chanop?) [09:24:15] yes [09:24:18] @token-remind [09:25:08] petan: \o/ [09:25:25] !log bots petrb: test [09:25:27] Logged the message, Master [09:29:41] !log bots petrb: this is a test ignore it [09:29:43] Logged the message, Master [09:31:05] a930913: so what was the issue? [09:31:40] AzaToth: I don't know the mw js too well. [09:32:18] a930913: they are (should be at least) documented at http://www.mediawiki.org/wiki/ResourceLoader/Default_modules [09:33:17] sadly some methods there have been created with a single target in mind, thus can be a bit limiting [09:33:50] AzaToth: I want to automate certain events, such as "revert current page with edit summary 'foo', load the user's page and template it with {{subst:bar}} under the l2 heading." [09:34:37] you can do such things either async or sync [09:35:13] i.e. on a mw.Api.post after another mw.Api.post or in the done callback [09:35:43] !log bots petrb: blabla [09:35:45] Logged the message, Master [09:36:03] in twinkle, we'll ignore good practice and just pump the requests to the server [09:36:09] AzaToth: Yeah I know, but your twinkle stuff does most of it already :) [09:36:16] especially on the batch modules [09:36:22] generally we do it async [09:36:39] only when we need the reply, we do it sync [09:37:30] Though in my example, you could do it all sync, because then you wouldn't template without reverting, and it would all be done in the background anyway :) [09:37:35] in twinkle we dont use mw.Api for edits as we made a own lib before mw folks decided to implement mw.Api [09:38:02] yea [09:38:17] reverting is something that can be aborted [09:38:51] for example here I'm making 3 get requests async: https://github.com/azatoth/twinkle/blob/master/modules/twinklearv.js#L335 [09:39:14] because they doesn't depend on eachother [09:41:30] How can you test, without committing a wiki edit? Is that possible? [09:42:40] a930913: you can test watching and unwatching a page or something. [09:42:46] or a purge [09:44:20] legoktm: ? [09:44:49] [02:41:30 AM] How can you test, without committing a wiki edit? Is that possible? <-- what are you trying to test? login session? [09:45:55] legoktm: Something like a revert and warn, for example. [09:46:20] what are you trying to test? your JS that would revert and warn? [09:47:03] legoktm: I mean, let's say you miss a semicolon. Of course you could catch that other ways, but testing it on the wiki, how could you catch it without committing it? [09:47:32] Erm, I don't follow. [09:47:43] Where is this semicolon missing? [09:47:47] In your .js code? [09:47:51] Run a JS linter.... [09:49:00] legoktm: Yeah, but let's say I put "msw" instead of "mw". [09:49:12] load it from your JS console [09:49:13] idk [09:49:47] legoktm: That's why I was asking AzaToth :p [09:50:14] :P [09:57:05] Warning: There is 1 user waiting for shell: Telmar (waiting 0 minutes) [10:02:57] a930913: either test on a test page or on a test wiki [10:08:16] Mmm, thought as much. [10:09:47] Should I make my wave wiki code simply "eval(msg)" and send all the code from the wave UI? [10:10:35] Warning: There is 1 user waiting for shell: Telmar (waiting 13 minutes) [10:12:38] never use eval [10:20:07] AzaToth: Is usually the kneejerk answer ;) [10:21:56] * a930913 -> afk. [10:24:02] Warning: There is 1 user waiting for shell: Telmar (waiting 27 minutes) [10:37:32] Warning: There is 1 user waiting for shell: Telmar (waiting 40 minutes) [10:41:31] !rq Telmar [10:41:31] https://wikitech.wikimedia.org/wiki/Shell_Request/Telmar?action=edit https://wikitech.wikimedia.org/wiki/User_talk:Telmar?action=edit§ion=new&preload=Template:ShellGranted https://wikitech.wikimedia.org/wiki/Special:UserRights/Telmar [11:10:07] legoktm: Are the permissions on your ~/.forward correct? (I. e., writeable only by user.) [11:10:19] * legoktm checks [11:11:09] -rw-rw-r-- 1 legoktm wikidev 28 Jun 4 06:53 .forward [11:11:11] so no [11:11:14] is that an issue? [11:12:19] legoktm: Yes: "legoktm@tools.wmflabs.org (generated from local-dbreps@tools.wmflabs.org) retry timeout exceeded" :-). "chmod 644 ~/.forward" should do. [11:12:54] {{done}}. sorry about that [11:13:16] No problem, you just missed the jsub spam from dbreps :-). I'll change that to "jsub -quiet" later. [11:57:44] !toolsdoc [11:57:48] !tooldocs [11:57:48] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help [12:45:09] petan: ping [12:45:53] hey [12:46:09] anomie: I saw the report, will fix it soon [12:46:26] petan: ok, that's what I was pinging about [12:46:39] Also, thanks for installing the crc package [12:46:50] you know you can always use normal memcache on 11211 which is stable :> [12:47:18] That's what I am doing for now [13:07:18] Moo! [13:09:41] Don't scare off the unicorn! [13:19:46] scfc_de I think that Coren, pretending he is a cow, is just trying to attract the unicorn in order to perform something pervert later... [13:20:03] :P [13:20:15] poor unicorn [13:20:17] !unicorn [13:20:18] http://www.ascii-art.de/ascii/uvw/unicorn.txt [13:21:27] !unicorn del [13:21:27] Successfully removed unicorn [13:23:01] !uncorn is what you think that labs are like: http://static.giantbomb.com/uploads/original/1/17172/1419618-unicorn2.jpg what labs are actually like: http://img3.etsystatic.com/000/0/5177778/il_fullxfull.306517207.jpg [13:23:01] Key was added [13:23:43] !unicorn is what you think that labs are like: http://static.giantbomb.com/uploads/original/1/17172/1419618-unicorn2.jpg what labs are actually like: http://img3.etsystatic.com/000/0/5177778/il_fullxfull.306517207.jpg [13:23:44] Key was added [13:23:47] !uncorn del [13:23:48] Successfully removed uncorn [13:48:51] petan: More like http://i.ebayimg.com/t/Steampunk-Fantasy-Mechanical-Robot-Unicorn-Damask-Dressage-Art-Print-8-5x11-/00/$(KGrHqZ,!g4E2eQ7SdtNBN1pzD3epg~~_1.JPG [14:24:47] heh [14:42:39] Coren: rehi - ssh issues look at http://pastebin.com/DRjwXUxP [14:43:30] Oren_Bochman: That looks like normal behavior for connecting to an instance part of a project you are not a member of. [14:43:45] ok [14:46:50] but I am a member of moodle project which has that instance [14:47:24] also projectadmin [14:52:17] Coren: [14:58:07] Oren_Bochman, I can't log in as root on that system either… seems broken. [14:59:00] oh, wait... [15:00:06] it is online and working [15:00:07] just soemthing wrong with ssh [15:01:54] I made a new instance he-moodle-25 yesterday and I can't access that either [15:01:55] can you log into any of the instances in that project? [15:02:01] no [15:04:24] Do you mind if I reboot he-moodle-25? [15:04:32] no [15:04:44] It has never been logged into [15:04:47] Hm… maybe I don't need to. [15:04:50] Try access now? [15:05:14] wow [15:05:16] ls [15:05:48] It looked like the processes in charge of /home were hanging… I killed and restarted everything. [15:05:49] Better? [15:06:08] yes [15:06:58] I was having a similar issue moving from toollabs-login to toollbas-dev [15:07:15] Similar how so? [15:07:40] sec [15:09:12] I could not ssh from tool-login into tools-dev [15:09:17] but now it works [15:10:40] ok then :) [15:12:47] does anyone know if there are any best practices regarding the uid and gid exposed in user database names? [15:12:55] just wondering before I push source code to bitbucket with them in it [15:13:00] if I shouldn't [15:18:30] Nettrom: I think they are fixed, so you can rely on them not changing. (I don't even know where you could get "50380" from -- OpenStack API?) [15:19:35] scfc_de: me neither, just wondering if there's any security concerns about exposing them [15:19:59] because I have no idea [15:22:36] Nettrom: I don't think that knowing the uid/gid could give an attacker any advantage, especially on an open system like Tools where accounts are given out freely. [15:24:52] AzaToth: I am interested in adding twinkle like CSD looging to my user scripts [15:24:58] AzaToth: how do I get strarted ? [15:25:40] well, most hardcore nerds would say to read the source ツ [15:26:19] scfc_de: that's what I thought too, couldn't really think of it as critical info... thanks! [15:26:53] Oren_Bochman: https://github.com/azatoth/twinkle/blob/master/modules/twinklespeedy.js#L1174-L1232 [15:27:03] that's the log code for CSD [15:27:36] Nettrom: The usernames are arbitrary; they're generated from uidgid, but they're set in stone. Even if we change the scheme eventually, only new accounts would be affected. [15:28:33] Nettrom: They're just a convenient method of generating known unique names. [15:28:40] AzaToth: thanks I'll read it and ask if I get stucj [15:28:51] you will [15:30:49] Cyberpower678: you there? [15:31:03] Yes. But not for long. [15:31:09] gonna pm you [15:31:15] ok [15:36:06] Coren: /data/project/anomiebot/labs-completion [15:37:36] ugh. *personally* I think the default bash is already annoyingly overloaded with completions, but I see nothing wrong with the concept. :-) [15:40:27] I just got sick of typing "become ano" and having it not work. Then I did "sql en" and got annoyed enough to make a completion file. But yeah, when the completion doesn't work right (e.g. they left something off the list of extensions a command can handle) it's annoying. [15:44:02] It confuses me more when bash tries to be smart and for example ignores files, because the parameter only takes directories as arguments. But this discussion got me to read the man page: "complete -r" :-). [15:45:11] Coren: Also, I'm surprised all AnomieBOT's processes use only 0M, after a startup peak of ~200M ;) [15:45:40] anomie: That... would also confuse me. How odd. [15:45:50] anomie: And they actually work? [15:46:10] Coren: Yeah. I guess that something or other about how I run them confuses the stats collector [15:46:42] anomie: Clearly. I'd love to know how. Clearly, you're still using VM and the ulimit will still work but it's a pain if we can't see how much they are really using. [15:51:44] Coren: If it helps, the command I use to submit each job is /usr/bin/qsub -q continuous -notify -j y -o "$home/botlogs/$jname.job" -cwd -N "$jname" -hard -l "h_vmem=$memlimit" -b y ./bot-instance.pl "$botnum" "$dir" [15:53:01] Yeah, nothing odd there. Does bot-instance fork and set a pgid? [15:53:30] Nope, I took out the forking when I split it into multiple jobs [15:54:12] Unless something internal in perl forks, I suppose [15:54:37] Meh. It's a cosmetic annoyance at worse, so I'll not fret over it for now. [15:56:55] Wow, load on tools-exec-04 is 132.5% [15:59:04] anomie: Small burst, from what I can see. Not that it's super panicking; 6 working processes over 4 CPUs isn't all that bad. [15:59:49] Coren: I just idly wonder what that bot is doing that it's using -04 all to itself [16:01:42] anomie: Actually, it's just fairly IO expensive from what I can see. [16:02:24] And whatever it was doing was relatively short-lived, the load is climbing back down. [16:04:52] Actually, it's going up and down like a seesaw, but the actual process isn't doing all that much. How interesting. [16:05:33] I think he was using MediaWiki underneath. [16:07:22] Coren: I figured out what's causing the 0M thing: Job "test1" is perl -e '$0="changed \$0"; sleep(600);', and "test2" is just perl -e 'sleep(600);'. test2 shows 31M, while test1 says 0M. [16:21:08] anomie: That doesn't seem to make sense to me. [16:22:08] I guess something is trying to match things up using the process name, so changing the process name makes it not find the process. [16:49:44] Coren: pokie [16:49:55] got following error: http://paste.debian.net/9711/ [16:52:04] AzaToth: In meeting atm, will look at it in a bit [16:55:43] AzaToth: IIRC, this is not Tools-specific. [16:55:52] ok [17:01:37] Coren: Turns out to be simpler, actually. At one point gridengine does a sscanf on the contents of /proc//stat to load utime, stime, and vsize. And having spaces in $0 makes it no longer match the format (%*s only matches non-whitespace). So I suppose it would also break if someone's bot script were named "foo bar.pl". [17:02:48] AzaToth: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=706284 seems to be the case here. [17:03:22] I see [17:03:32] I've not noticed it on my local debian [17:04:03] Do you have an entry with multiple options in /etc/*/sources.list.d? [17:42:51] petan: soooooo.... [17:42:54] Coren: ^^ [17:42:59] hi [17:43:01]