[01:36:47] [bz] (8NEW - created by: 2Yuvi Panda, priority: 4Unprioritized - 6normal) [Bug 51990] Allow one tool to add another tool to its group - https://bugzilla.wikimedia.org/show_bug.cgi?id=51990 [02:03:10] YuviPanda: sorry ;) [03:26:23] When did Cron Daemon starts to send mail again? [03:28:00] zhuyifei1999: It always did, but the mail was stuck in the queue for a while. [03:29:39] Coren: it haven't sent any for a long time. But at Wed Jul 24 01:00, suddenly a mail appeared. [05:27:25] Hi all [05:27:42] I have 2 Qs [07:04:38] [bz] (8VERIFIED - created by: 2Chris McMahon, priority: 4Unprioritized - 6major) [Bug 47015] beta cluster: Special:SpecialPages internal error - https://bugzilla.wikimedia.org/show_bug.cgi?id=47015 [07:09:25] [bz] (8RESOLVED - created by: 2Peter Bena, priority: 4Normal - 6normal) [Bug 51936] Set up a server for query relaying - https://bugzilla.wikimedia.org/show_bug.cgi?id=51936 [07:09:42] [bz] (8RESOLVED - created by: 2Peter Bena, priority: 4Low - 6normal) [Bug 51935] wm-bot to tools - https://bugzilla.wikimedia.org/show_bug.cgi?id=51935 [07:11:42] !log tools petrb: created /var/log/glusterfs/bricks/ to stop rotatelogs from complaining about it being missing [07:11:44] Logged the message, Master [07:14:54] @seenrx trigon [07:14:55] petan: I have never seen trigon [07:17:00] Hi petan [07:17:04] Elph: what language is your bot written in [07:17:13] you need to use some mysql lib [07:17:23] yes [07:17:53] i use mysql [07:18:56] ok so what exactly you need to know [07:20:03] when i run my codes after some minutes or seconds code stoped [07:21:35] some error? [07:22:25] give me less than a minute to copy/paste the error here for you [07:22:25] petan: MOTD on deployment cluster: 'Usefull stuff:' should be 'useful' (one 'l') [07:22:38] ok :o [07:22:56] Traceback (most recent call last): [07:22:58] File "catswithoutlanglinks.py", line 39, in [07:22:59] passwd = config.db_password) [07:23:01] File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect [07:23:02] return Connection(*args, **kwargs) [07:23:04] File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__ [07:23:05] super(Connection, self).__init__(*args, **kwargs2) [07:23:07] _mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL server on 'arwiki-p.db.toolserver.org' (110)") [07:23:09] You have new mail in /var/mail/elph [07:23:16] pastebin [07:23:40] Elph: you have invalid hostname [07:23:44] arwiki-p.db.toolserver.org [07:23:57] oh my god [07:24:07] what's the correct ? [07:24:29] arwiki.labsdb [07:24:41] thanks [07:24:44] yw [07:24:49] 2th Q [07:25:07] What's means "You have new mail in /var/mail/elph" [07:25:23] that means you have a new mail... type mutt to read your mails [07:25:51] thanks again [07:35:38] Elph talk in here pls [07:35:48] you need to provide a correct password [07:35:51] passwd = config.db_password) [07:35:52] File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect [07:35:54] return Connection(*args, **kwargs) [07:35:55] File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__ [07:35:57] super(Connection, self).__init__(*args, **kwargs2) [07:35:59] _mysql_exceptions.OperationalError: (1045, "Access denied for user 'elph'@'10.4.0.220' (using password: YES)") [07:36:29] in change "arwiki-p.db.toolserver.org" but i face the error again [07:39:53] !paste [07:39:53] http://tools.wmflabs.org/paste/ [07:39:58] this is a better thing to paste [07:40:20] the error is that you have invalid password [08:15:17] petan: i promise I'll bee off bots-bsql01 today :> [08:15:28] no problem :D [08:15:31] there are still many others [08:15:33] so long as http://ganglia.wmflabs.org/latest/?c=tools&h=tools-db holds up ;p [08:15:54] I can't remove it until they all move off [08:16:01] who else is there? :> [08:16:05] Damianz [08:16:12] ahh, cluebot? :P [08:16:15] probably many others too [08:16:23] I would need to check the sessions [08:17:31] :o [08:21:23] petan: care to help me work out why the webservers cant use / access replica.my.cnf ? :> [08:21:40] addshore: sure [08:21:55] what should the default permissings be? :) [08:21:56] I /think/ because they just don't have rights to read its content [08:22:05] well I don't know this, Coren might [08:22:16] for some reason if it doesn't have www-data [08:22:20] it doesn't work :( [08:22:24] but it doesn't make sense [08:23:03] should it be 600 or might 660 work? ;p [08:23:13] or even 640 [08:23:35] depends [08:23:46] if group is www-data it should be 5400 [08:23:49] * 640 [08:23:52] lol typo [08:23:54] xD [08:24:07] mhhm, goup is the service group [08:24:12] if you are going to trust that nfs works properly, then just 600 [08:24:28] but who knows if thats default, that may have been me 'take'ing it [08:24:33] you definitely shouldn't change group to www-data unless you want anyone to be able to read it ;) [08:24:49] [bz] (8ASSIGNED - created by: 2Niklas Laxström, priority: 4High - 6major) [Bug 48203] Purging does not work on deployment-prep / beta labs - https://bugzilla.wikimedia.org/show_bug.cgi?id=48203 [08:24:50] you know what you can try... copy replica to public_html [08:25:03] mhmhm [08:25:05] cp replica* public*/.my.cnf [08:25:15] would that really help though? :P [08:25:17] idk [08:25:19] *checks permissiong of public [08:25:22] apaches are weird [08:25:33] mhhm, doesnt look like it should change anything [08:25:50] but you must be able to do it as Im sure everyone doesnt have the user and password hardcoded ;p [08:27:09] *goes to see if he can find an example or docs ;p* [09:04:39] [bz] (8PATCH_TO_REVIEW - created by: 2Antoine "hashar" Musso, priority: 4Low - 6enhancement) [Bug 41285] [OPS] foreachwiki on beta use all.dblist instead of all-wmflabs.dblist - https://bugzilla.wikimedia.org/show_bug.cgi?id=41285 [09:34:25] [bz] (8NEW - created by: 2Krinkle, priority: 4Normal - 6enhancement) [Bug 49350] Tool Labs: Change logo - https://bugzilla.wikimedia.org/show_bug.cgi?id=49350 [09:34:56] [bz] (8REOPENED - created by: 2MZMcBride, priority: 4Normal - 6normal) [Bug 36885] wmflabs.org does not resolve - https://bugzilla.wikimedia.org/show_bug.cgi?id=36885 [11:19:02] !shell [11:19:02] http://bit.ly/10eZZoa [11:19:37] Coren, can you please fulfill the shell requests? [11:20:25] petan, ^ [11:21:31] Can any shell admin please fulfill the shell requests? [11:21:55] Cyberpower678: doing [11:22:03] YuviPanda, thanks. [11:22:28] is MBQ someone known? [11:22:35] I don't want to grant shell only for it to turn out to be spam [11:22:51] YuviPanda, no I want WTT [11:22:58] Cyberpower678: already done for WTT [11:23:11] Thank you. [11:23:47] Cyberpower678: yw [11:24:05] YuviPanda, can you also grant tool access? [11:24:15] Cyberpower678: sadly no [11:24:15] ok. [11:24:17] that's only petan, Coren and scfe_de [11:24:25] (and ryan lane, etc) [11:24:42] YuviPanda, ok [11:24:55] Where can you request tool access? [11:25:37] there's a URL, unsure where it is... [11:25:53] !tools [11:25:54] Cyberpower678: http://tools.wmflabs.org/ has 'request access' [11:25:54] There are multiple keys, refine your input: tools-admin, tools-bug, tools-help, tools-request, tools-web, [11:26:14] !tools-requests [11:26:57] !tools-request [11:26:57] https://wikitech.wikimedia.org/wiki/Special:FormEdit/Tools_Access_Request [11:34:58] hi [11:35:24] Cyberpower678 what u need [11:35:45] to give worm that turned tools access. [11:35:53] ok [11:39:22] granted [11:46:10] Thanks. [11:46:16] We're going to work on a new tool together. [12:01:59] YuviPanda: can you mark the request as completed when you finish it? [12:02:08] petan: whoops, forgot. will do [12:02:11] did you already do that? [12:02:11] ok [12:02:13] if not i'll do it now [12:02:14] no [12:02:16] ok [12:03:06] petan, where did that nova thingy go where I can create a new project in tools? [12:03:13] Cyberpower678: does cyberbot report on irc? [12:03:16] done now [12:03:22] Cyberpower678?? [12:03:27] what you talk about [12:03:27] T13|sleeps, no. Why? [12:04:14] There used to be this thing. I forget what it's called. You can see all of the projects you are a member of and add and remove members to it. [12:04:27] petan, ^ [12:06:10] !log tools petrb: test [12:06:12] Logged the message, Master [12:06:35] Just curious if it was also an irc bot in a channel that would allow people to list candidates, see stats of a candidate, etc. [12:06:55] T13|sleeps, interesting proposal. [12:07:15] I think I can conjure up something for IRC. [12:07:58] Cyberpower678 Special:NovaProject [12:08:34] Cyberpower678 what you talk about which candidates? [12:08:57] petan, ?? [12:09:06] (14:06:35) Just curious if it was also an irc bot in a channel that would allow people to list candidates, see stats of a candidate, etc. [12:09:07] (14:06:55) T13|sleeps, interesting proposal. [12:09:32] Calling up RfX candidates on IRC. [12:09:39] aha [12:13:34] hi, I want to set up a nightly distributor in labs. something like this: http://toolserver.org/~pywikipedia/nightly/ [12:13:38] can someone help me [12:13:42] Would only need to be in one channel. Although, it could have a couple lists of tasks to join, leave a full RfX report, and part private channels once or twice a day. [12:15:03] petan: ^ [12:15:11] T13|sleeps, I'm currently backlogged in technical work right now. [12:15:23] Amir1: what you need to help with [12:15:25] T13|sleeps, it'll have to wait a bit. [12:15:35] how i can set it up [12:15:52] Amir1: you cron a job that make a build and publish it [12:16:34] petan: good but a question how i can get some comments about last revision [12:16:51] for example on that page wrote: [12:16:54] r11780 | siebrand | 2013-07-22 15:55:28 +0000 (Mon, 22 Jul 2013) | 1 line [12:16:55] Localisation updates from http://translatewiki.net. [12:17:05] No rush Cyberpower678, and depending on the language you pick, id be happy to help you with it.. [12:17:23] T13|sleeps, always PHP. :p [12:18:20] Php irc bot? [12:18:26] Interesting... [12:18:58] Amir1: probably using git [12:19:23] petan, No Nova credentials found for your account. [12:19:24] I want to use git [12:19:34] What does that mean? [12:19:36] Cyberpower678: relog [12:19:45] petan: is it possible to set a cron in service groups? [12:19:47] relog what? [12:19:51] Amir1: yes [12:19:57] Cyberpower678 wiki [12:20:13] How do I log a wiki? [12:20:19] Cyberpower678 logout login [12:20:25] oh. [12:20:31] ok [12:20:36] you meant relogin. :p [12:20:43] kind of :P [12:21:02] let me check I'll come here if i'll have any questions [12:21:03] thanks [12:21:06] ok [12:21:16] 0: [12:43:10] can someone clear elasticsearch2.pmtpa.wmflabs out of dns? it seems to be stuck after I deleted it. [12:43:48] can't do sorry :( [12:44:18] thanks though! [12:44:32] I think if you delete something and then try to recreate it quickly it gets stuck [12:44:46] it gets removed from the interface but it isn't truely gone [12:50:03] Coren: you helped me with my stuck DNS last time. When you get this, could you pretty please unstick elasticsearch2.pmtpa.wmflabs? I think machines get stuck when you delete them and recreate them "quickly". [13:02:33] manybubbles: That wasn't me; I think it was Ryan. [13:02:57] I don't think he left notes about what the problem is and how to fix it. [13:29:05] Coren: It was Ryan! Sorry, I still get irc handles smashed together sometimes. The problem was that it was "stuck" in DNS.... [13:36:06] [bz] (8RESOLVED - created by: 2Chris McMahon, priority: 4High - 6major) [Bug 51988] login broken on beta labs - https://bugzilla.wikimedia.org/show_bug.cgi?id=51988 [14:24:46] are the diff sizes stored in the DB (I'm talking about those diffs shown in user contribs page) [14:37:03] OrenBochman: No, but you get rev_len so the diff size is the difference. [14:47:54] Coren, YuviPanda, can you talk me through the groups-in-groups bug? What will it accomplish to add one service user to a different service group? [14:48:12] andrewbogott: let one tool access files of the other tool. When you want to share config [14:48:15] or similar things [14:48:19] and want to be selective [14:48:22] and not put them all in /shared/ [14:48:25] readable by everyone [14:48:28] It will allow "meta" tools that share configs. Think "toolkit" [14:48:44] Ah, ok, so we're trying to provide access to tools running as the service group. [14:48:51] * Coren nods. [14:48:52] Not users who are members of the group [14:48:58] yup [14:49:00] That is what I was missing! [14:49:07] Exactly. No transitive stuff there, it really is the /tool/. :-) [14:50:01] Hierachical grouping of tools with member trasnclusion would be amusing, but I doubt there's a serious use case for it that's worth the implementation nightmare. :-) [14:50:22] OK -- I'll get back to work on the gui code. It's a bit hairy since service users are totally different object types from normal users. [14:50:37] Coren, glad to hear it, that idea was making my head spin. [14:51:36] :) [14:51:57] andrewbogott: Ryan closed that bug with a summary of 'this will totally destroy all security!', but I've no idea what he means [14:52:02] and he might have also gotten the wrong idea? [14:52:06] oh, hm. [14:52:15] I think he was thinking of users, than tools [14:52:33] can you link me to the bug? [14:52:57] moment [14:53:16] https://bugzilla.wikimedia.org/show_bug.cgi?id=51990 [14:53:22] andrewbogott: he said 'sanity', not security [14:53:22] my bad [14:53:34] (or my sleep deprived hallucination) :) [14:53:54] Coren: you able to lend a hand? scripts that work on the TS dont work on labs [14:54:20] Betacommand: Sure. Most of the time it's minor details. Point me at where you have issues. [15:16:48] [bz] (8NEW - created by: 2Krinkle, priority: 4Normal - 6enhancement) [Bug 49350] Tool Labs: Change logo - https://bugzilla.wikimedia.org/show_bug.cgi?id=49350 [15:26:44] [bz] (8NEW - created by: 2Liangent, priority: 4Unprioritized - 6normal) [Bug 52033] liangent-py tool is missing in the tool list - https://bugzilla.wikimedia.org/show_bug.cgi?id=52033 [15:27:22] Coren: and… once the user has requested that a service user be added to a service group, what should actually happen? I can add the service user as a member in ldap, but I note that user local-sgroup is not currently a member of group local-sgroup [15:29:44] andrewbogott: Just adding as a member is fine. The service group users aren't members of their groups because it's their primary group. [15:29:58] (I.e.: it's be pointless to also have it as additional member) [15:31:14] Hm… ok. I guess I don't fully understand how permissons get translated from ldap into actual system settings. [15:43:02] [bz] (8PATCH_TO_REVIEW - created by: 2Antoine "hashar" Musso, priority: 4Low - 6enhancement) [Bug 41285] [OPS] foreachwiki on beta use all.dblist instead of all-wmflabs.dblist - https://bugzilla.wikimedia.org/show_bug.cgi?id=41285 [16:25:29] Coren 10x [16:27:18] Coren, YuviPanda, https://gerrit.wikimedia.org/r/#/c/75897/ [16:29:42] andrewbogott: (+2'ed, not merging until you feel you are ready) [16:30:37] I think gerrit took the initiative on merging :) It's not live on wikitech though -- probably good to give Ryan a chance to veto. [16:30:51] OrenBochman: "10x"? [16:31:19] Dafu? I very specifically didn't submit. [16:32:32] petan: I made cron as you said but I'm not sure It's working [16:32:39] how can i check that [16:32:49] Coren: thanks for the diff tip from before [16:32:49] or is there any log? [16:35:18] Coren: dude. [16:35:30] I marked that bug as resolved wontfix [16:35:56] ... hm, so you have, noting while I was about to mark it resolved fixed. [16:36:02] Why in blazes? [16:36:25] we're just asking for problems if we allow this [16:36:33] ... I don't see it. How so? [16:37:00] it makes the interface more complex, it makes the internal logic more complex, it makes the docs more complex [16:37:11] it makes support more complex [16:37:34] take, for instance [16:37:35] or become [16:37:46] group permissions [16:37:47] ... I still don't see how. Granted, it's a more "advanced" feature, but hardly world-rocking. [16:38:11] None of those things are affected. [16:38:23] a user can become a tool user [16:38:29] Sorry, Ryan_Lane, I reponed because I thought you were vetoing "all members of project A in project B" since that's roughly what the bug title was at the time. [16:38:31] then that user will write to folders [16:38:55] it'll have group permissions of something and it'll be wrong [16:38:58] then people will want to take it [16:39:04] Ryan_Lane: No it won't. [16:39:06] but it'll be in the wrong service group [16:39:19] Wait, what do you think this does, in practice? [16:39:31] it allows a tool user to be a member of another tool [16:39:36] No it doesn't. [16:39:45] Wait, yes. [16:39:47] It does. [16:39:51] exactyl [16:39:52] But that's all it does. [16:40:04] that adds the entire group of people in tool A into tool B [16:40:26] which means that people in tool B can effectively be in tool B without tool B people knowing about it [16:40:32] err [16:40:38] switch the last B with an A [16:40:57] it's a form of recursive groups [16:41:05] not a fan [16:41:19] I don't see how it does. For one, the only plausible use case of this is tools sharing maintainers in the first place. [16:41:35] For toolkit-like tools. [16:41:44] then they should use a single account and put all their tools under it [16:42:05] or they should make their configs and data public read [16:42:17] I really don't see where you feel the problem is. Perhaps if you gave me an example? [16:42:36] I become tool A [16:42:48] I write some stuff into /tmp meant for tool B [16:42:53] but I don't use the correct group [16:43:03] now tool A needs it [16:43:09] tool A can't tak [16:43:11] *take [16:43:23] tool A can't take stuff in /tmp period. [16:43:37] You can only take stuff in a directory you own. [16:43:57] you know what I mean ;) [16:44:09] The point of allowing too A permission to tool B is exactly so that it can read and/or write to tool A's home. [16:44:24] I.e.: the opposite scenario than the one you said wouldn't work. [16:44:37] And then, the group would necessarily be correct because of SGID. [16:45:04] AzaToth: your labs username? [16:45:19] does anyone actually need this feature right now? [16:45:44] I could use it [16:45:44] I've had it requested by three maintainers of large sets of tools (who are, clearly, the most obvious users) [16:45:50] to share redis 'secret' keys [16:45:52] now I've to copy paste [16:46:37] guys how can check a cron is working or not [16:47:11] Amir1: There really isn't a way except checking for side effects unless your cron entry has output (in which case it'd be mailed to you) [16:47:42] there is outputs [16:47:49] I'm not sure It's mailing to me [16:48:10] what's my mail in Wikitech [16:48:29] everytime I login It says "you have mail" but I don't where it is [16:48:39] Ryan_Lane: I would expect that most "tools" that include other tools in their permissions are in fact toolkits that do not do anything on their own but are used by other tools (including credentials, etc). I know that's MZM's use case at least. [16:48:40] Amir1: can you check in wikitech.wikimedia.org preferences? [16:48:50] let me check [16:48:52] sumanah: Wrong mail. :-) [16:49:01] Oh. [16:49:03] * Ryan_Lane grumbles [16:49:09] Amir1: Unless you set up a .forward, your mail remains local. You can use 'mailx' to read it from the console. [16:49:33] sumanah: hi [16:49:36] not a fan of the complexity of this change, but I see the use case [16:49:36] how you're doing? [16:49:40] YuviPanda: azatoth [16:49:44] AzaToth: adding [16:50:11] username is shell login name right here? [16:50:17] I think so [16:50:39] Hi Amir1 - I'm okay! And you? [16:50:45] Congrats on your pywikipedia work. [16:50:54] good [16:51:01] AzaToth: done [16:51:01] Ryan_Lane, should I add a warning or interstitial that says "checking this box means you trust everyone in this project, and everyone in projects in this project, etc."? [16:51:05] you heard that [16:51:06] thank you [16:51:13] andrewbogott: yes [16:51:14] please [16:51:26] a warning is fine [16:51:35] interstitials are painful [16:51:37] YuviPanda: have never used role accounts before, so I assume I would need some pointers [16:51:44] andrewbogott: Yeah, big scary warning would help scare away people who aren't sure that's what they want. [16:51:46] !tools-doc [16:51:49] hmm [16:52:02] I'm currently seting this up [16:52:02] http://tools.wmflabs.org/pywikipedia/ [16:52:40] for nightly distribution but I'm not sure corn of this service group is working properly [16:52:44] *cron [16:53:11] Amir1: Your mailbox should contain the output of the cron runs. [16:53:15] * sumanah goes to get lunch [16:53:25] Coren: how i can set up a forward mail? [16:53:46] Amir1: Just but an email address, alone on a line, in a file named '.forward' in your home. [16:53:56] Amir1: Any new mail will be sent to that address. [16:54:00] ok [16:54:30] AzaToth: are you logged in to tools-login? [16:54:31] * Coren checks that this is part of kma500's documentation plan. [16:55:27] figured it out [16:55:35] had to logout and in again for become to work [16:55:56] YuviPanda: I notice the lack of timestamps in the err file ヾ [16:56:53] AzaToth: err, look at ~/logs/lolrrit-wm [16:56:55] .log [16:56:58] has timestamps [16:57:07] AzaToth: and yeah, .err is generated by SGE. no timestamps [16:57:26] sge? [16:57:27] .err is just the stderr of the job sent untouched. [16:57:31] k [16:57:51] A useful trick is to start the scripts to send to the grid with something like "date >&2" [16:57:59] So that it sends a timestamp to stderr. [16:58:14] But that makes your .err grow even if there are no errors. So YMMV [16:58:24] nice one, Coren. [16:58:43] (Still useful when debugging though) [17:03:33] dang, htmlform never does what I'm expecting. https://wikitech-test.wmflabs.org/w/index.php?title=Special:NovaServiceGroup&action=addmember&projectname=sg&servicegroupname=local-corn&returnto=Special%3ANovaProject [17:03:43] YuviPanda: perhaps set debug: true, showErrors: true [17:03:59] AzaToth: for node irc? [17:04:18] Coren: The one tool that I have that uses an auth system I've mostly deprecated, BTW. [17:04:22] Ryan_Lane, Coren, sufficient? ^ [17:04:30] watcher is now replaced by the info action in core. \o/ [17:04:41] YuviPanda: yea [17:04:54] AzaToth: hmm, add it and restart tool? [17:04:56] :D [17:05:02] Elsie: Yeay! [17:05:05] might get spammy, but for now it could be relevant [17:05:29] AzaToth: yeah. but then we've to wait for a disconnect [17:05:32] live hacks are okay :) [17:06:08] Coren: https://en.wikipedia.org/w/index.php?title=Main_Page&action=info#mw-pageinfo-watchers [17:06:17] It's even got row highlighting now. [17:06:19] So fancy. [17:06:38] Elsie: Plus with got OAuth and OpenID coming Real Soon Now™ which will cover most other use cases. [17:07:28] I think there is problem in my cron [17:07:30] http://paste.ubuntu.com/5911814/ [17:08:01] my project is "pywikipedia" not "pywikibot" [17:08:11] If you want I can copy my cron too [17:08:29] Amir1: Also, the file seems to be named 'run_sync' not 'run_syn' :-) [17:08:34] (PS1) AzaToth: adding gitreview [labs/tools/grrrit] - https://gerrit.wikimedia.org/r/75908 [17:08:54] Coren: this is my cron http://paste.ubuntu.com/5911825/ [17:09:12] sorry It's not my cron [17:09:16] It's commands [17:09:41] YuviPanda: ok, how I correctly restart it? [17:09:57] AzaToth: ./run.bash [17:09:59] Amir1: I think the error is in your crontab. [17:10:01] made a live hack [17:10:26] YuviPanda: from the src dir? [17:10:33] yup [17:10:36] this is my cron [17:10:37] looks like you found it :) [17:10:40] ツ [17:10:42] PATH=/usr/local/bin:/usr/bin:/bin [17:10:44] 01 * * * * /data/project/pywikipedia/commands [17:11:14] what i linked is contents of "commands" file [17:12:07] * Elsie holds his breath. [17:12:15] YuviPanda: thought there was a standard way to do it instead of a custom bash script [17:12:23] AzaToth: you do a jstop and then a jstart. [17:12:29] that's what that script does :P [17:12:32] err [17:12:32] jsub [17:12:50] Coren: what do you think? [17:13:06] heh [17:13:19] with a 'sleep' in between to make sure the job has stopped [17:13:57] Coren: why no jrestart thingi? ツ [17:14:22] AzaToth: ... because I never considered it. :-) [17:14:30] can someone help me? [17:14:59] Coren: heh [17:15:15] Coren: btw, at last I figured out how to handle the userindex issue [17:15:28] Coren: http://paste.debian.net/18582/ [17:15:28] Amir1: I'm not getting it; this should work. Do you mind if I become your tool to check it out? [17:15:35] nice right? ツ [17:15:46] of course I don't mind [17:16:00] AzaToth: Iz improved! [17:16:06] do whatever you want to make it works [17:16:07] ! [17:16:08] There are multiple keys, refine your input: !log, $realm, $site, *, :), ?, access, account, account-questions, accountreq, add, addresses, addshore, afk, airport-centre, alert, amend, ask, awstats, bang, bastion, beta, bible, blehlogging, blueprint-dns, bot, bots, botsdocs, broken, bug, bz, chmod, cmds, console, cookies, coren, Coren, credentials, cs, Cyberpower678, damianz, damianz's-reset, db, del, delete, demon, deployment-beta-docs-1, deployment-prep, doc, docs, domain, enwp, epad, etherpad, evil, extension, failure, flow, forwarding, gerrit, gerritsearch, gerrit-wm, ghsh, git, git-puppet, gitweb, google, group, hashar, help, helpmebot, hexmode, home, htmllogs, hyperon, info, initial-login, instance, instance-json, instancelist, instanceproject, ip, keys, labs, labsconf, labsconsole, labsconsole.wiki, labs-home-wm, labs-l, labs-morebots, labs-nagios-wm, labs-project, labstore3, labswiki, leslie's-reset, link, linux, load, load-all, log, logs, logsearch, mac, magic, mäh, mail, manage-projects, meh, mobile-cache, monitor, morebots, msys-git, nagios, nagios.wmflabs.org, nagios-fix, nc, newgrp, new-labsuser, new-ldapuser, nova-resource, op_on_duty, openstack-manager, origin/test, os-change, osm-bug, pageant, password, pastebin, pathconflict, petan, petan..., petan-build, petan-forgot, ping, pl, pong, port-forwarding, project-access, project-discuss, projects, proxy, puppet, puppetmaster::self, puppetmasterself, puppet-variables, putty, pxe, pypi, python, pythonguy, pythonwalkthrough, queue, quilt, rb, reboot, remove, replicateddb, report, requests, resource, revision, rights, rq, rt, rules, Ryan, Ryan_Lane, ryanland, sal, SAL, say, screenfix, search, searchlog, security, security-groups, seen, sexytime, shellrequests, single-node-mediawiki, smb, snapshits, socks-proxy, ssh, sshkey, start, stats, status, StoneB, stucked, sudo, sudo-policies, sudo-policy, svn, t13, taskinfo, terminology, test, Thehelpfulone, tooldocs, tools-admin, tools-bug, tools-help, tools-request, tools-web, trout, tunnel, tygs, unicorn, venue, vim, vmem, whatIwant, whitespace, wiki, wikitech, wikiversity-sandbox, windows, wl, wm-bot, wm-bot2, wm-bot3, wmflabs, zhuyifei1999, [17:16:39] AzaToth: Hi, I'm big fan of your work "Twinkle" [17:17:48] someone localized it for Persian Wikipedia I don't why and more importantly "HOW" but It has been widely used in fa.wp [17:18:07] Amir1: thanks [17:18:24] * AzaToth read first fa.wp as "fap" [17:19:01] Amir1: localization is difficult mostly becuase policies, procedures and such differs a lot [17:19:33] and whole twinkle relates to a fuckload of templates [17:19:40] Amir1: I've made your /data/project/pywikipedia/commands into a proper script, and made it executable. That's the only thing I can see. [17:20:01] You might want to invoke it manually (as your tool) and see if it does what you expect; this should be the same as what cron does [17:20:14] Coren: heh, I made "User.find_by_user_name('C').revisions.last.page.revisions.first" and got: "" [17:20:20] Yeah It's very difficult specially when your wiki is non-Latin and RTL [17:20:26] Amir1: heh [17:20:50] Coren: I checked that way [17:21:01] It worked [17:21:45] Amir1: I recommended everyone who wanted to localize it to use github so they can follow and merge updates: https://github.com/azatoth/twinkle/network [17:22:07] Amir1: mostly it's jimmyxu who is clearly updating always [17:22:51] AzaToth: sounds good I'll tell Reza (my friend who localized your tool) [17:23:01] ツ [17:25:32] Coren: your onwiki name isn't Coren right? [17:26:22] AzaToth: Depends on what hat you mean. "Coren" in most places, "MPelletier (WMF)" with my staff hat on on the main projects. [17:27:29] Coren: never got a result trying to do "SELECT `revision_userindex`.* FROM `revision_userindex` WHERE `revision_userindex`.`rev_user` = 12013 ORDER BY `revision_userindex`.`rev_id` DESC LIMIT 1" [17:27:32] it timed out [17:27:54] coren: Do you have a minute to help me with some questions on database stuff? I have them marked on the etherpad: http://etherpad.wmflabs.org/pad/p/Tool_Labs_Sprint_July_23 [17:28:20] AzaToth: That's odd. That should have been fairly fast. [17:28:26] kma500: Sure, gimme a minute. [17:28:28] Coren: 12013 is you btw [17:28:35] thank you! [17:28:52] Coren: tool 11 sec in sql direct [17:29:37] now it's a bit faster [17:29:39] D, [2013-07-25T17:29:15.970151 #22732] DEBUG -- : RevisionForUser Load (9899.1ms) SELECT `revision_userindex`.* FROM `revision_userindex` WHERE `revision_userindex`.`rev_user` = 12013 ORDER BY `revision_userindex`.`rev_id` DESC LIMIT 1 [17:30:25] Coren: goes half a second faster on each try now [17:30:30] or not [17:30:44] seems to settle on 8.5s [17:31:00] Coren: though imo it should take ms [17:31:03] Not sure rev_id is actually indexed. order by rev_timestamp desc gives (0.03 sec) [17:31:49] Coren: I assumes always that the primary ID _is_ indexed [17:32:13] AzaToth: That's not always true on mediawiki tables, and certainly is often false on the very old tables. [17:32:22] hmm [17:32:48] that fucks a lot of things up [17:35:03] MariaDB [enwiki_p]> show index in revision_userindex; [17:35:03] Empty set (0.03 sec) [17:35:15] that's helpful... [17:35:15] AzaToth: Ah, I see why: [17:35:31] (You can't show index in the _p databases; they are views) [17:35:44] yea, figured that [17:35:51] The primary key is (rev_page, rev_id) [17:35:58] Not just rev_id [17:36:00] oh [17:36:26] Porbably hysterical raisins. [17:37:07] There is, OTOH, an index on (rev_user, rev_timestamp) which is why /that/ query takes ms to complete. [17:38:10] It /is/ using the rev_id index (which isn't primary) but since there is no (rev_user, rev_id) index you're stuck having to traverse an index with cardinality 367217909(!) [17:39:28] (I.e. it has to walk the index in order until it finds the first revision with the right rev_user) [17:40:09] In other words: if you want to sort a user's revisions, do it over the timestamp. :-) [17:42:32] better [17:42:53] RevisionForUser Load (415.9ms) SELECT `revision_userindex`.* FROM `revision_userindex` WHERE `revision_userindex`.`rev_user` = 12013 ORDER BY `revision_userindex`.`rev_page` DESC, `revision_userindex`.`rev_id` DESC LIMIT 1 [17:42:59] down to a half second [17:43:38] Still, IMO, heavier than it needs to be. rev_timestamp would still do what you want and involves no index scan at all. [17:44:31] Coren: but... [17:44:38] (rev_user, rev_timestamp) is unique? [17:45:02] a user cannot make two revisions in the db at the same time? [17:45:52] AzaToth: It's not unique; it's /possible/. If you want to guard against that just /also/ sort by rev_id desc [17:46:45] But honestly, is the difference significant? What is the use casE? [17:46:56] kma500: Looking at it now. [17:47:05] great. thank you! [17:48:05] Coren: the use case is to not having to take internal database thingis into account [17:48:53] No, I mean, in your use case does it matter if the revision this returns is one of (possibly) a few that were made in the same second? [17:49:34] it limits to 1, so it will take one of those who was made at that time [17:49:35] Because the mediawiki schema is old and klunky, and has a lot of artefacts. You're never going to really get around some idiosyncracies in what is indexed how. :-) [17:49:43] heh [17:49:49] {{sofixit}} [17:50:08] how hard could it be to make a schema change! [17:50:39] AzaToth: Never say that in front of our DBAs, they'll trout you into a stupor if they survive the stroke. :-) [17:50:47] hehehe [17:50:56] * YuviPanda moves everything from binary to utf8 [17:52:43] kma500: Responded. [17:53:14] thanks, coren. The pad just kicked me out again, but I'll look when I can get in again. [17:54:36] hm [17:54:40] instance deletion has a bug :( [17:54:40] Coren: if I add rev_id it's 140ms, and without it's 27ms [17:54:59] Ryan_Lane: prolly because some DBA forgot to add a index [17:55:03] :D [17:55:18] I think it's due to my recent refactor [17:55:22] Ryan_Lane: about elasticsearch2 - it looks like instances get deleted from the list _instantly_ but that isn't really true [17:55:33] manybubbles: yeah, I have a bug open for that ;) [17:55:36] AzaToth: That's normal, because now it needs to calculate an intersection. [17:55:57] coren: Thanks! I had a couple more questions further down (through 7.3) , all marked with <<<<<<<<<<<<<----------- [17:55:59] nova responds with "accepted", as it does the action asyncronously [17:56:11] I have to run to a meeting, but I'll be back around one. [17:56:13] the interface should change the status to ACTIVE (deleting) [17:57:03] kma500: Will hunt them down. [17:58:12] thanks! [18:12:28] hi xqt [18:12:39] I added you [18:12:56] Amir1: I hope so ;) [18:14:25] :) https://gerrit.wikimedia.org/r/#/admin/groups/uuid-0b39cb41841ce0965df5eb6c73ab7c74749f674e [18:14:32] xqt: ^ your name [18:17:54] Amir1: I am curious whether the access works as expected. [18:18:33] It's a little complicated [18:19:33] when you submit a patch people review it some bots make pep8 on it and after accepting by one of us It would be merged [18:19:36] I'm confused about environment setup in Labs: if I need to run my script with qsub, how can I ensure an adequate running environment for it? E.g. can I make sure some Ruby gem is installed? [18:19:53] Also, where can I store files? Is there a shared NFS mount or something? [18:27:46] abartov: The simple answers: [18:28:04] (1) any package that isn't there that you need, we deploy via puppet. [18:28:22] (2) your home and your tool's home are both on NFS and accessible from all compute nodes. [18:29:16] (Although, for ruby gems, you can also do a local installation; since it's on NFS it's available everywhere too) [18:38:02] Coren, thanks! re 2. that's neat, a local RVM would be the most convenient. [18:38:55] hi :) What's the tool-labs channel? [18:39:22] What about space? Specifically, I'm thinking of experimenting with the dumping tools, and at some point this would include testing with images (e.g. to build a ZIM file). Can I use up, say, 10GB for storage in my homedir with impunity? [18:40:04] fale_: this one ;) [18:40:40] abartov: the dumps are already available in shared storage, and I think 10G should be fine :) It's NFS though, so not going to be *that* fast. I'm sure Coren will poke you if you're using too much disk :) [18:40:53] YuviNoPower: I dunno why but I thought this was for all labs project and that there was a specif chan for -tools :D [18:41:01] fale_: nope :) [18:41:23] YuviNoPower: I've seen you've requested a labs/tool/PROJECT too :) [18:41:32] YuviNoPower: I was feeling lonely :D [18:41:34] fale_: indeed, it's running greg-g right now [18:41:35] aaargh [18:41:36] grrrit-wm: [18:41:37] not greg-g [18:41:49] :( [18:42:06] fale_: I'm going to add a few more shortly. labs/tools/gerrit-to-redis, labs/tools/SuchABot, and perhaps labs/tools/github-to-redis :) [18:42:19] * YuviNoPower runs greg-g too [18:42:49] fale_: will probably do tomorrow :) [18:43:13] YuviNoPower: :) I'm planning too to increase my labs/tool/ projects, but I'm always stopped by the fact that I've always to use Coren (or other admin) time :( [18:43:15] YuviNoPower: speed is not the issue, so that's fine. [18:43:29] fale_: poke ^demon. who isn't here. grr [18:43:41] fale_: we should make a request queue [18:43:43] YuviNoPower: the point is to _test_ dumping (for arbitrary users, not dumping by WMF), so existing dumps are irrelevant. [18:44:10] YuviNoPower: yes, in this way we don't interrupt admins while they do more urgent work :) [18:44:17] fale_: I usually just put them on github first, then have it created on gerrit, and then import [18:44:20] abartov: ah, right. [18:44:39] abartov: should be fine. If you want more resources, you can always create your own labs project / instance :) [18:45:00] YuviNoPower: yep, last time I tried... you had to save me because I was merging gerrit into github instead of the opposite :D [18:45:27] fale_: yeah :D [18:45:27] git is fun [18:45:27] because you can be 'saved' eventually. no code loss [18:46:09] YuviNoPower: that's the good part...even if the history is "peculiar" since commit 1 is 20 days ahead than commit 2 :D [18:46:16] yeah :) [18:47:21] YuviNoPower: how's named the shell that fakes the connection for easy completitions to be able to work even with crappy line? [18:47:31] fale_: mosh [18:47:36] YuviNoPower: thanks :) [18:47:40] yw [18:48:26] * fale_ had some kind of bad-update in his ubuntu laptop and a lot of programs are not able to read their own configs and history :( [18:49:51] bastion does not support mosh :( [18:52:55] Coren: there is a bug in output of cron [18:53:01] /bin/sh: 1: /data/project/pywikipedia/commands: Permission denied [18:54:54] fale_: I use that with tools-login all the time [18:55:42] YuviNoPower: on tools-login works for me too... but I was trying to login to beta and since I0ve found no public ip I thought to connect through bastion [19:00:20] beta labs is 503 right now? [19:00:30] hmm, maybe not [19:04:17] Coren, is there a way to retrieve table setups? [19:05:28] petan, ^ [19:24:26] [bz] (8NEW - created by: 2Tim Landscheidt, priority: 4Unprioritized - 6trivial) [Bug 52048] generic::gluster-client's logrotate.d file relies on directory but doesn't ensure its existence - https://bugzilla.wikimedia.org/show_bug.cgi?id=52048 [20:13:17] [bz] (8RESOLVED - created by: 2Chris McMahon, priority: 4Unprioritized - 6major) [Bug 50622] Special:NewPagesFeed intermittently fails on beta cluster; causes test failure - https://bugzilla.wikimedia.org/show_bug.cgi?id=50622 [20:13:47] [bz] (8RESOLVED - created by: 2Chris McMahon, priority: 4Unprioritized - 6major) [Bug 50623] Entering AFTv5 feedback causes error - https://bugzilla.wikimedia.org/show_bug.cgi?id=50623 [20:13:47] php is giving me bad errors :( I think the cause is that the php version on tools-login is old :( [20:14:51] * fale_ 's theory has been confirmed by the php doc. [20:15:12] Coren: will we see soon php >=5.4.0? [20:15:37] depending on how you define "soon" [20:15:48] Coren: next week :D [20:16:06] Then no. [20:16:13] It's months away. [20:16:15] Coren: the fact is that php has changed a lot from 5.3.x to 5.5 (actual one) [20:16:30] Coren: I see [20:16:51] That large number of change is the reason why it's a ways off. [20:17:08] Coren: it should not brake any old script [20:18:01] *break [20:55:46] YuviPanda: I think I will require new repos tomorrow too :D. Will you create the queue or I will do it? [20:55:54] fale_: do it! [20:56:05] YuviPanda: on wikitech? [20:56:10] yup [20:56:34] YuviPanda: plain page, isn't it? (aka nothink weird like the shell request) [20:56:47] yeah plain page is good enough [20:56:47] *nothing [20:56:57] YuviPanda: oki :) [20:57:00] :D [20:59:12] @notify russavia [20:59:13] I'll let you know when I see russavia around here [20:59:39] coren: thanks for the database answers! I was looking at the grid section. Can you clarify what this means: "You do not normally control which execution host will eventually run the job, and should therefore only access directories that are shared between all hosts" [what does access mean in this context?] [21:00:29] "read or write files in" [21:00:51] thanks [21:00:53] Or, for that matter, chdir to. Access. :-) Not sure how else I'd put it. [21:15:58] a question: http://tools.wmflabs.org/pywikipedia/core/ [21:16:15] "this webserver does not list directory contents by default" [21:16:22] how can i enable it? [21:16:51] i want to enable list directory for a certain service group [21:17:02] Is that even possible? [21:17:21] Amir1: yeah, lemme find the setting [21:17:37] legoktm: hi [21:17:40] Amir1: also, you know we already had a "pywikibot" tool? :P [21:17:56] local-legobot@tools-login:/data/project/legobot/public_html$ cat .htaccess [21:17:57] Options Indexes [21:18:03] Amir1: ^create that .htaccesss [21:18:26] this is for nightly distributions [21:18:37] http://tools.wmflabs.org/pywikipedia [21:19:16] > Q: OMG BUG!!!!!!1111oneoneone <-- :> [21:19:37] I copied from toolserver.org/~pywikipedia [21:20:21] legoktm: what should be content of .hataccess [21:20:22] ? [21:20:33] "Options Indexes" [21:22:39] ok [21:22:58] Amir1: legoktm - have you been adding to http://etherpad.wmflabs.org/pad/p/Tool_Labs_Sprint_July_23 today? :) [21:23:15] sumanah: er what's that? [21:23:32] oh, i skimmed the ML post [21:23:40] sumanah: what should we be adding? :) [21:23:42] legoktm: today's the last day of a documentation sprint for Tool Labs where people have been collaborating on improving the HOWTO for Tool Labs [21:23:53] legoktm: Stuff about pywikipedia would be very welcome [21:24:31] sumanah: let me see [21:24:35] Thanks! [21:26:07] legoktm: do you have time to do it? [21:27:27] not really, i have to leave soon. i see someone already added my help page on how to install pwb-core though [21:28:19] sumanah: I think he is not here [21:35:10] sumanah: It's 2 AM here and I'm really tire (half sleep already) I'll do it if i have a little more time [21:35:30] Thanks Amir1 - it's also fine to help improve the documentation on the wiki once it's moved there after the sprint :) [21:35:36] I appreciate any help you can give [21:37:28] sumanah: can you give me deadline? [21:37:46] Amir1: No, sorry. [21:54:04] sumanah: what the hell I'll do it as soon as coffee kicks in [21:59:54] (PS1) AzaToth: adding color filter for the IRC output [labs/tools/grrrit] - https://gerrit.wikimedia.org/r/76016 [21:59:55] (PS1) AzaToth: adding package.json for easier npm install [labs/tools/grrrit] - https://gerrit.wikimedia.org/r/76017 [23:28:23] OK, I understand this might be a silly question, but can anyone help me troubleshoot my sshing into tools-login? [23:37:57] !log tools added myself to lolrrit-wm tool [23:37:59] Logged the message, Master [23:42:14] twkozlowski , what kind of trouble are you running into? [23:42:27] And, are you on linux, windows, mac, ??? [23:43:19] andrewbogott: well, I can't connect to tools-login at all :-) [23:43:39] (blablabla) debug1: Server accepts key: pkalg ssh-rsa blen 277 [23:43:42] Connection closed by 208.80.153.224 [23:45:17] Have you been able to connect in the past? [23:45:27] And, can you tell me what ssh client you are using? [23:46:08] OpenSSH via command line from under a Linux machine, andrewbogott [23:47:22] …and have you been able to connect in the past? [23:47:34] I'm not exactly sure. I might have connected once. [23:47:51] Ryan_Lane: fyi, 3 unrelated irc bots just all timed out again in labs from freenode (over the course of 20 minutes) [23:47:52] Just to try it out, andrewbogott [23:47:56] after almost 2 days of up time [23:48:11] ah. I wonder if that's the issue with grrrit-wm [23:48:23] Could be, in that case it'd be a problem at freenode [23:48:30] though I don't see any non-labs bots timing out anywhere [23:48:42] except for gerrit-wm [23:48:58] but that could be affected in theory by something at wmf's end somehow [23:50:11] Krinkle, Ryan_Lane, just now I was ssh'ing to tools-login and it hung for >1 minute before finally connecting. [23:50:18] Maybe related [23:50:26] indeed, same here [23:50:31] took a long while to connect [23:50:51] the bots run on sge, so it isn't the login host itself being overloaded afaik, but the connection thing in general could be related [23:50:57] grid* [23:53:46] Krinkle: actually, morebots timed out on wikitech-static at the same time [23:54:03] indeed [23:54:23] Ryan_Lane: Where is wikitech-static btw? [23:54:34] rackspacecloud [23:56:45] Ryan_Lane: Does the linode stuff still exist? [23:56:47] https://wikitech.wikimedia.org/wiki/Wikitech [23:57:26] Krinkle: no [23:58:53] did you make wikitech-static and change wikitech? [23:59:02] * Ryan_Lane links to wikitech-static from wiktiech [23:59:02] I did, but then you undid it [23:59:22] or rather I did [23:59:25] I didn't undo it, I edited it at the same time as you using VE and it didn't throw a conflict [23:59:26] or not [23:59:33] right