[02:40:50] Cyberpower678: Hi CP, nice to meet someone on the night shift ;) [02:41:22] I'm only here for another hour or two. [02:41:34] hedonil, where's petan? [02:41:56] Cyberpower678: I haven't seen him in a while [04:45:41] Cyberpower678: you there? [04:46:29] * Cyberpower678 is really going to bed now. [04:46:56] maybe. I don't know his/her timezone. :-( [04:47:30] Pratyya: look at the nick change right after you left the message, as a hint ;) [04:47:55] What was your question anyway, Pratyya ? [04:48:23] He was helping me in bn wiki. about a template which isn't working there. [04:48:31] https://bn.wikipedia.org/wiki/%E0%A6%9F%E0%A7%87%E0%A6%AE%E0%A6%AA%E0%A7%8D%E0%A6%B2%E0%A7%87%E0%A6%9F:Statustop [04:48:31] Which? [04:48:34] * huh looks [04:48:43] but working fine at test.wikipedia.org [04:50:05] Pratyya: I see icons in the top right of https://bn.wikipedia.org/wiki/ব্যবহারকারী_আলাপ:Pratyya_Ghosh [04:50:06] but no status [04:50:07] hm [04:50:58] what do you think? [04:51:03] Pratyya: show me the equivalent on testwiki? [04:51:38] https://test.wikipedia.org/wiki/Template:Statustop [04:51:41] * huh is not very active on bnwiki however https://bn.wikipedia.org/wiki/বিশেষ:Contributions/PiRSquared17 [04:51:49] Maybe a CSS issue [04:53:22] where I'm gonna find Pi17? [04:53:31] Me. [04:54:32] Pratyya: I think we've met before [04:54:33] you use your nickname "huh" ? [04:54:37] Currently, yes. [04:54:47] yeah we've met before. [04:55:07] You granted me autopatrolled in meta, also I used your global UP script. [04:55:43] Btw you should give users a chance to add a custom summary at Global UP huh [04:56:02] all edits are "editing Global userpage" [04:56:04] that would be easy to implement BTW [04:56:15] maybe tomorrow [04:56:31] * huh still checking out statustop [04:57:18] I don't read Devanagari script very well, and even if I did, I don't know Bengali. [04:57:22] But I'll try my best [04:57:42] to understand what you're trying to do [04:57:53] you need any translation ask me directly. [04:57:56] huh: [04:58:28] Okay, will do [05:04:03] Pratyya: meaning ব্যবহারকারী and ব্যস্ত​ [05:04:12] * huh too lazy to look up [05:04:37] ব্যবহারকারী:-User, ব্যস্ত​:-busy huh [05:07:51] Fixed. :S [05:08:08] Please be more careful when you copy over templates and then translate them [05:08:10] ;) [05:08:43] sorry for taking so long, I was also investigating something on tool labs [05:09:02] Pratyya: ^ Fixed, right? [05:09:18] https://bn.wikipedia.org/w/index.php?title=টেমপ্লেট%3AStatustop&diff=1516151&oldid=1515260 :P [05:09:35] thank you. yeah it's fixed. but I need to change something there? Cause now I'm busy, but it's adding online :P [05:09:46] hmmm [05:09:51] what was causing the problem? [05:10:04] {{SUBJECTSPACE}} gives localized names [05:10:15] you needed to use the Benagli word for User and User talk [05:10:43] {{NAMESPACENUMBER}} would be more portable between wikis [05:11:14] but meh I don't really care [05:11:47] Pratyya: are you sure it says you are online not busy? [05:12:06] It says "ব্যস্ত​" [05:13:23] Pratyya: are you there? [05:14:32] Pratyya: if the color if the only thing wrong, change the part with "ব্যস্ত​" in https://bn.wikipedia.org/w/index.php?title=টেমপ্লেট:Statustop&action=edit to be Ledorange instead of Ledgreen [05:14:39] bye [05:20:29] sorry net problem huh. anyway what were you asking? [12:07:04] Krinkle: ping [12:20:45] !log wikistats - editthis (error 991) - delete 'bridge' (readapi denied), delete 'kanji' (No db present), delete 'kickipedia' (no reply), delete 'lost20' (no reply) [12:20:46] Logged the message, Master [12:36:29] !log wikistats - metapedias (error 302) - sk.metapedia, et.metapedia, fix stats URLs (yea, use /m/ instead of /w/ now :P) [12:36:30] Logged the message, Master [13:00:08] !log wikistats - metapedias - fix broken table header, add language columns [13:00:10] Logged the message, Master [13:18:06] mutante: Are you back in europe? [13:18:21] Reedy: correct [13:18:31] for the month of Jan. ,until fosdem [13:18:43] then i'll switch back and confuse once again [13:18:56] I was confused why you were apparently working at like 5am :D [13:19:11] Pratyya, you rang? [13:19:14] nah 2 pm :) [13:19:21] and I was on vacation [13:19:31] first real work day again [13:19:41] yeah. but that problem is fixed and was fixed by huh Cyberpower678 [13:19:54] Pratyya, ok [13:19:54] I won't see you in SF then! :( [13:19:54] Hello. [13:19:57] YuviPanda: Just went through the instance list and stumbled upon zombying tools-mc. Turns out that the memcached there isn't used, but Redis is used by Suchabot on tools-exec-04. Could you move that to tools-redis and then MDK tools-mc for good, please? [13:20:02] You'll be going back when I go back tho the UK [13:20:22] hah, yea, i am missing mw summit , right [13:20:23] Pratyya / Cyberpower678 hi [13:20:38] huh, good night [13:20:40] but this way i could go to congress and fosdem and combine leisure and work [13:20:46] err, good morning [13:20:52] hi you were telling me something huh, My net connection got disconnected so I was to disconnected from IRC. [13:20:53] fosdem will also be great..wiki room [13:21:06] when I came back I saw you are gone.. [13:21:30] Pratyya: you said the template still was broken [13:21:37] Pratyya: if the color if the only thing wrong, change the part with "ব্যস্ত​" in https://bn.wikipedia.org/w/index.php?title=টেমপ্লেট:Statustop&action=edit to be Ledorange instead of Ledgreen [13:21:51] I fixed that :-) =D [13:23:55] good :) [13:28:01] anyway huh if I use {{BASEPAGENAME}} {{PAGENAME}} etc in Template:Statustop the result will be "Statustop" But is there anyway to bring "Template:Statustop"? [13:28:29] Pratyya: would you like to include the subpage as well? [13:28:49] So in Template:X/Y the result is Template:X/Y [13:29:32] I want "Template:Statustop" huh [13:29:47] I mean not without the "Template:" [13:30:16] {{FULLPAGENAME}} but that includes subpages too [13:31:50] Pratyya: why? [13:32:24] Just askin. [13:33:05] Anyway you've created https://meta.wikimedia.org/wiki/User:PiRSquared17/Global_Userpage huh . Any plan to create a Global preference changer? [13:33:27] I'd rather have an extension do these things [13:33:43] the script is just temporary until a better solution is found [13:34:30] I can work on it if you want [13:35:56] Obviously I want it. huh (ping me) [13:36:14] Pratyya: ping [13:36:53] I told ya to ping me if you reply :P =D [13:38:16] Pratyya: by the way, why do you keep editing your userpage globally? ;) [13:38:38] I last edited my global UP 1 months ego ;-) [13:38:50] You do know it edits 700 wikis, right? [13:39:00] yeah. 720. [13:42:06] by the way I last used that in 17 Nov 2013 huh [13:42:39] Okay [13:43:24] Pratyya: the script is really only useful for people who are active crosswiki, like you are (or anyone in #cvn-sw ) [13:43:45] anyway point is will you be able to make a prefchanger script or extension? [13:43:54] (y) [13:44:11] ow we don't have any like here... :-( [13:44:13] Maybe. Give me some specs and I will [13:44:22] Pratyya: thumbs up ;) [13:44:49] yeah thumbs up, but it isn'there [13:44:58] isn't here* [13:45:21] heh how much time it takes to create this kind of global script huh ? [13:45:47] I'd have to read the API docs for some thing [13:46:19] ow. [13:47:14] CORS is a pain [13:47:29] (cross-origin / mutli-domain stuff ...) [13:48:32] https://www.mediawiki.org/wiki/API:Options [13:48:42] you mean like en.wikipedia, bn.wikipedia? [13:49:52] Yes, or like wikipedia vs wikibooks vs wiktionary vs *.wikimedia [13:49:59] and all the subdomains.... [13:51:03] this can be used to get a list of prefs ... https://en.wikipedia.org/w/api.php?action=query&meta=userinfo&uiprop=options [13:51:15] but do they exist on all wikis? [13:54:08] scfc_de: kill it, I moved SuchABot over a long time ago, I think [13:54:18] scfc_de: or do you want me to kill it? [13:56:47] YuviPanda: ATM there's a suchabot process on tools-exec-04 sending data on port 56223 to Redis on tools-mc, so apparently the move wasn't complete :-). [13:56:58] scfc_de: hmm, ok [13:57:02] let me look at it in an hour or s [13:57:04] so [13:57:29] YuviPanda: No problem, tools-mc has been overdue for months now :-). [13:57:52] scfc_de: yeah, I thought we had already killed it :| [13:59:15] btw huh in wiktionary can I use Oxford? I mean can I copy and paste meanings from Oxford? [13:59:38] not online dictionary but the book. [13:59:59] if it's in the public domain, yes [14:00:05] if not, probably no [14:00:28] YuviPanda: You did, but then it came back. [14:00:35] whaat [14:00:36] how [14:01:10] Pratyya: only if they're in the public domain [14:01:14] I have a fat Oxford dictionary book. Can I use that? It's not online thing huh ? [14:01:20] YuviPanda: I spent a couple of minutes reading through old labs-l, and it had been decided to kill tools-mc several times :-). [14:01:22] Probably not. [14:01:35] scfc_de: I vaguely remember hitting delete, I think [14:01:37] apparently not [14:01:46] YuviPanda: petan resurrected it. [14:01:47] why? [14:01:49] wat [14:01:53] it's the actual meaning, huh [14:02:13] scfc_de: delete it. I don't have my phone so can't login [14:02:17] Also I'm not a master of language and it's meaning. ;-) [14:02:20] scfc_de: suchabot is pretty much 'down' anyway [14:02:24] Pratyya: you can ask in #wiktionary ? [14:02:36] scfc_de: and also gerrit-to-redis is on tools-redis anyway, so it is dead [14:02:43] I'll kill the suchabot instance in a bit [14:03:01] scfc_de: any idea *why* it was recreated? [14:03:37] YuviPanda: http://permalink.gmane.org/gmane.org.wikimedia.labs/1505 ff. [14:03:51] YuviPanda, ping [14:03:54] scfc_de: I thought he was 'asking' [14:03:55] meh [14:04:48] YuviPanda, pm? [14:04:58] YuviPanda: There's no hurry. I'll just look the next days if you amended suchabot, and then delete the instance. [14:06:28] !log tools deleted instance tools-mc, didn't know it had come back from the dead [14:06:31] Logged the message, Master [14:06:41] scfc_de: don't think I'll have time until the weekend :( [14:07:08] scfc_de: and besides, if gerrit-to-redis is feeding into tools-redis, then suchabot can't be working off tools-mc anyway [14:08:14] @seen petan [14:08:14] Cyberpower678: Last time I saw petan they were quitting the network with reason: Remote host closed the connection N/A at 1/3/2014 7:40:01 AM (3d6h28m13s ago) [14:08:25] When is going to come back online. [14:08:57] scfc_de: found my one time access codes, back on wikitech now :) [14:08:57] YuviPanda: Even next month would be okay, don't worry :-). [14:09:01] scfc_de: :) [14:09:26] YuviPanda, ping [14:10:12] YuviPanda, can I ask a question? [14:11:21] I guess not. [14:12:04] * Cyberpower678 leaves [14:12:33] scfc_de: I already deleted and logged it, btw [14:14:20] YuviPanda: Thanks! [14:14:26] yw [14:14:32] scfc_de: thanks for bringing it to my notice :) [14:14:44] scfc_de: just found that the signpost tool also was pointing to tools-mc. changed it [14:15:31] YuviPanda: BTW, yesterday there was some discussion regarding recent changes feeds. I remember some idea using Redis for this. Did I dream this, or do you know if there has been any work on that? [14:15:50] scfc_de: it's landed in core months back. [14:15:53] scfc_de: not turned on yet [14:16:35] scfc_de: we probably can't expose it to the internet from a redis machine in core, but we can easily expose it to tools [14:18:16] YuviPanda: But Redis has replication so it could be fed to other hosts? (wmf-redis-master => tools-redis-master => clients?) [14:18:34] scfc_de: indeed, that's why we can easily do that [14:19:02] (I thought that we would have to parse IRC to Redis, but if it's already in MediaWiki, excellent.) [14:19:02] scfc_de: setup a redis master in cluster, setup a redis slave in tools, have the slave read from the master, and setup ip rules to not allow anyone else to read from the master [14:19:45] YuviPanda: Thanks. I'll make a mental note (or a bug report) to remember that. [14:19:53] scfc_de: a bug report! [14:21:56] YuviPanda: Do you have a pointer for the recent changes code? http://www.mediawiki.org/wiki/Redis lists as use cases only session storage, job queue and "features". [14:22:10] YuviPanda, hi, may I know why you just created a new project on Tool Labs to copy an already existing tool ? (without even respecting patternity btw) [14:22:22] quentinv57: hmm? [14:22:35] scfc_de: let me find a commit [14:22:52] quentinv57: I haven't created a new tool in months [14:23:11] YuviPanda, You are listed as the maintainer of the sulinfo project. [14:23:13] YuviPanda, I'm not speaking about creating but about copying [14:23:20] can you be more specific? [14:23:31] what did I copy? [14:23:41] YuviPanda, the sulinfo tool. [14:23:47] YuviPanda, http://tools.wmflabs.org/sulinfo [14:23:52] err [14:23:54] that's not mine [14:23:57] ah, yes, the old slow one [14:24:17] YuviPanda, you're listed in the maintainers, that's why we contact you ;) [14:24:25] This URI is managed by the sulinfo tool, maintained by Petrb and Yuvipanda. [14:24:30] quentinv57: sorry, I don't know who added me there. [14:24:45] in that case you probably want petan, who is peterb, who isn't here [14:24:48] YuviPanda: can you redirect it to quentinv57/sulinfo at least? [14:24:51] I don't even know what it does :) [14:24:59] Since petan isn't here we're contacting you. [14:25:02] it's like CentralAuth but slower :P [14:25:10] and it shows info for unattached accounts [14:25:12] no, I'm just going to remove my name from the maintainers list, since I have no idea what tool that is [14:25:12] YuviPanda, am I on your ignore list btw? [14:25:28] YuviPanda: Why do you ignore Cyberpower678 btw? [14:25:46] hmm, I might have him on my /ignore list [14:26:19] quentinv57: I removed myself. I don't know if it is being maintained - if not, you should poke petan and take over that tool [14:26:41] huh: I have taken him off it now. [14:26:43] YuviPanda, can you see me now? [14:26:50] yes, Cyberpower678 [14:26:57] BTW, sulinfo was created prior to Jun 15 2013, so I'm not sure who stepped on whose toes here :-). [14:27:02] YuviPanda, why was I on your ignore list? [14:27:11] Just curious. [14:27:18] "YuviPanda, You are listed as the maintainer of the sulinfo project." [14:27:20] Cyberpower678: *shrug*. I was annoyed about something. [14:27:24] "YuviPanda, the sulinfo tool." [14:27:31] "Since petan isn't here we're contacting you." [14:27:34] "YuviPanda, am I on your ignore list btw?" [14:27:41] huh: https://wikitech.wikimedia.org/wiki/Special:NovaServiceGroup [14:27:47] huh: oh, thanks [14:27:55] huh: i have removed myself. [14:28:02] It helps to know what annoyed you. [14:28:32] huh: quentinv57 I think perhaps you can email labs-l, and we can figure out a way to see who gets ownership of that? [14:28:41] Cyberpower678: let me PM? [14:29:16] YuviPanda, funny you mention it, I've been sending you PMs but you haven't been responding. [14:29:35] YuviPanda, okay, thank you for your help :-) [14:29:49] have a nice day [14:29:55] quentinv57: yw. you too [14:30:43] !searchlogs [14:30:47] YuviPanda: I actually have little to do with sulinfo, but I am working on other quentinv57's tools [14:30:50] !searchlog [14:30:51] http://bots.wmflabs.org/~wm-bot/searchlog [14:30:54] huh: ah, ok :) [14:31:27] !logsearch [14:31:28] http://bots.wmflabs.org/~wm-bot/searchlog [14:32:24] YuviPanda: Found includes/rcfeed/RedisPubSubFeedEngine.php. [14:32:38] scfc_de: yup, that's the one [14:54:52] scfc_de: thanks for the bug report [14:54:59] scfc_de: you should also cc ori :D [14:55:03] scfc_de: oh nevermind [15:08:47] hi Coren, I'd like to run a script on the deployment-prep host that would send an email notice to the WMF QA mail list. do you know if there is an MTA I could use from the deployment hosts? [15:18:04] chrismcmahon: I don't think there is one configured by default. [15:18:36] Coren: could one be created do you think? [15:20:08] I should say that the best thing you could do is to setup a simple relaying exim for the task. Alternately, you can SMTP directly to mchenry if your only possible target is a mailing list [15:20:31] (It mostly depend on what you are sending the mail /with/) [15:21:34] Coren: I was thinking smtp. What is mchenry? I tried 'ping mchenry.wmflabs.org' and got no response. [15:21:47] mchenry.wikimedia.org. [15:21:50] ah [15:21:54] good morning Coren [15:22:25] It's the MX for incoming @wikimedia.org email [15:22:54] Coren: sweet, mchenry is reachable from the deployment hosts. (I wasn't sure it would be.) How would I get credentials to use it? [15:22:55] Isn't it outgoing as well? [15:23:26] scfc_de: It is, but that's not relevant to that particular question. :-) [15:23:45] chrismcmahon: Since your destination is local, you don't need credentials. [15:24:21] Coren: wow, this might actually be easy. :-) Thanks! [15:24:52] chrismcmahon: It gets more complicated if you want to email outside @*.wikimedia.org, but that shouldn't be a concern for you. :-) [15:25:28] (03PS2) 10Tim Landscheidt: Fix build and run-time dependencies [labs/toollabs] - 10https://gerrit.wikimedia.org/r/104951 [15:25:57] hi halfak [15:25:58] * YuviPanda waves at Coren [15:26:12] Heyo YuviPanda [15:26:15] Coren: talking about mails. Is there anything in place that would allow to send an email to @wmflabs.org and possibly redirect to the maintainer? [15:26:16] Hi huh. [15:26:27] Sorry to bail on Saturday. [15:26:35] Coren: nope, only ever to qa@lists.wikimedia.org (at least for now) [15:26:35] it's okay [15:26:48] My wife and friends kidnapped me on Friday for my birthday. [15:26:50] hedonil: That's planned for the short term, but we'll not do it before the migration to eqiad. [15:26:53] I go back late last night. [15:26:57] happy birthday halfak! [15:27:02] Coren: 'k [15:27:03] thanks :) [15:27:10] brainwane: ^ [15:27:30] Happy birthday! [15:27:42] brainwane: Heya, you well-schooled hacker. :-) [15:27:50] Hi there! :) Happy New Year. [15:27:51] hedonil: https://bugzilla.wikimedia.org/show_bug.cgi?id=58796 [15:28:15] scfc_de: thx [15:28:34] Coren: when is the move to eqiad again? [15:28:46] (sorry, been away from most things for a while) [15:28:48] huh: I've got a bit of stuff to work on this morning, but if you don't mind me multitasking, I'd like to keep trying to help with your script. [15:28:59] I have an idea for redis/memcached that might work for you. [15:29:15] YuviPanda: Mike is hard at work w/ Ryan helping; I'm *hoping* they'll be ready with the Labs setup in eqiad to start moving stuff early Feb. Tools is the first project to move. [15:29:15] Okay. I'd like to hear how that works. [15:29:26] Coren: ah, sweet [15:29:29] brainwane: Hi, did you fix your issues with escaping & encoding ? I would like to test your tool again ;) makes fun [15:29:44] I'll hop to PM to cut the noise in this channel. [15:30:08] halfak: pfft, you should do it in channel anyway, so others can jump in if that'll be useful [15:30:12] hi hedonil! I'm sorry that I have not fixed them yet! I mean to, and when I do, I will inform the wikimedia labs mailing list :) [15:30:22] (03PS3) 10Tim Landscheidt: Fix build and run-time dependencies [labs/toollabs] - 10https://gerrit.wikimedia.org/r/104951 [15:35:00] brainwane: My suggestion: If you would use Python's »oursql« , you could take advantage of parameterized querys which does all the escaping stuff [15:35:05] brainwane: http://tools.wmflabs.org/paste/view/14a27b0f [15:36:27] interesting! will check it out. Thanks hedonil [15:36:32] (03CR) 10Tim Landscheidt: "Memo to self: ./configure && Co. gets executed as root, make check runs as pbuilder." [labs/toollabs] - 10https://gerrit.wikimedia.org/r/104951 (owner: 10Tim Landscheidt) [15:36:39] brainwane: yw [15:39:47] YuviPanda, missed your message. We can do that. [15:41:23] I just got done recommending that huh set consider using redis to persist an API call he makes regularly. [15:41:45] this one is to gather the list of namespaces for all/a large set of languages. [15:42:14] Does anyone know if there is a managed install of redis available for tools? [15:42:21] (Tool labs) [15:42:32] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Redis ? [15:45:50] Cyberpower678: tell halfak what the speed improvement was using UNIONs [15:46:03] 1 minute to 5 seconds or so? [15:46:43] Cool. What do you mean "using UNIONS" [15:46:55] Are you querying all table available on the server at once? [15:47:07] I think you do one slice at a time, but basically yes [15:47:18] slice == server [15:48:41] Also == shard. I think pretty much every 'S' word that vaguely applies has been used at one time or another. :-) [15:52:29] sliver! [15:52:43] soupcon [15:53:07] * brainwane should have proper diacritic on the c [15:54:59] halfak, Using UNIONS speeds up performance greatly. [15:55:18] The hard part is sorting out the results since they are jumbled together. [15:55:49] halfak: I looked at the tutorial you gave me, but I'm still not sure how to store data in the tools-redis [15:55:50] The queries also need matching columns for UNION to work right. [15:56:01] Cyberpower678: they will have that [15:57:06] halfak, the best way to sort data is to place a new column in the results and place the dbname into it where the results were pulled from. [15:59:12] YuviPanda: did you test the erb magic in https://gerrit.wikimedia.org/r/#/c/105687/1 ? [16:10:01] cyberpower678: Thanks, I know how unions work. :) [16:10:38] It's not really about "UNIONS" so much as it is querying all of the dbs on the server at once. [16:10:49] But unions are a good way to throw that data together. [16:13:17] halfak: how else can you do it? [16:15:01] I suppose you could do it with a left join. That wouldn't make much sense. [16:15:15] andrewbogott: around? [16:15:30] we need someone to look at https://wikitech.wikimedia.org/wiki/New_Project_Request/Scrumbugz [16:15:37] I'm not sure how much overhead you'd have if you sent the query to a persistent connection. [16:15:42] aude -- ok -- remind me again in 15? [16:15:46] Is that what you were doing before, huh? [16:15:48] ok [16:16:38] Currently I have a separate query for each wiki [16:16:50] (grouped by slice/server/shard/sliver) [16:17:27] andrewbogott: heh, not yet :D [16:23:15] halfak: can you help me to set up/connect to redis? [16:24:11] andrewbogott: do you have an instance where that can be tested? [16:24:33] andrewbogott: or alternatively, can you do the testing? :D [16:24:35] Oh yeah. Sure. So the guide that I linked to should get you pretty far with connecting. I'll work on a function for getting/storing namespaces. [16:24:37] huh: can you send me the current version of your code? [16:25:46] http://tools.wmflabs.org/pirsquared/iw2.php.txt [16:26:33] YuviPanda: um… puppet-testing-1? But I can do it in a bit. [16:26:54] andrewbogott: thank you :) [16:28:08] oh god! So many tabs! [16:28:16] halfak: You added them, not me [16:28:22] Remember? [16:28:38] heh. I might have gotten a strange version of the file. There are tabs, but scope doesn't seem to line up. [16:29:09] hm [16:29:18] I'll just give you the untabbed one [16:29:37] http://tools.wmflabs.org/pirsquared/iw.php.txt [16:39:18] This version of the script skips the API call, right? [16:39:42] huh: ^ [16:40:10] I see "get_file_contents()" commented [16:40:13] i2 did, yes [16:40:13] *iw2 [16:40:16] I commented it out to see how much faster it was [16:42:32] Yeah. I remember suggesting that. [16:42:42] okay [16:42:46] I figured you might have re-enabled it by now, so I thought I'd ask. [16:43:11] halfak: I enabled it in iw.php, but disabled it in iw2.php [16:43:53] I actually haven't worked on the file since last week [16:44:03] I've been helping with another tools project instead [16:44:23] Gotcha. [16:44:31] Wait... so there are two different versions of the file? [16:44:37] OK, now… aude, you just wanted me to create that project? [16:44:43] They're almost exactly the same [16:44:51] except two lines are commented out or so [16:45:22] halfak: http://tools.wmflabs.org/pirsquared/iw.php.txt has the api [16:45:33] but we're going to remove it anyway... [16:45:37] Sure. [16:47:50] andrewbogott: yes please [16:48:50] aude: OK, should be all set. Please use the dynamic http proxy for web access, if/when you need it. [16:48:57] sure [16:49:09] we are switching all our stuff to use that, much as possible [16:49:24] cool [16:50:19] andrewbogott: I should spend some time at some point writing up some hit stats for things that use the proxy [16:50:25] should be easy enough to do and fun [16:50:32] * YuviPanda is slowly easing himself back into non apps work [16:50:35] the break was fun [16:57:11] YuviPanda: or just make the logs available for other fabulous tools ;) [16:57:23] YuviPanda: Coren: https://bugzilla.wikimedia.org/show_bug.cgi?id=59222 [16:57:42] hedonil: will need to scrape off personally identifiable information [16:58:03] hedonil: this is for dynamnicproxy, which toollabs doesn't use (yet) [16:58:04] YuviPanda: for sure! [16:58:26] hedonil: shouldn't be *too* hard to do, of course, since those are just standard nginx logs and I bet tools already exist for that [16:58:30] YuviPanda: the famous Yuvi-proxy? [16:58:39] heh, that is one of its names, yes [16:58:48] YuviPanda: hehe [16:59:33] hedonil: tools will move to that during/right-before eqiad migration though [16:59:59] YuviPanda: fine ;) [17:01:15] * hedonil rejoices. operations server gadolinium is back to the living \o/ \o/ \o/ [17:01:31] ...a and delivering pagecount dumps again [17:01:34] YuviPanda: Specifically, the eqiad "half" of tools will use your proxy from the start; the pmtpa half will just deprecate the apache proxy gradually. [17:01:51] Coren: true, so eventually it'll just be eqiad I guess [17:01:57] * Coren nods. [17:02:18] huh: http://pastebin.com/9uDqQarj [17:02:19] Coren: did you get around to writing port-grabber to talk to the new port-granter? I would probably be able to help out next week [17:02:34] huh: oops. forgot something. one sec. [17:02:35] * huh looks [17:02:36] but not sure. Trying to fix my health, etc so have less hacking time. [17:02:37] ok [17:02:52] YuviPanda: I got distracted (a lot) with the new DC search. [17:02:58] Coren: ah, ofcourse :) [17:03:09] I should check ops list sometime to see how that is coming [17:03:56] huh: do you already have the connection bits for redis figured out or should I add that too? [17:04:07] Hi all, happy new year! [17:04:14] couldn't figure it out :S [17:04:22] OK. I'll add it. [17:04:23] And a prosperous 2014 to you, Silke [17:04:29] Coren: Is Sumana back? [17:04:48] note that if you are using redis in tools, you should use a prefix. https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Security [17:04:51] huh: halfak ^ [17:04:58] Is the host supposed to be tools-redis? [17:05:01] otherwise you might get weird bugs where other tools trample on your code [17:05:11] Silke_WMDE: indeed! she's brainwane for now [17:05:39] Silke_WMDE: she's not on this channel at the moment, however [17:05:41] ok thx YuviPanda! [17:05:42] Silke_WMDE: She's back, and catching up. [17:06:03] Very good. [17:06:24] huh: http://pastebin.com/Dt69dBNX [17:06:46] No guarantees that it will work. I never tested it, but I put the skeleton together for you. [17:10:19] * huh tries,thanks halfak [17:11:07] No prob. Note how I wrapped some complexity in that get_namespaces function. You might like doing for some other bits if your script. :) [17:12:40] Yeah, I wasn't thinking ahead when I wrote the script [17:17:45] [06-Jan-2014 17:17:14 UTC] PHP Parse error: syntax error, unexpected '[', expecting ')' in /data/project/pirsquared/public_html/iw.php on line 111 halfak [17:18:12] Yeah. You're on your own for syntax errors. I figure they'll be easy to deal with. [17:18:33] ok [17:32:29] halfak: Is Predis even installed? [17:33:04] According to the wiki, it should be. [17:33:11] https://github.com/nicolasff/phpredis is installed [17:33:16] :( [17:33:34] not https://github.com/nrk/predis [17:33:42] * huh rewrites stuff [17:34:14] You shouldn't have to change much for phpredis. [17:34:49] Hmmm http://redis.io/clients calls predis "Mature and supported". It seems like we should have that library installed. [17:35:07] Both get a star [17:35:33] star = recommended by redis.io [17:36:11] halfak: IIRC, we went with phpredis since that's what mediawiki uses [17:36:28] Gotcha. [17:38:22] YuviPanda: why not both? [17:38:30] YuviPanda: Coren: AFAIK apergos is the one who manages this log stuff in operations. maybe one could borrow some lines of his script, so that logfiles are summarized per hour and share the same structure. Seems to be powerful as it handles all syslogs and accesslogs from the varnishes. [17:38:54] hedonil: would be a bit different at least, since this is nginx logs and varnish logging is rather very different [17:39:14] huh: nobody had asked for predis yet :P [17:40:21] huh: predis isn't in the repos, I think. that might make it harder [17:40:24] YuviPanda: a job for sed, the emperor of tools ;) [17:40:45] !log wikimania-support Updated scholarships-alpha to 713f90b [17:40:46] hedonil: heh, true [17:40:47] Logged the message, Master [17:41:14] huh: I think that was perhaps also why php-redis was chosen over predis for mediawiki - since predis isn't in debian repos [17:41:26] huh & YuviPanda: makes sense. I only picked predis because I saw it at the top of the rec list a moment ago. [17:41:38] :) [17:41:56] phpredis could very well be a better connector. [17:42:31] true [17:42:37] but ultimately it doesn't matter too much, I guess [17:45:52] Coren: Could you +2 some package additions if you got some time, please? https://gerrit.wikimedia.org/r/#/c/105192/, https://gerrit.wikimedia.org/r/#/c/105453/, https://gerrit.wikimedia.org/r/#/c/102630/, https://gerrit.wikimedia.org/r/#/c/84288/ [17:47:46] works, halfak :) [17:48:18] thank you [17:48:19] Woot! Now to build a second script to populate those redis keys periodically. [17:48:30] No problem. happy I was able to help :) [17:52:29] Coren: just out of curiosity - why are looking for DC's? I thought eqiad is it. Or are you plotting your own business corentools.org? ;) [17:53:00] hedonil: eqiad replaces tampa, but you always want to have 2 [17:53:31] mutante: at least, true [17:53:34] fail-over. and it's better for West Coast and Asian users [17:53:40] faster [17:53:53] if you have one on each coast [17:54:13] peering matters a lot as well [17:54:22] more than physical distance [17:54:46] mutante: yeah! Bigger Better Faster and interconnected with 1TB/s [17:55:05] yep, well i think 10G:) [17:55:11] mutante: ;) [17:57:17] halfak: and you were right, the commands in phpreddis are very close to those in predis [17:57:23] I only had to change a few [17:57:39] *phpredis [17:57:53] I'm glad for that. Where you able to still use a pipeline for the store-and-set-expiration commands? [18:04:02] halfak: yes [18:04:19] they have $redis->multi()-> ... ->exec() [18:04:24] pretty similar [18:04:46] Awesome. I like that pipeline bit. You can do some interesting stuff with redis if you can manage a transaction. [18:05:09] At least atomic sequences of operations. [18:05:14] halfak: you can do crazy amounts of things with Lua + Redis, btw. You can script redis completely atomically with embedded Lua [18:05:53] YuviPanda: Interesting. I'm not sure why I'd want to. I guess if I wanted to perfom a complex operation on the redis server and then get a small amount of data back. [18:05:54] Coren: So, bug https://bugzilla.wikimedia.org/show_bug.cgi?id=58997 -- have a minute to talk about that? [18:06:30] andrewbogott: Sure, I'm just done with my site visit notes. [18:06:36] halfak: indeed. it's also fully atomic, so that opens up interesting possibilities [18:06:51] I have a bunch of code written to support dueling schema for that. [18:06:55] halfak: grrrit-wm uses that to form a proper almost-persistent queueing mechanism, for example [18:07:03] but I still don't know what to do about the service user. We don't really want two at once I don't think... [18:07:31] YuviPanda: That sounds builtin redis functionality. [18:07:58] ^ a persistent queue [18:08:05] halfak: built in redis functionality is pubsub, which is not persistant [18:08:05] Does it really matter? In practice, servers will have their ldap.conf point at one /or/ the other; and even in case of charlie foxtrot passwd db functions with duplicates without issue. [18:08:23] I guess I haven't looked into the details of pubsub yet. [18:08:33] I'm not sure what you mean when you say it's not persistent. [18:08:44] There is a risk of the data diverging, in theory, but it's only a transition measure so it shouldn't have time to diverge. [18:09:44] halfak: pubsub? [18:09:46] halfak: if the client is not connected, it misses the messages [18:09:47] We don't have two different ldap dbs though. So the two schema need to coexist within the same DB. [18:10:15] YuviPanda: I see. I figured you guys would be using a redis type you could pop a job from. [18:10:30] * halfak reviews types.  [18:10:42] halfak: we do, but lua is needed to make that happen atomically [18:10:48] I suppose a list or a sorted set. [18:10:55] a list, yeah [18:11:00] andrewbogott: Sure, but they'll be under different OUs [18:11:17] halfak: so a single key contains list of target queues to publish an event to, and then we run this lua script that publishes to them all atomically [18:11:28] andrewbogott: The legacy has the per-project OU thing; the new ones will (should) all be under the same OU. [18:11:42] Coren: So, you're suggesting that we go ahead and define both service users at the same time? [18:11:58] andrewbogott: IMO, that's the easiest way to migrate from one to the other. [18:12:13] halfak: it's only an issue when you have lots of queues to publish to, however [18:12:18] I feel like defining two different posix users with the same UID but different names will cause… unusual things to happen [18:12:37] YuviPanda: Makes sense. :) [18:12:46] halfak: :) [18:12:47] andrewbogott: Nah; even if we mistakenly point a server at both OUs POSIX copes with that. [18:13:08] andrewbogott: But in practice, the ldap.conf is puppetized so that can't even happen unless we try hard at it. :-) [18:13:41] OK, maybe I don't properly understand how the ldap user -> posix user mapping happens [18:14:38] Looking at a stock ldap.conf I don't see how it's happening at all [18:15:04] andrewbogott: Look at nss_base_passwd [18:15:09] (and nss_base_group) [18:15:25] That points at the OU which will be searched for users and groups [18:15:52] ok… but /that/ isn't changing with the new schema. [18:16:01] Oh! I get what you mean now! [18:16:11] We'll have both local-foo and bar.foo users at the same time [18:16:14] with the same uid [18:16:27] both under ou=people,dc=wikimedia,dc=org [18:16:33] … ok :) [18:17:39] You're looking at the wrong ldap conf. :-) [18:18:03] andrewbogott: Look at modules/ldap/templates/nss_ldap.erb [18:18:59] But those users exist on the tools instances as well don't they? [18:19:22] I mean, I know they do. So the dueling user thing will happen on instances even if it's moot on the storage servers. [18:21:46] Oh, ok, wait, I'm reading more... [18:22:26] What will a service user's entry look like in the new schema? [18:23:54] Not sure what you mean by "look like"? [18:24:24] I mean, except for the username proper, they should look like they do now except not in the per-project OU [18:24:42] the 'not in the per-project' is the thing I'm wondering about. [18:24:49] Will they just look the same as any other user? [18:25:19] andrewbogott: We can put them in a different OU that "normal" users (in fact, we probably want to); just not in different ones depending on project like we do. For instance: [18:26:55] * Coren goes to fetch an example [18:27:09] ^^ [18:28:23] cn=local-abot,ou=groups,cn=tools,ou=projects,dc=wikimedia,dc=org [18:28:27] andrewbogott/Coren: any idea what's the deal with virt1000's ldap? [18:28:32] is the user 'local-abot' on tools [18:29:05] paravoid: virt100* being worked on by Mike/Ryan, so their status will fluctuate [18:29:20] paravoid: in theory though it's in sync with virt0. [18:29:46] Coren: That's the 'before' or 'after'? [18:29:54] it's down [18:29:59] The 'new' user would be something like 'cn=tools.abot,ou=groups,cn=servicegroups,ou=projects... [18:30:19] The first was the current value. [18:30:35] Ah, ok. [18:31:16] So, instances, will look at 'ou=groups,cn=servicegroups,ou=projects...' rather than 'ou=groups,cn=$MYPROJECT,ou=projects...' [18:32:20] Coren, are instances looking in ou=groups,cn=$MYPROJECT,ou=projects... now? [18:32:35] Yep. [18:33:49] Because I'm looking at ldap.conf on tools-login and I don't see... [18:33:53] Well, they should. I can't find where that is now that much of that has been refactored. :-) [18:35:08] (And yet, clearly it works now) :-) [18:35:09] Ah; nslcd.conf [18:35:14] ok [18:35:27] I'll have some patches for you to read in a few hours… they'll need a close look. [18:35:43] kk. But ops meeting in ~25m [19:11:05] (03PS4) 10Tim Landscheidt: Fix build and run-time dependencies [labs/toollabs] - 10https://gerrit.wikimedia.org/r/104951 [21:25:03] Coren: For starters… https://gerrit.wikimedia.org/r/#/c/105827/ [21:25:21] * Coren digs in. [21:25:56] It'll be a bit longer before I have anything interesting -- that's just preliminary cleanup [21:28:10] Coren: Sorry if I just caused an edit collision in the engineering report :( [21:28:36] My edit went through; it's all on you. :-) [21:28:47] Are you done for the moment? Otherwise I can stand back [21:34:08] Coren, another easy one: https://gerrit.wikimedia.org/r/#/c/104485/ [21:34:25] andrewbogott: I'm done for now. [21:35:41] approach any regex I write with great suspicion [21:38:08] Heh. I doubt you'd have had trouble with /that/ one. :-) [21:46:07] andrewbogott: For shellaccountname, we only allow lowercase letters. Conscious decision to allow uppercase for service groups? [21:46:30] conscious, yes, although possible incorrect. [22:54:55] * Coren considers. [22:55:30] andrewbogott: Uppercase does widen the space, but maybe it could lead to confusion? I'm agnostic on that point. [22:56:08] I guess it's inconsistent… I'll remove that i [23:00:14] does anyone have figures on the relative popularities of Cite web vs cite book vs cite news vs cite journal? [23:01:15] notconfusing: could you just get a transclusion count? [23:02:15] huh, Yes, I can do that. I also have a particular subset, and I wanted to know if the distribution of citations is uniform across all articles? [23:03:27] Coren: Oh, this was moot, that field was already validated elsewhere. [23:03:35] And, correctly. So I'm going to just revert that patch. [23:03:43] Ah! :-)