[07:47:32] Would it be an issue for the tool labs infrastructure if I run a node.js process continuously? It will idle most of the time but it should watch recent changes and act sometimes .... [08:43:51] It's the lack of a trailing slash that's causing a particular problem at the moment [08:43:57] e.g. http://tools.wmflabs.org/jarry-common --> http://tools.wmflabs.org/jarry-common/ [08:44:34] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Roadmap en was modified, changed by 101.219.135.122 link https://www.mediawiki.org/w/index.php?diff=962841 edit summary: /* Overview of available features in Tool Labs */ [08:45:19] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Roadmap en was modified, changed by Glaisher link https://www.mediawiki.org/w/index.php?diff=962842 edit summary: Reverted edits by [[Special:Contributions/101.219.135.122|101.219.135.122]] ([[User talk:101.219.135.122|talk]]) to last revision by [[User:Silke WMDE|Silke WMDE]] [08:48:41] Is there a tool like traceroute installed on tool labs ? [08:49:36] I want to know whether my bot should encrypt login data when logging in to a wiki. [08:52:21] Coren, YuviPanda, I've reopened https://bugzilla.wikimedia.org/59926 . That's the problem Josve05a was having with /checkwiki [09:53:08] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #1: ABORTED in 54 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/1/ [09:56:53] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #2: FAILURE in 1 min 4 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/2/ [10:00:21] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #3: STILL FAILING in 11 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/3/ [10:05:42] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #4: ABORTED in 2 min 12 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/4/ [10:10:09] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #5: STILL FAILING in 3 min 28 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/5/ [10:22:07] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #6: STILL FAILING in 2 min 51 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/6/ [10:31:22] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #7: ABORTED in 1 min 39 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/7/ [10:36:00] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #8: STILL FAILING in 4 min 26 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/8/ [10:37:52] go away, this isn't -operations [10:38:26] mw would be -dev :p [10:38:32] its everywhere [10:38:36] this isn't -dev either [10:38:58] where does it come from? config in gerrit? [10:58:16] hello [10:58:25] i have a labs account now [10:59:13] how do i get into the project tools? [10:59:31] i have ssh access to the bastion host already [11:00:34] tools is down atm? [11:02:17] Steinsplitter: https://tools.wmflabs.org/xtools/pcount/index.php? causes 404 [11:03:27] https://tools.wmflabs.org/add-information/ works [11:03:31] hm... [11:12:36] http://tools.wmflabs.org/steinsplitter internal error [11:12:40] and a lot of other tools [11:14:19] error log emoty [11:14:21] working now [11:35:34] nosy: Did you fiddle out how to connect to the tools project? [11:36:19] rillke: i use magnus howto to get a tool labs project [11:36:29] @ http://tools.wmflabs.org/steinsplitter internal error -- this is https://bugzilla.wikimedia.org/59926 [11:36:43] currently i am stuck at the point where i need to be "in the project tools" says the wiki [11:37:01] ssh seems to be working already though [11:37:20] nosy: You have to ask someone to add you to this project, I think. You don't need bastion ... [11:37:36] ok [11:37:47] i want to make my own project [11:38:02] https://wikitech.wikimedia.org/w/index.php?title=Special:NovaServiceGroup&action=addservicegroup&projectname=tools [11:38:06] tells me [11:38:12] Your account is not in the project tools. [11:38:30] Coren ^^ [11:38:50] might be a bit early [11:39:56] nosy: https://wikitech.wikimedia.org/wiki/Special:FormEdit/Tools_Access_Request [11:40:03] did you already fill this in ? [11:40:09] yes [11:45:55] nosy: everything is fine. your request https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Nosy is ok [11:46:20] hedonil: thx :) then i ll wait [11:46:26] I see https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Nosy -- petan is also project admin [11:46:39] hello [11:46:40] nosy: your request will be handled by Tim (scfc_de) or Coren [11:46:54] or me :P [11:47:02] petan: or PETAN [11:47:05] ;) [11:47:08] great [11:47:10] :) [11:48:35] nosy: do you know a unix tool alternative for traceroute ? [11:48:42] mtr? [11:49:18] you could even take perl and try to implement traceroute yourself :D [11:50:03] nosy: Thanks... [11:51:12] worked perfectly ... wanted to check how labs is connects to WMF wikis [11:52:22] nosy: second thing (after shell access is granted) is to login with PuTTy keys and wondering why it possibly fails ... [11:52:30] nosy beware that any user can access your tool's data by default (except replica.my.cnf) [11:52:48] oh fine i guess people want that [11:52:48] nosy: third thing is to submit a php job and wondering why it fails with error [11:52:51] libgcc_s.so.1 must be installed for pthread_cancel to work [11:52:56] lol [11:53:16] nosy: no worries, we are here! :P [11:53:19] the db is something people use on ts to map wikis to languages [11:53:24] thats why [11:53:42] hedonil: thanks for the hints [11:54:46] oh i am in! [11:54:58] woo. \o/ [11:55:45] thx [12:06:38] !hey bot [12:07:21] !What's the Answer to the Ultimate Question of Life, the Universe, and Everything ? [12:07:41] !you are no smart bot [12:07:41] !will [12:10:12] !! [12:10:12] hello :) [12:10:30] !hello [12:15:41] ok...had not .lighttpd.conf...copied the default one...has a parse error... [12:15:49] just for the records [12:20:35] who is user s51892? [12:21:33] nosy: if it's in your tool's replica.my.cnf , it's your MySQL user alias for your tool [12:21:44] and else? [12:22:06] nosy: else? [12:22:15] i am another user [12:22:39] on enwiki instance seems to exist something like what i wanted to migrate [12:22:49] id not have to do work twice [12:22:54] nosy: you are many users 1.) user account 2) tool account 3) mysql account [12:23:00] so it would be a good idea to corrdinate with the user [12:23:07] its a mysql user [12:23:51] nosy: yep you can't use your generic user/tool account to connect to the databases [12:24:08] hedonil: i guess you get me wrong [12:24:17] everybody get me wrong *cry* [12:24:21] ;) [12:24:23] nosy: hehe [12:24:46] * hedonil has got something for nosy [12:24:46] id want to know how can i reach another user whos mysql user is s51892 [12:25:23] handkerchief? [12:25:57] nosy: ahh, user reverse engineering ;) [12:26:03] yes [12:26:21] nosy: i think there will be only on way. admins [12:26:29] what is the logprefix for jenkins? [12:26:34] nosy: "getent passwd 51892" (don't know if that is the correct(TM) way) gives tools.toolserverdb, "getent group tools.toolserverdb" gives nosy?! [12:26:42] !log integration upgrading elasticsearch on integration-slave1002 [12:26:44] Logged the message, Master [12:27:52] ah scfc_de thx [12:28:01] getent passwd 51073 gives what i want to know [12:28:36] scfc_de: wooo. hack. [12:28:54] scfc_de: ah you are a member, cool [12:29:49] found s51073__toolserver on enwiki, which holds the table namespace [12:30:16] just had the idea it might be a copy of a toolserverdb table but i have no access to it [12:31:02] nosy: No, that was just a test whether the claims in the bug report hold up. The meta table needs to be properly set up as a non-tool DB. [12:31:31] scfc_de: i see [12:32:05] thats true anyway [12:32:17] ok :) thx for the help so far [12:36:39] melting s5 down :D [12:36:50] select * from revision limit 1 on federated table [12:37:08] sucks the whole table into memory and then does seek [12:37:29] not a problem on ts but can be with federated tables [12:38:16] ok...killed the qery [12:47:23] scfc_de: fwiw, I don't think https://www.mediawiki.org/wiki/Analytics/Hypercube is anywhere near 'prioritized' [12:50:47] YuviPanda: I know :-); so I'd prefer more space for now. [12:51:25] scfc_de: :) [13:07:24] hashar: I ran a couple of resize tests yesterday... [13:07:33] In all cases, it says 'resizing…. 0%' [13:07:34] andrewbogott: awesome! [13:07:39] then errors out and sometimes the instance doesn't run anymore [13:07:40] oh [13:07:41] less awesome [13:07:48] so I think that's a non-starter. [13:07:55] so resizing is definitely broken in our nova version right ? [13:09:13] Well… it certainly isn't working now. I haven't investigated much [13:09:27] Probably not worth spending all day debugging unless it will take you multiple days to rebuild ganglia [13:10:04] I think in all history it's only ever worked for us once [13:11:36] I was taking that opportunity to trick you in investigating nova resizing :] [13:11:49] I thought it would be nice to give it a try and document it since that might be handy from time to time [13:12:04] for ganglia, I can just create a new instance, apply the class, manually clone graphite-frontend [13:12:11] then send a puppet patch to update the destination address [13:12:11] I'll ask in #openstack if there's an easy fix [13:12:16] not a big deal. Can do that tomorrow [13:12:34] I was merely using Ganglia on labs as an excuse :] [13:24:24] andrewbogott: hi, can i build a trusty image in labs? [13:24:41] matanya_: not yet. Getting trusty working with puppet is proving… complex. [13:25:03] hmm, thanks. need any help on this ground? [13:25:19] Right now it's in ryan's court. I should check in and see if he's still working on it. [13:50:55] hello labs folks [13:51:03] I've got a weird issue [13:51:07] http://test-reportcard.wmflabs.org/vendor/vendor-bundle.min.js?1397742317012 [13:51:17] that file is cut off (check the end of it, it's in the middle of doing some js) [13:51:25] but the source file is fine on disk [13:51:51] this is happening both on an instance I just deployed to, AND on an instance I haven't touched for a while [13:52:17] have there been any changes to the proxies or something that would explain? [13:53:00] andrewbogott: ^ might you know anything? [13:53:37] milimetric: unlikely to be a proxy thing, but let me look... [13:53:50] the problem started happening sometime this morning [13:54:24] it was fine around 07:01 EST, so almost 3 hours ago [13:55:57] milimetric: try a wget locally? That can eliminate the proxy from the equation [13:56:30] yes, andrewbogott, i tried curling locally and I get the full file [13:56:38] oh! That's interesting. [13:56:52] YuviPanda: any reason the labs proxy would truncate files as per ^ [13:57:09] oh? [13:57:10] * YuviPanda reads [13:57:42] it shouldn't. [13:57:47] i've no max_filesize defined [13:57:47] from limn1 in eqiad: [13:57:48] curl localhost:8001/vendor/vendor-bundle.min.js?1397742317012 | wc [13:57:56] returns the proper character count [13:58:19] the count of what's being served is about 189k [13:58:31] the proper count is about 421k [13:58:41] (sorry I got my standup to run, I'll brb) [14:27:41] milimetric: when I wget that url locally I get (I think) the whole thing, size 421547 [14:27:52] so maybe it's just the browser truncating when you view the file [14:28:51] hm, curl and wget produce different results [14:32:21] andrewbogott: yeah, curling locally does this for me: [14:32:22] curl: (18) transfer closed with 356305 bytes remaining to read [14:32:42] milimetric: actually… looks like /var is full on the proxy, probably causing weird proxy behavior [14:34:25] milimetric: better now? [14:35:15] Coren, can you suggest a good existing pattern for moving logs out of /var? (I can think of plenty of ways to do it, wondering if there's a standard) [14:35:20] all better andrewbogott [14:35:24] thank you! :) [14:35:31] milimetric: it won't last, that log is growing at knots [14:35:37] but I'll try to make a proper fix [14:35:57] Also: bad nginx, changing your behavior just because you can't log! [14:36:07] andrewbogott: I don't think there's a standard. If it's a lab instance, you can always use a lvm_volume with a mountat => '/var/log' [14:36:23] Coren: yep, that was going to be my first choice [14:36:28] maybe I'll make a role for that [14:36:37] "biglogs"? :-) [14:36:43] yeah [15:07:03] Coren: So… of course now none of the /var/log/ subdirs are available because I clobbered /var/log :( [15:07:19] Is there any way to access the old /var/log that I replaced with my new mount? [15:07:49] andrewbogott: Oy, right, that makes sense for a new instance but not a running one. Hm. [15:08:11] andrewbogott: Well, the easiest way -- by hand -- is to just umount it from /var/log and mount it elsewhere first. [15:08:27] Yeah, I'll just do that. [15:08:38] The proxy is actually working fine after that change, but presumably lots of services are failing to log now [15:12:01] !log project-proxy rebooting dynamic proxy to move logging to a bigger partition [15:12:03] Logged the message, dummy [15:27:02] Coren, looking at what Eloquence said yesterday: "[wma is] embedded in the chrome for articles with geocoords" [15:27:16] Am I misunderstanding, or does that mean that production wikis are hitting that service? [15:27:49] You're understanding correctly, I think, which makes me think we should start looking at making this prod; or we need to sit down and discuss exactly where the line lies. [15:28:35] the /var/log issue overnight is because there are so many requests to wma that the proxy log of those requests filled the partition with the current un-rotated log [15:28:42] We're generally not set up for that kind of hammering [15:29:14] do you recall the project name? [15:29:32] oh, um… miniatlas something [15:30:24] hm, nope, that's not it... [15:33:26] aude: you're involved in the 'maps' project? [15:33:40] I'm hoping to start an email thread, wondering if you have a mailing list for involved people [15:35:45] andrewbogott: It's arguable whether we should try to revise how they're doing thing or whether we want to start considering how we can adapt our architecture. [15:36:27] Hm… I'm pretty happy with a 'no production reliance on labs' policy. [15:36:35] That doesn't tell us what they should do instead, of course. [15:37:02] I guess there are two issues, though, 'reliance' and 'traffic' [15:37:30] (tools is an exception, of course.) [15:40:51] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #9: STILL FAILING in 3 min 27 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/9/ [15:41:14] andrewbogott: Mostly I think we need to consider failure modes. [15:41:42] andrewbogott: tools has gotten a bit more robust to tools being hammered; but it's when things break the issue lies. [15:42:43] Coren, there's the question of what happens when things break, and the question of 'if you do this things will certainly break' [15:43:02] when labs is hooked into production in often amounts to a DOS attack [15:43:06] *it [15:43:44] True, but that means we either need to prevent it, or to find a reasonable mechanism to allow it. The latter probably means throwing resources/money at it. [15:44:33] Ah, you mean a technical way to prevent it vs. just saying "Don't do that" [15:51:40] Anyone have an email contact for dschwen? [16:17:50] Coren: Ping [16:20:27] wtf i get an internal error on all index.php's`? [16:20:40] ^^ [16:26:28] Steinsplitter: Which ones? [16:26:34] marktraceur: all [16:26:45] "500 Internal Server Error" on all Tools web pages [16:26:46] All on *labs* or all wikis everywhere [16:26:48] OK [16:27:04] Steinsplitter: Can you link to one example? [16:27:14] http://tools.wmflabs.org/delinker/ [16:27:19] http://tools.wmflabs.org/steinsplitter/ [16:27:21] etc. [16:27:41] Both work for me [16:27:56] oh, indeed [16:27:58] back [16:28:00] Heh [16:28:15] Steinsplitter: The perennial solution of "complain about it, when someone else looks at it it will be fixed" :) [16:28:29] :) :D [16:29:14] marktraceur: the error is back :P [16:30:48] Disagree. [16:30:54] Maybe something is yoyoing. [16:33:33] marktraceur: try without "/" at the end http://tools.wmflabs.org/delinker [16:33:54] gives me an internal error, but with "/" works :D strange [16:33:59] Aha. [16:34:05] That is weird. [16:34:09] indeed [16:34:36] mabye somthing in lighty is broken :/ [16:34:36] Sounds not-urgent though [16:34:52] it is on all tools page O_O [16:35:20] it not urgent, yes :P [16:35:28] i hope *g* [16:58:33] Steinsplitter: Actually, there is a non-trivial solution I need to work on. Strictly speaking, the URIs without the slashes are simply invalid -- you're not allowed to have a directory-like object return content whithout the slash. In *practice* webservers will simply slip a redirect from /foo to /foo/ in (and that is the default behaviour) but our proxy setup makes that more complicated. [16:59:13] k, thx [16:59:37] But per the standards, "http://tools.wmflabs.org/delinker" is not valid if "http://tools.wmflabs.org/delinker/" is. Yeah, it's annoying and I do want to work around it. [17:26:15] And https://bugzilla.wikimedia.org/64058 for the tracking bug. [17:27:57] Coren, so… it may be that I was overreacting regarding the proxy. The /log file/ is growing like crazy, but I don't see any evidence that the box can't actually handle the load. [17:28:08] I'm just generally alarmed at use not intended by the manufacturer. [17:29:27] * Coren chuckles. [17:29:33] "For external use only" [17:29:50] wikim...atlas was previously running at the Toolserver, so it should be manageable. [17:29:54] Truth be told, I'd prefer if we were really gingerly with logs anyways. [17:30:05] miserly* [17:30:19] Keep few, keep them for brief periods, purge often. [17:30:25] Yeah, I'm thinking that I'll tell ddschwen to stand down, and instead just turn off logging [17:30:36] well, make it configurable so we can turn them on on demand for a short time. [17:31:34] Let's see… I have 88M in 2 hours, that means 1g of logs every 24 hours. So if I jigger logrotate to purge two-day-old logs [17:31:42] Coren, preference? Quick purging, or turn off logging entirely? [17:32:12] I'd rather keep at least a couple hours of logs; you kneecap our ability to debug otherwise. [17:32:32] ok [17:32:38] * andrewbogott digs in logrotate [17:34:09] All of that said, there /is/ a good case to be made to have a real hardware proxy /in front/ of labs rather than as a labs instance for the more general case. [17:34:33] If only because that's a problem we already know how to solve. :-) [17:35:12] * andrewbogott nods [17:37:39] And, on a more philisophical basis, I still don't agree that a hard line between "prod" and "labs" is useful in the long run. It's a matter of managing expectations and "SLA"s; but the pie-in-the-sky objective IMO should always be that the line between the two is fuzzy and best examined case-by-case. [17:38:36] In other words, if we can make labs scale well to near-production stuff, then we win. [17:39:20] I'd be happy for labs to support production-level uses... [17:39:40] but with non-tools projects, it seems very unlikely that we can guarantee any kind of comparable uptime [17:40:17] At least we'd need more than two ops to pay attention to it :) [17:48:25] andrewbogott: Tru dat. Though, to be fair, our general reliability and performance have been going stadily up. [17:48:37] Despite increasing usage; that's a good sign. :-) [17:49:06] Until openstack provides some kind of in-place upgrade path, we're committed to substantial periodic downtime... [17:49:15] well, where 'period' is 'annual' [17:49:51] hrm. i'm getting permission denied trying to log into my labs instance via ssh. [17:49:56] Permission denied (publickey) [17:50:01] my key hasn't changed, as far as i know. [17:52:26] jorm: What instance? [17:52:31] mwreview [17:52:39] ssh to unicorn.wmflabs.org [17:55:34] jorm: The box seems strangely hosed. The actual mount/points/ for the NFS stuff are gone, as though they were rmdir'ed. [17:55:49] nice. [17:55:53] Coren: That box was migrated… maybe it needs autofs purged? [17:56:34] Aha! Yes, that is it. [17:57:25] Well, it's purged, but still tied into the kernel. Needs reboot. [18:02:37] jorm: Looks all better now, and fully modern. :-) [18:42:18] Project CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox build #10: STILL FAILING in 3 min 24 sec: https://integration.wikimedia.org/ci/job/CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox/10/ [19:33:27] Coren: Hrm. What would be the easiest way to send received e-mails to a running process? That process would be 'somewhere' on the grid, and I'm not quite sure what the easiest way to contact it would be. [19:34:20] well, actually, a file queue would be the easiest, but that's explicitly disallowed ;-) [19:34:58] valhallasw: maybe a redis queue, works for me in a similar way [19:34:58] valhallasw: redis queue! [19:35:10] YuviPanda: :-) [19:35:29] hedonil: as in: you already have a mail-to-redis working? [19:35:42] * valhallasw reads up on redis [19:35:57] valhallasw: not mail, but other messages [19:36:02] ah, ok [19:36:30] grrrit-wm: runs off a gerrit to redis thing [19:37:16] Actually, a redis queue /does/ sound like a great idea. [19:38:13] YuviPanda: now that I think of it, that confuses me. If it's a key-value store, what exactly do you store? And how do clients know there's a new message? [19:38:32] valhallasw: it's not just a key value store. it has a 'list' type. [19:38:45] Sure. What's the first element in the list? [19:38:49] valhallasw: so the producer 'LPUSH'es messages into a queue with a shared key [19:38:58] valhallasw: One of the value types redis has is 'list' from which you can push and pull [19:39:00] valhallasw: and the consumer 'RBPOP'es messages out of the queue with a shared queue [19:39:03] ... what Yuvi said. [19:39:05] valhallasw: RBPOP is a 'blocking' pop [19:39:27] valhallasw: so your client will wait until there is a new message (you can specify infinite timeout) and return only when something has been LPUSHed [19:39:45] but if one client pops it, why is it not lost for all other clients...? [19:39:45] What's the size limit for the value? [19:40:19] valhallasw: The model would have your worker job be the only client, with incoming mail being lpushed [19:40:22] valhallasw: that's the semantic for RBPOP. if one client pops it it is lost. [19:40:30] valhallasw: as in, it is given to the client and removed from the list [19:40:45] valhallasw: http://redis.io/commands/BRPOP [19:40:49] Right. [19:41:05] I was confused because I thought the gerrit-to-redis queue could be used by multiple clients [19:41:17] which, if I understand correctly, is only possible if they use different queues [19:43:04] "A String value can be at max 512 Megabytes in length." (http://redis.io/topics/data-types) So that could accomodate a mail. [19:43:40] yeah, I don't think bugzilla sends emails that are that large ;-) [19:51:06] !search [19:51:06] http://bots.wmflabs.org/~wm-bot/searchlog/index.php?action=search&channel=%23wikimedia-labs [19:51:23] bleh 404 :/ [19:51:41] petan: ^ [19:55:44] valhallasw: yeah, for gerrit-to-redis you can have multiple queues [20:00:13] * valhallasw wonders if python 3.3 is available on tools yet [20:00:26] * valhallasw puts on his sad face [20:01:00] finally I have a reason to fiddle with python 3 (asyncio) :P [20:02:14] Coren: any plans to have exec hosts with the 14.04 LTS? [20:02:25] valhallasw: shouldn't be too hard if we can find an appropriate package... :D [20:02:34] (for 3.3 that is) [20:02:37] there's a PPA, but those are illegal(TM) apparently [20:02:49] https://launchpad.net/~fkrull/+archive/deadsnakes [20:02:56] even has 3.4 [20:04:33] valhallasw: I'm hoping to have working Trusty exec hosts sometime around June. [20:04:50] ok [20:05:41] * YuviPanda goes off [20:05:42] night [20:05:45] gn YuviPanda [20:05:53] night valhallasw! [20:19:29] zz_yuvipanda: actually redis seems to also have a real 'publisher-consumer' model: http://redis.io/commands/publish [20:19:58] zz_yuvipanda: but redis-py doesn't fully support it [21:16:19] * valhallasw cheers [21:17:21] valhallasw: You can haz success? [21:17:23] stdin-to-redis is working \o/ [21:17:44] That's actually useful enough that it'd be a Cool Thing if you made that generally available. [21:17:58] not sure whether to also add 'add to list' as an option, over just 'send to subscribers' [21:18:15] both are generally useful [21:25:00] Coren: anyway; /data/project/wikibugs/toredis.py [21:26:00] * hedonil loves redis; simple & powerful [21:26:05] Ima probably snag this and make it directly usable from the MTA given how useful it is. [21:26:20] Beats having to indirectly call it from the grid. [21:26:38] Coren: I'm still working on rpush, but publish is working [21:26:54] (once you're done) [21:27:55] woot, rpush also works. That was surprisingly easy :-) [21:31:36] Coren: NFS seems slow today :/ [21:32:19] (specifically: saving toredis.py and running python toredis.py) [21:32:21] The NFS server is nearly network-saturated. We've got a port addition planned soon. [21:32:46] Right now, it has peak usages around this hour and ~ 6h from now that make things a bit sluggish. [21:33:07] Usage has grown some 300% in the past months. [21:33:13] I see. [21:40:23] okay, extended to accept every redis command that could evah be useful (append, lpush, lpushx, rpush, rpushx, sadd, pfadd, publish) [21:41:49] hedonil, ping [21:45:52] Coren, hi [21:49:03] Hi CP [21:49:29] Cyberpower678: thanks for putting me on your SPAM blacklist :P [21:49:58] hedonil, ah so you know. :p [21:50:18] googlebot and baiduspider hits were getting out of control. [21:50:36] Cyberpower678: I assume this was by accident [21:50:39] So they are now blocked from the tool. [21:50:57] * hedonil is sure not being baidu or googlebot [21:51:05] Actually, anything identifying itself as spider and bot are now banished;. [21:55:06] hedonil, I was testing the blacklist before making it go live. I accidentally left the keyword "mozilla" in the contains array, meaning that any UA containing "mozilla" was blocked from the tool. That's about every user in existence. :p [21:55:25] Cyberpower678: haha [21:57:14] Cyberpower678: it requires a certain amount of guts to block a mozilla UA ina FOS environment :P [21:58:14] hedonil, French Onion Soup? [21:58:59] Cyberpower678: hehe no! Fructooligosaccharide [21:59:09] ;) [21:59:18] FOSS FOSS FOSS [22:00:40] Fruit of the Spirit is also a very nice term [22:57:21] Hey all [22:57:33] How do I find out if I have ssh access to a group on Labs? [22:57:41] Specifically the editor engagement group? [22:59:41] StevenW: Go to https://wikitech.wikimedia.org/wiki/Special:NovaProject and see if you can select the project [22:59:55] I can [22:59:59] Gracias amigo [23:00:17] AFAIK all project members have ssh access and you can only see a project there if you are a member [23:59:00] is ganglia.wmflabs.org supposed to be working?