[00:01:05] ok. updated [00:01:21] looking [00:02:14] can't just say /vX/. need to actually split the doc apart per version [00:02:20] since the urls themselves can change too [00:03:28] {tenant_id} in openstack is actually a uuid, but we don't use uuids for projects [00:03:36] which actually means we can't rename projects :D [00:06:14] one plus of using hostnames for map id is that enforcing ownership by project is easier [00:07:11] looks good to me, Ryan_Lane [00:07:15] cool [00:07:23] you've a couple of extra / left [00:07:31] fixed em already :) [00:08:45] Ryan_Lane: heh, right. [00:09:10] Ryan_Lane: what if... I didn't use python, but used some other language? like... node? [00:09:16] for the API [00:09:31] I'm totally cool with using Python, since I know that is what most people are comfortable with [00:09:39] but JS has a lot of people comfortable with it too [00:09:46] python would be good [00:10:06] we're basically making an openstack api [00:10:19] the rest of openstack is python, so it's good to stick with the ecosystem's language [00:10:26] makes sense [00:10:44] oh, btw. use Apache2 license :) [00:11:04] Ryan_Lane: I was thinking this would be just a module in the puppet repo? [00:11:18] hm [00:11:20] Ryan_Lane: since the rest of the stuff is there, makes sense to keep this there too? [00:11:26] Ryan_Lane: but then I can't really +2 things to it, which is bad. [00:11:36] a separate repo would be good [00:11:39] hmm, yes. [00:11:45] dual licensed to Apache2 and WTFPL :P [00:11:46] we can deploy to it using git-deploy [00:11:56] heh [00:11:57] ooo! [00:12:03] would need virtualenv too [00:12:16] shouldn't be too hard. [00:12:17] dual licensing makes things murky ;) [00:12:35] but it's your code [00:12:35] sure, I'll just put WTFPL on it :P [00:12:43] that's an evil license [00:12:47] pfft. [00:13:25] Ryan_Lane: btw, where exactly does wikitech run from? [00:13:30] virt0 [00:13:34] ah, right. [00:13:36] not 'cluster' [00:13:39] yep [00:13:49] so we can easily firewall proxy-project-project-proxy and wikitech [00:13:57] so no SSL, etc. [00:14:00] fine, fine [00:14:04] SSL would be preferable :) [00:14:26] inside the cluster? [00:14:28] well, I guess that would require a real cert [00:14:34] yes [00:14:49] this would need to travel through the network node [00:14:51] uwsgi supports ssl out of the box if you can give it a cert. [00:15:07] well, we're going to need a cert for the webserver anyway ;) [00:15:13] Ryan_Lane: oh, yes. that is true. [00:15:56] Ryan_Lane: btw, tools will have its own instance. [00:16:06] of proxy. [00:16:08] not use the labs-wide one [00:16:14] why's that? [00:16:30] because we want to use... identd for auth :D [00:16:39] eh? [00:16:43] idea is [00:16:50] there's a different deamon that'll sit there. [00:17:04] and we write wrappers for jsub for different languages (node, uwsgi, go, etc) [00:17:17] that'll start the job on SGE, and then communicate the hostname/port to the proxy API [00:17:27] ah [00:17:30] Ryan_Lane: and then the proxy thing will call identd on the host to confirm who is the caller [00:17:30] that seems sane [00:17:50] Ryan_Lane: and the mappings will be to tools.wmflabs.org//*whatever* [00:17:54] Ryan_Lane: and not just domains. [00:17:54] so [00:18:07] is ident actually a sane way of doing that? [00:18:23] Ryan_Lane: of verifying that the person making this connection is the tool user? yes! [00:18:24] isn't that pretty spoofable? :) [00:18:29] Ryan_Lane: only if you've root [00:18:33] ah. ok [00:18:39] Ryan_Lane: and if you've root on toollabs, we're fucked anyway. [00:18:42] yep [00:18:57]

[00:18:58] well, if that's the case, we don't really need the api or anything for tools [00:19:04] it can just write to redis directly [00:19:13] Ryan_Lane: ah, no. [00:19:16] no need for wikitech to handle it [00:19:22] that makes things easier :) [00:19:29] Ryan_Lane: yeah, no wikitech for toollabs. [00:19:50] Ryan_Lane: you can't make toollabs write directly to redis, since then anyone can remap anyone else :) [00:20:01] Ryan_Lane: at least with wikitech you can put access controls on the wikitech side :) [00:20:25] Ryan_Lane: with that, yes - if you agree to put the data in the wikitech db, then the API layert is useless [00:20:31] well, I mean some other thing that Coren handles will handle tools' proxy [00:21:03] Coren|Away: won't have to do anything with uwsgi tyrant, and the current tools-webproxy can also go away :) [00:21:22] our apaches will be degraded to lowly php-execution-hosts, and bwahahah! :P [00:21:29] * Ryan_Lane wants mediawiki to do less and less in the future [00:21:41] yeah, a sentiment I agree with :) [00:21:53] at some point we may want to switch to using horizon rather than MW + OSM [00:21:59] * andrewbogott fell asleep, oops [00:22:07] andrewbogott: hah. I had that issue last week [00:22:11] But, yeah, I will work on packaging. [00:22:50] YuviPanda: the other plus of writing an API is that your service handles all the logic [00:23:12] andrewbogott: do I need to give more details about the package? [00:23:26] Ryan_Lane: oh yeah, +1. I can change how stuff is done on Redis side without worrying about wikitech [00:24:31] yep [00:24:42] as long as the API works the same way, the logic can change [00:25:11] and the logic is within the same language, rather than spread between them :) [00:25:13] YuviPanda, I think we have a git repo for the existing nginx builds… figuring I'll just use that to build the tip (plus whatever security patch.) Is there more to it than that? [00:26:18] andrewbogott: the lua module in even teh latest nginx-extras is waaay outdated. [00:26:44] andrewbogott: and needs updating. same with nginx. I'd need at least 1.3 of nginx and 0.6 of lua - and preferably 1.5 of nginx and 0.8 of lua. [00:26:55] andrewbogott: so I guess you need to pull in new sources? [00:29:18] Yeah, I meant 'the tip' of nginx-extras [00:33:22] andrewbogott: hmm, I think that is what I meant, yeah :) [00:33:29] andrewbogott: 'tip' -> from upstream? [00:33:37] yeah [00:33:40] right! [01:03:14] [01:03:38] Coren|Away: ping [01:04:13] Where's petan? [01:04:41] Anybody working at labs right now? [01:04:47] Crashed hours ago. [01:05:04] -.- [01:05:04] @seen petan [01:05:04] Technical_13: Last time I saw petan they were quitting the network with reason: Ping timeout: 264 seconds N/A at 8/27/2013 5:36:31 PM (7h28m33s ago) [01:05:15] @seen Coren|Away [01:05:15] CP678|Webchat: I have never seen Coren|Away [01:05:32] @seenrx Coren.* [01:05:32] Technical_13: Last time I saw Coren they were changing the nickname to Coren|Away and Coren|Away is still in the channel #wikimedia-operations at 8/27/2013 11:22:31 PM (1h43m1s ago) (multiple results were found: Coren|AFK, Coren|Sleep, |Coren|, Coren|Dinner, Coren|Lunch and 14 more results) [01:05:54] @seen Coren [01:05:54] CP678|Webchat: Last time I saw Coren they were changing the nickname to Coren|Away and Coren|Away is still in the channel #wikimedia-operations at 8/27/2013 11:22:31 PM (1h43m22s ago) [01:06:06] Lovely. [01:06:18] Perfect time for labs to crash. [01:06:26] What's up? [01:06:43] You in charge of xtools now? [01:06:44] I'm trying to monitor my spambot program. [01:07:00] For the most part yes. [01:07:22] CP678|Webchat: What's up? [01:07:22] Update the new page counter. :) [01:07:54] Technical_13: I'm busy. [01:08:10] It doesn't do all ns [01:08:19] Well... [01:08:37] It does if you mess with tbe url [01:09:17] Coren: labs is broken. [01:09:36] And I'm guessing that my bot is still in operation. [01:10:01] Which means, without any control, it might start mass tagging pages tomorrow morning. [01:10:56] Ah, webserver-01 ran out of ram again. xtools once more. [01:12:18] Yeah, X!'s editcounter is br0ked. [01:13:47] * Coren tries to find out what is wrong with it. [01:20:06] Hmmm. Nope, something deeper is going on. [01:20:13] Coren: The edit counter and all of his tools ran just fine on toolserver. [01:24:00] yeah, grrrit-wm left! [01:30:47] just had my IRC bots go down [01:31:10] (on tools) [01:32:20] rschen7754: I'm on it. Should return shortly. [01:32:25] :D [01:32:27] thx [01:35:02] Yeah, there's definitely something wrong with the xtools atm; either it spams error messages or just hangs there consuming memory. [01:41:06] There are several hundred thousand errors in the log in the past few days; most being syntax errors. I'm thinking there was a change recently that broke something. [01:47:54] Coren: er, so what's breaking the rest of toollabs? [01:48:40] Yas again? [01:49:00] legoktm: I have no idea. A lot of stuff is running hard and it's not clear why. [01:49:39] There's a LOT of filesystem traffic. [01:49:51] &ping [01:49:54] @ping [01:50:06] Nope.. guess that is gone. [01:50:30] Looks like it's all coming from -exec-04 [01:50:36] and slowing everything else down. [01:51:09] back up now, or still down? [01:51:32] rschen7754: Still iffy. One of the exec nodes is crunching so hard it's causing pain elsewhere. [01:51:45] ok, so kicking would not be a good idea then :P [01:52:10] tools-exec-04 Load: 436.7% [01:52:14] * legoktm facepalms [01:52:17] :( [01:53:36] Yeah, -exec-04 is completely bust. [01:54:03] * Coren headdesks. [01:54:10] Which is why I just restarted -02. Of course. [01:54:14] * Coren is a moron. [01:54:33] At least the gridengine will just reschedule stuff. [01:54:37] Coren: are yoy using ? [01:54:37] Yeay gridengine. [01:55:03] !blink [01:56:36] Coren: ping [01:56:53] Pong [01:57:19] Coren can you tell me if a file on xtools was changed recently. I have no access at the moment. [01:57:26] :55] !blink [01:57:26] [21:55] Technical_13: When you use , you're not just hurting yourself, but also hurting those around you [01:57:31] !blink del [01:57:31] Unable to find the specified key in db [01:57:39] ah, even better :D [01:57:46] CP678|Webchat: I'll be able to tell you in a minute; I'm making sure the compute nodes go back up. :-) [01:58:01] !YuviPanda del [01:58:02] Unable to find the specified key in db [01:58:30] xtools hasn't been causing problems like this before. [02:00:13] That first puppet run on reboot is soooo slow. [02:00:30] CP678|Webchat: Lemme check to see if I see dates that look askew [02:02:21] [28-Aug-2013 01:58:09 UTC] PHP Warning: syntax error, unexpected '=' in Unknown on line 33 [02:02:21] That's about the worse error message ever. in "Unknown"? [02:03:07] At least you know it's on line 33... [02:03:09] :/ [02:03:11] WTF is Unknown?!? [02:03:13] hey, better than 'Unknown error, in index.html line 0' [02:03:19] (PhoneGap once gave me that) [02:03:25] oooo. perhaps caused in the parse_ini_string(); so it'd come from the ini file.addimagepath [02:03:57] Coren: have you found any changes within the last 2 months. [02:04:12] CP678|Webchat: Looking now. [02:04:36] The dates on every file should be around 1.5 months old, aprrox. [02:05:01] YuviPanda: it could be worse, it could be: Syntax error at '}'; expected '}' [02:05:30] find takes forever: there are directories in there with 100K+ files! [02:06:20] I have a feeling JackPotte's been tampering with the files again. [02:06:33] Anything he touches, tends to break the code. :/ [02:06:40] Ryan_Lane: at least with that you can think 'damn, encodings!' or 'damn, indents!' [02:06:52] Ryan_Lane: *try* finding a js error in line 0 of index.html [02:07:06] :) [02:07:08] I'm seeing mostly config files changed Jun 13 [02:07:33] Can you tell which user changed them? [02:07:49] No, not really. [02:07:55] Oh wait. Is it a mass change? [02:08:37] Coren: ^ [02:08:54] No just a couple. [02:09:16] Can you give me the file paths for those files? [02:10:03] public_html/articleinfo/data is really evil when you try to find; it forces 100K+ stat() calls over NFS [02:10:16] ... that should really go in a DB, not on the filesystem. [02:11:24] Coren: actually, that's a temp directory. The files should actually delete after a certain amount of time. [02:11:59] Does it hurt anything if I blow its contents away? [02:12:07] No [02:15:21] Coren: anything new? [02:15:25] Coren: is labs back up? [02:15:39] YuviPanda: Everything seems to be working atm [02:15:47] Coren: jobs weren't restarted? [02:16:36] YuviPanda: I see no continuous jobs in error mode, and lots of restarted ones. Which one were you worrying about? [02:16:49] Coren: lolrrit-wm [02:17:00] Coren: Ive got jobs that didnt restart [02:17:43] Ryan_Lane: I miss grrrit-wm. [02:18:21] Coren: it seems 'stuck'? [02:18:28] job 748654 [02:18:29] !jobs [02:18:37] Please do not restart my jobs. I beg you. [02:18:54] CP678|Webchat: I only see config files and templates changed. No code. [02:19:17] Coren: 748654 seems stuck in some sort of... deletion purgatory? [02:19:19] When was the latest change? [02:19:22] Coren: job 748654 is already in deletion [02:19:29] Coren: but that's been like that for a minute or so now [02:19:54] YuviPanda: Yeah, I haven't seen that state before. Gimme a sec to find the info for CP678|Webchat then I'm all yours. [02:20:03] Coren: alright. [02:20:15] Elsie: that 'job 748654 is already in deletion' is from grrrit-wm tho [02:21:16] Hm? [02:21:20] CP678|Webchat: Outside of Peachy, most recent I see is sitenotice.php on Jul 28 [02:21:53] Elsie: you said you missed it :) [02:22:05] I'd like to know what's going on in Gerrit. [02:22:12] Without grrrit-wm, I'm a lost child. [02:22:20] That was me. Hmm. Something seems to be going haywire. I set it to display all errors on the page, so I can be notified of them real quickly. [02:22:42] * CP678|Webchat puts Elsie up for adoption [02:22:52] Elsie: stalker [02:23:17] Coren: thanks. [02:23:30] Peachy has a lot of changes in August. [02:23:52] Coren: that would be the autoupdater at work. [02:24:11] Peachy updates itself. [02:24:19] YuviPanda: That's an odd state. "running but deleted"; looks like it's stuck trying to kill it. [02:24:29] Coren: can you put it to rest? [02:25:30] Coren: still zombie'd [02:25:38] Coren: actually one last question. Can you tell when xtools began to flood error messages? [02:26:14] The edit counter seems to be hanging up. :? [02:26:44] Great. I get to do some massive debugging tomorrow. If I can even find the time. [02:26:46] Hm. Whatever happened earlier seems to have made more than one compute node sick. [02:26:51] CP678|Webchat: Lemme check. [02:31:50] Actually, I see errors pop up in large bursts starting around 31 Jul [02:33:06] Coren: any luck in killing it? [02:33:15] And then a big bunch starting the 25 [02:33:21] (Aug) two days ago [02:33:49] YuviPanda: The grid seem to be mildly wedged. It should unstuck as soon as -03 reports in (a few minutes) [02:33:57] Coren: oaky [02:34:45] grrrrit-wm, otoh, seems to not have restarted at all for some reason. [02:36:20] Coren: err. I pushed https://github.com/cyberpower678/Peachy/commit/e16773bfcea03ffba42eee32bd3185a604977837 to Peachy on July 31. [02:37:02] That seems unlikely to be related. [02:37:54] Coren: so, I've been trying to kill it myself [02:37:55] no luck either [02:38:04] Other than that, I can't find a connection to these error surges. [02:38:07] tried agian. nothin [02:38:26] YuviPanda: Yeah, I know; the scheduler is confused about the state of a node. [02:38:38] Well, I'm going to bed. Thanks for your help Coren. Good night everyone. [02:38:39] alright, I'll wait some more :) [02:38:54] i'd want to get grrrit-wm back up before sleeping, but it is alread 8:00 AM [02:39:22] The puppet runs on boot make the boxes take *forever* to startup [02:40:19] Ah, it's back up so now the delete went through okay. [02:42:43] Coren: yes :) [02:43:10] I really don't get what happened. Every single instance seems to be wonky to some degree. [02:46:41] [bz] (8REOPENED - created by: 2MZMcBride, priority: 4Normal - 6normal) [Bug 36885] wmflabs.org does not resolve - https://bugzilla.wikimedia.org/show_bug.cgi?id=36885 [02:53:36] ... there seems to be a lock held on the page table of enwiki_p. [02:53:46] Perhaps that's what caused all those things to pile up and wedge. [03:06:33] Boom. Headshot. [03:07:08] oh come on. we did spray bullets elsewhere first :P [03:09:17] And X!'s edit counter lives again. Not entirely surprising that it'd wedge if it couldn't access the page table. [04:28:26] Hi all [04:30:31] I have Q [04:30:54] Is some one there to help me [04:32:03] Elph: ? [04:32:53] I want to get report from wikiquote [04:33:29] labs question: I'm working with an older MediaWiki labs instance, does anyone know where the admin password was stored prior to the orig/adminpass setup? [04:33:43] What changes should I do to wikipedia's reports code [04:33:45] ? [04:34:29] idk about wikiquote [04:34:55] Elph , you might want to ask on #mediawiki, I'm not sure how that's a labs question unless you have a labs instance where you're running the report [04:35:23] thanks [04:36:36] Elph, I'm not sure what you mean by "report", but there are dumps of the content, e.g. http://dumps.wikimedia.org/enwikiquote/20130827/ [04:36:53] or dbs [04:37:10] report means the result of a query :) [04:37:23] Elph: API? [04:37:30] sql [04:37:37] hi [04:37:37] oh [04:37:43] need one help [04:38:13] it's simple query [04:38:24] on login to server [04:39:21] I am getting error like " The user key you've selected is not registered in the remote server" [04:39:36] i have updated the public key in openstack [04:40:39] Elph: connect to a server like "commonswiki.labsdb" (checking about wikiquote) [04:42:10] "enwikiquote.labsdb" [04:42:52] database "enwikiquote_p" [04:43:08] for ar [04:43:25] "arwikiquote.labsdb" ? [04:43:41] and usename & password are stored in "~/replica.my.cnf" [04:43:58] db list: https://noc.wikimedia.org/conf/all.dblist [04:44:32] thanks a lot [04:44:55] yw [04:45:14] naveenpf: ? [04:49:20] !google xshell The user key you've selected is not registered in the remote server [04:49:20] https://www.google.com/search?q=xshell+The+user+key+you've+selected+is+not+registered+in+the+remote+server [04:53:34] @seen petan [04:53:35] zhuyifei1999: Last time I saw petan they were quitting the network with reason: Ping timeout: 264 seconds N/A at 8/27/2013 5:36:31 PM (11h17m4s ago) [04:53:46] @notify petan [04:53:46] I'll let you know when I see petan around here [06:28:43] hello is there a problem with bastion, or the instances ? I cannot connect ot openid-wiki and openid2-wiki [06:45:25] [bz] (8NEW - created by: 2Chris McMahon, priority: 4High - 6enhancement) [Bug 53061] support Flow on beta cluster - https://bugzilla.wikimedia.org/show_bug.cgi?id=53061 [07:37:57] [bz] (8NEW - created by: 2Antoine "hashar" Musso, priority: 4Unprioritized - 6normal) [Bug 53457] setup a DB backed parser cache - https://bugzilla.wikimedia.org/show_bug.cgi?id=53457 [07:40:08] [bz] (8NEW - created by: 2Antoine "hashar" Musso, priority: 4Unprioritized - 6normal) [Bug 53458] adapt the MariaDB puppet manifests for beta - https://bugzilla.wikimedia.org/show_bug.cgi?id=53458 [07:40:09] [bz] (8NEW - created by: 2Antoine "hashar" Musso, priority: 4Unprioritized - 6normal) [Bug 53339] migrate beta databases to MariaDB - https://bugzilla.wikimedia.org/show_bug.cgi?id=53339 [12:34:09] Coren: What's up with job 744318? It's showing 0 CPU time and 0 memory usage, although it does seem to be running. And it's not the spaces-in-$0 problem this time (those are nbsp, heh). [12:34:25] anomie: Lemme see. [12:36:54] anomie: Would that be ArticleCreationGrapher? [12:37:26] Coren: No, ArticleCreationGrapher is 744309 [12:37:40] * Coren doesn't know how to tell those apart. :-) [12:38:02] Ah, the number (999) [12:38:28] * anomie stops typing out all the tasks since Coren found the number [12:39:18] hm. It's running fine indeed; I'm thinking the gridengine process that collects stats is easily confused because it's not picking those up [12:39:36] Perhaps the "[SourceUploader]" in the args? [12:40:07] Hasn't been a problem before though, as far as I know. But I'll see about testing it later. [12:41:28] anomie: As far as I can tell it really is just a collection problem; the process is running and happily chugging away at regular interval. [12:42:28] Coren: I see entries in the bot's logfile, so it is running (somewhere). But I see another oddity. By submitting jobs that do "ps auxwww", I see AnomieBOT processes that don't seem to be attached to jobs: pids 1128, 1278, and 1673 on exec-03 and pids 2082 and 2083 on exec-05. [12:42:53] Actually, nevermind that [12:43:13] * anomie modifies script to delete old logfiles [12:43:34] I don't see either 2082 or 2083 [12:43:41] Oh. :-) [12:44:44] FYI, in case you wanted the numbers, 999 has 4:26 accumulated CPU times and just under 203MB of Vmem [12:50:03] anyone know if there is an open elasticsearch index that can be queried? mentioned in the search roadmap https://www.mediawiki.org/wiki/Roadmap#Search [12:51:38] edsu: not deployed to production yet. [12:51:57] edsu: should be interesting to get that on toollabs when it is deployed [12:53:40] i guess it's running somewhere in labs already? [12:54:03] edsu: hmm, it was, and then NFS was being hammered by it, and... I don't know what they did after that [12:54:10] edsu: might be running in betalbs, though [12:56:06] looks like there some older solr images in the 'search' project https://wikitech.wikimedia.org/wiki/Nova_Resource:Search [12:58:21] edsu: ^d / ^demon and manybubbles are the ones who work on it, IIRC [13:00:45] YuviPanda: cheers [13:01:08] manybubbles: ping [13:12:09] :q [13:12:16] :D [13:17:41] manybubbles: following on from edsu's Elasticsearch questions, do you know if the new search API will allow more_like_this queries? [13:18:22] here! [13:19:10] on phone on moment [13:21:03] ok, off phone, gonna read chat log [13:22:16] edsu: the elasticsearch index isn't open to the internet but is open to our network. It exists in beta and for test2.wikipedia.org right now and will grow mediawiki.org today. [13:22:51] Coren: My job running on tools-exec-07 doesn't seem to be logging anything, when it should. Is NFS working right there? [13:24:13] fresco: the new search api will be mostly a clone of the old one. we wanted to get feature parity before diving into more, well, features. more_like_this is currently unimplemented but I'd be happy to implement something when the bugs quiet down. [13:24:15] anomie good question [13:24:47] manybubbles: this is the query i'm using with a local elasticsearch index of wikipedia: https://gist.github.com/hubgit/6365895 [13:24:57] it is working but for some reason there is long IO wait [13:25:35] it's possibly a bit more cpu-intensive than a standard query, but it'd be nice to be able to run it against the source index instead of mirroring [13:28:29] fresco: hmmm - I'll have to talk to ^d about it but we're really not at the point where we are willing to expose elasticsearch for arbitrary queries outside of folks with production cluster access. [13:29:01] fresco: we might be able to come up with something but I can't promise it'll be as soon as you'd like [13:29:03] manybubbles: that's fair enough - most of that query could be turned into parameters for a "morelikethis" query, though [13:31:29] fresco: so my suggestion is to file a bug against cirrussearch and I'll look at it. also, do you plan on doing this against enwiki or others? [13:31:44] just enwiki, personally - thanks, i'll do that [13:32:33] I promise I'll at least look at the bug and respond this week. Maybe by the time we get rolled out to enwiki I'll have something for you. [13:47:41] manybubbles: does elastic search do replication of some sort? [13:47:49] manybubbles: perhaps when it is all done we can replicate it to toollabs :) [13:47:56] YubiPanda: so many sorts [13:48:13] manybubbles: you can type Yu to complete my name [13:48:21] or just put a rate-limiting proxy in front [13:48:36] YuviPanda: yay tab! [13:48:41] :D [13:49:07] YuviPanda: snapping it back to toollabs on a scheduled basis is a good idea. [13:49:09] guess you'd need to guard against DELETE [13:49:23] yes, I meant also limiting to certain queries etc. [13:49:47] paravoid: as edsu said you'd need to guard against delete and other things. Also I bet you can poison it with nasty nasty queries pretty easilly. [13:50:08] readonly! [13:50:13] like how we have db replicas [13:50:25] essentially, if we wanted to expose it we'd have to be super careful [13:54:32] manybubbles: true, true. there was a good amount of work needed to expose the dbs [13:54:47] manybubbles: but I'm sure if we can expose the dbs, we can expose tehse too. eventually. at some point. ;) [13:54:56] *not* over the internet [13:55:03] but to toollabs, which is in-network [13:55:20] Yu: thus, why I don't think we'll have an answer as soon as we'd like. [14:03:30] manybubbles: sure, that's fine. [14:08:58] anomie: Nothing wrong with NFS that I can see. [14:10:03] anomie: What file did you expect its output to go to? [14:14:13] hey Coren :) [14:14:15]