[00:03:54] average, what's the likelihood of us upgrading hive any time soon? only subqueries aren't introduced until 0.13, if I'm reading right [00:03:59] (non-urgent, a nice-to-have) [00:07:38] Ironholds: I use sub-queries like.. right now.. literally (as we speak) [00:08:17] (and I've used them on an26 and they worked there as well) [00:08:41] okay, that's...strange. [00:08:51] apache claims they only exist from >=0.13 [00:09:05] oh, wait [00:09:07] * Ironholds slaps forehead [00:09:14] sorry, confusion of terms; it's not subqueries generally but placement [00:09:22] so, have you been using them in where statements or from statements? [00:09:36] in FROM statements [00:09:43] that explains it; sorry, I meant in WHERE statements - see https://cwiki.apache.org/confluence/display/Hive/LanguageManual+SubQueries [00:10:55] ah I see [00:11:44] but I suppose you can re-write your query and work around the problem [00:11:48] using some joins [00:11:58] indeed [00:12:01] or a temp table [00:12:06] yeah [00:12:13] except every time I try...anything, at the moment, I'm getting block retrieval errors. [00:13:41] hmm might be something related to hdfs. maybe e-mail ottomata with the query you ran and the error you got ? [00:15:10] yeah, I shall [00:15:26] everything I try leads to one of two outcomes; the query doesn't even start, or the query does start and then bugs out [00:16:29] Ironholds: I recommend that you set up a hive on your machine in local mode. And you play with a query until you're like really sure it's working well there. When you get past that state you can throw it on an26 (or stat1002) and try it in production [00:17:18] that's how I work right now, and things are working well [00:17:36] but when I get errors in local mode on my machine, I can fix them much faster (and I don't depend on HDFS) [00:18:04] sure, but you work from the perspective of an experienced developer who works on kraken itself, and I work from the perspective of a guy who only started programming a year ago and has a busted HDD ;p [00:18:35] I'll try to work on it when my new machine gets in; at the moment I'm just digging myself into a little frustrated hole. [00:37:34] I can't say I'm good with hive, usually SQL people are very good at it [00:37:49] I'll document this local hive setup [00:42:36] average, thanks; that'd be really helpful :D [00:42:44] a big chunk of it, I think, is not knowing where to start [00:52:39] halfak: yt? [01:24:27] Hey DarTar. [01:24:39] hey [01:24:45] nm, problem solved [01:26:24] Hokay. [03:03:06] Ironholds: I have a problem making some graphs [03:03:11] Ironholds: can you help me please ? [03:38:58] (CR) Edenhill: [C: 1] Initial debian version (1 comment) [analytics/kafkatee] (debian) - https://gerrit.wikimedia.org/r/110620 (owner: Ottomata) [14:56:44] (PS2) Milimetric: [Work in Progress] Implementing Scheduled Reports [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111623 [15:23:06] (CR) Ottomata: Initial debian version (1 comment) [analytics/kafkatee] (debian) - https://gerrit.wikimedia.org/r/110620 (owner: Ottomata) [17:39:49] average, still need help? [17:39:59] unrelated: so, stupidest hive thing of the day - I've managed to lock myself out of my own temp table. [18:38:20] nuria: is hash(IP+UA+salt) still on the agenda for anonymization/sanitization purposes ? [18:45:01] ottomata, any chance you're around? [18:45:09] yup [18:45:11] in a meeting but here [18:45:17] meeting will be over shortly i think [18:45:24] kk :) [18:45:31] what'sup? [18:46:26] so, I appear to have locked ironholds_temp so I can't write to it [18:46:37] quite how, I'm not sure (I swear I specified myself as the owner when I created it) [18:52:01] mysql? [18:52:14] tnegrin: yt? [18:52:23] Ironholds: ^^ [18:52:35] ottomata, hive ;p [18:52:39] hm [18:53:11] Ironholds: what database? [18:53:25] ah, in wmf [18:53:38] this isn't your problem, but you should avoid creating tables in the wmf database [18:53:45] you should create one under your username [18:53:47] 'ironholds' [18:53:48] gotcha [18:53:49] and, oops. [18:53:52] just to keep wmf clean [18:54:13] so what's the problem? [18:54:15] the error? [18:54:45] oh, it was "permission denied by sticky bit setting" [18:55:10] hm, that's a weird error [18:55:13] how'd you get it? [18:55:48] hey DarTar [18:56:26] ottomata, attempting to INSERT OVERWRITE onto ironholds_wmf [18:56:30] *ironholds_temp [18:56:34] yo tnegrin [18:56:39] in the office? [18:56:40] hey [18:56:44] yeah [18:56:48] you on 6? [18:56:49] I'm on 6, starting in 5 [18:56:49] hmmm [18:56:52] coming up [18:56:58] bring leila [18:57:34] Ironholds: I'm going to try to move the table into your db, s'ok? [18:57:44] ottomata, realistically, feel free to just junk it [18:58:03] it doesn't contain what I was wanting (hence the INSERT OVERWRITE); no idea how, but apparently I missed a LIMIT when filling it. [18:58:52] yeah hmmm [18:59:01] ok yeah i dotn' think i can move it [18:59:07] hmmm [18:59:13] ok i can drop it? [18:59:24] yup [18:59:25] feel free [19:01:06] ok done [19:01:12] they actually may have been your problem, i am not sure [19:01:15] lemme know if you have trouble in your db [19:01:17] dunno though [19:01:48] hey our last sprint for this quarter ends April 1st: https://wikimedia.mingle.thoughtworks.com/projects/analytics/cards/1438 [19:02:02] who wants to show off our great work with HyperKraken? [19:02:08] :P [19:04:34] HyperKraken!?!?! [19:04:43] whoa mingle gettin' slick [20:06:50] (PS3) Milimetric: Add scheduler mode to wikimetrics [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111623 [21:20:37] bargh [21:20:41] internet death! [22:45:02] (PS1) Milimetric: Allow reports to be rerun [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111914 [23:15:18] wb, tnegrin [23:15:23] you around in meatspace, or have ye gone home? [23:15:32] beavais [23:15:55] Aha. Lemme know when you're free for five minutes? And when those five minutes are convenient to you ;p [23:29:39] oh, hive [23:29:41] never change [23:41:57] any of our kraken engineers still conscious? ;p