[00:24:19] Ironholds: hey [00:24:25] you still around? [00:25:39] milimetric: yep. So, ah. Toby gave Dario and I access to the hadoop cluster, right? [00:25:43] but left out something important [00:25:50] :) [00:25:50] specifically, nobody has actually told us where it /lives/ ;p [00:25:58] you happen to know the hadoop bastion's hostname? :P [00:26:09] so the cluster is a bunch of machines [00:26:19] but if I'm guessing right, you're wanting to do some hive? [00:26:28] indeedy [00:26:41] I just ssh to analytics1011.eqiad.wmnet [00:26:47] thanks kindly! [00:26:51] that's a good place to start [00:26:55] that's the bit nobody told us [00:27:05] do you know how to mess around with hdfs / hive? [00:27:43] yup [00:27:47] or, I can learn ;p [00:28:02] k, ping us if you have questions, it can be frustrating [00:28:06] the basics [00:28:08] shaall do [00:28:17] hdfs dfs gives you basic access to the hadoop file system [00:28:24] hdfs dfs -ls [00:28:55] and hive works pretty similarly to mysql [00:28:59] yup [00:29:28] here's a script that imports one hour of webstatscollector data into a hive table and makes the appropriate partition for it: [00:29:39] https://gist.github.com/milimetric/6830990 [00:29:46] k, shutting up now [20:56:57] milimetric: so you store your key on bastion/s1/whatever, for hive access? is that not frowned upon, or am I misremembering? [20:57:37] I try very hard to not get frowned upon [20:57:44] and simultaneously think ssh is magic [20:58:11] so between those truths lies the fact that I have never stored any key anywhere and relied on others to do such things [20:58:38] I assumed said others to be benevolent, but maybe I was wrong [21:01:22] aha. Okay, I'm now very confused; I'll track someone down. Thanks :) [21:02:30] I'm asking ottomata in -analytics Ironholds, you're welcome [21:02:52] *welcome to join and see if he answers (I believe he's the one that set me up if it wasn't dschoon) [21:04:14] milimetric: danke!