[12:10:41] (CR) QChris: [C: -1] "(29 comments)" [analytics/kraken] - https://gerrit.wikimedia.org/r/75349 (owner: Stefan.petrea) [14:33:47] hiya qchris! [14:33:58] Hi ottomata [14:34:04] any idea why I'd see this when I try to mvn package on my labs instance? [14:34:05] [ERROR] /home/otto/camus-ottomata/camus-api/src/main/java/com/linkedin/camus/schemaregistry/SchemaRegistry.java:[16,31] generics are not supported in -source 1.3 [14:34:12] [ERROR] (use -source 5 or higher to enable generics) [14:34:12] [ERROR] public interface SchemaRegistry { [14:34:26] Java 1.3 does not support generics :-/ [14:34:37] The switch [14:34:43] -source 1.3 [14:34:46] requests Java 1.3 [14:35:25] Does your code rely on using Java 1.3? [14:35:34] don't think so [14:35:36] but i'm running 1.6? [14:35:48] ava -version [14:35:48] java version "1.6.0_27" [14:35:48] OpenJDK Runtime Environment (IcedTea6 1.12.6) (6b27-1.12.6-1ubuntu0.12.04.2) [14:35:48] OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode) [14:36:04] Yes, but you can use 1.6 to compile for 1.3. [14:36:09] hm [14:36:22] i don't see it asking for -source 1.3 anywhere [14:36:24] See man javac [14:36:25] where would I look? [14:36:35] In the place where you invoke javac [14:36:53] maven does that... [14:37:00] just a grep of the codebase for '1.3' returns nothing [14:37:29] "javac -version" Does return something >1.3 right? [14:38:13] javac -version [14:38:13] javac 1.6.0_32 [14:38:17] You could grep the maven files for something like or [14:38:32] nothin [14:38:59] interesting :-) [14:39:18] Can I get acces to the machine? [14:39:30] Or is the setup complicated? [14:40:02] i think you shoudl just be able to [14:40:09] kraken-namenode-standby.pmtpa.wmflabs [14:40:13] if you have labs access [14:40:15] then ja [14:40:17] i think you can get in [14:40:23] Let me try that. [14:40:26] /home/otto/camus-ottomata [14:40:27] mvn package [14:42:18] I cannot access /home/otto [14:42:31] Can you grant me acces or copy it ower in my home directory? [14:44:18] ottomata: ^ [14:44:23] psshh [14:44:38] k try now [14:46:48] Ok I can access it. Trying to build it. [14:49:56] Is that code fresh code from us or is it from some upstream? [14:50:06] the maven-compiler-plugin seems unconfigured. [14:50:24] ottomata: May I try mangling files? [14:50:52] please [14:50:55] it is pretty fresh [14:51:07] only thing i changed were offset error we had yesterday [14:51:13] and the base pom.xml (added camus-wmf as a target) [14:51:24] ottomata: Or does that destroy anything? [14:51:31] ? [14:51:37] no go right ahead [14:51:39] mangle away [14:51:43] i just cloned it a few minutes ago [14:51:59] ok. Great. [14:55:00] As I lack permission, could you try bringing /home/qchris/pom.xml to /home/otto/camus-ottomata/pom.xml? [14:55:29] (Or grant me permission to destroy that file :-) ) [14:55:29] ottomata: ^ [14:55:38] pssh, sure [14:55:44] i'll just copy the whole dir to your home [14:55:45] oh [14:55:46] you shoudl ahve sudo [14:55:47] btw [14:55:48] on labs [14:55:58] * qchris facepalms myself. [14:56:01] Sorry. I'll do that [14:56:04] copying to your homedir [14:56:12] you can sudo chown -R it :) [14:56:26] ah poop [14:56:28] it copied the contents [14:56:29] sigh [14:58:26] ok there, it is in your home now, you shoudl sudo chown -R qchris ~/camus-ottomata [14:58:32] Thanks. [15:03:36] ottomata: Using /home/qchris/camus-ottomata/pom.xml [15:03:40] ottomata: It compiles. [15:03:52] Just copy it over to your directory [15:04:00] (and fix spacing) [15:04:06] And mvn compile should go through [15:04:20] just the pom? whatdid you change? [15:04:34] I told maven to compile for java 1.6 in the pom [15:04:50] Diffing the old an new version gives a really short diff. [15:04:52] hmm, strange [15:04:53] ok cool! [15:04:54] i see it [15:05:16] I am not sure about the encoding part. [15:05:24] It's not strictly needed, but it's typically used. [15:05:39] If it bothers you, you should be safe to just remove that tag. [15:05:53] danke! [15:05:58] Bitte, gern :-) [15:06:00] ok cool [15:06:41] ottomata: I'll be away soonish to do the sprint writeup. Are you currently blocked on other Java problems? [15:06:59] not at the moment, im' kinda trying out intellij [15:07:13] You're using the dark side of the force :-D [15:07:27] Hope you like it nonetheless. [17:03:31] standup? [17:03:35] oh nobody probably, eh? [17:03:50] tnegrin: we already talked, nobody else is here i guess, eh? [17:06:08] no one else [17:07:05] hokay, i'm going to grab lunch then [17:08:12] kk [20:35:30] ottomatta -- can you point me at the hdfs health page for kraken? [20:37:55] yeah, but you need to do the ssh tunnel thing [20:37:58] because it doesn't have a public IP [20:38:03] you wanna get that working now? [20:38:09] (shouldn't be too hard) [20:41:49] tnegrin: ^ [20:42:01] yes -- let's do it! [20:42:57] ok so [20:43:04] clone kraken repo, if you haven't already [20:43:08] there's a script in there that will help you [20:43:13] https://github.com/wikimedia/kraken [20:43:36] also, make sure you can log into analytics1001.wikimedia.org [20:43:41] ssh tnegrin@analytics1001.wikimedia.org [20:44:19] ok in [20:44:46] great [20:44:57] should I d/l the mac git client thing? [20:45:03] meh? i use CLI [20:45:05] but up to you [20:45:35] oo, also, you'll need docopt, my favorite python cli parser [20:45:40] hmm, do you have pip? [20:46:00] I a (l)user [20:46:02] I have nothing [20:46:07] ok [20:46:15] ooook [20:46:20] This is perl 5, version 12, subversion 4 [20:46:37] hehe [20:46:49] uhhhhhhh ok, do you just want an ssh command to run to get you to the namenode web gui right now then? [20:46:50] why do I need all this stuff to hit a web page? [20:46:52] hahaha [20:46:56] yes [20:46:58] let's do that [20:46:59] that is what I got in big trouble for back in january [20:47:05] i tried to make this easier [20:47:06] oic [20:47:16] because these services don't have public IPs [20:47:24] so you need to get into the backend network somehow [20:47:30] normally, we'd just have a vpn [20:47:32] mmm -- we don't have vpn access? [20:47:34] right... [20:47:34] but we don't [20:47:37] really? [20:47:39] nope, i set a very minimal one up [20:47:43] for the analytics cluster [20:47:45] :) [20:47:47] and that's what I got in trouble for [20:47:50] I think heard about that [20:47:57] yup, big fiasco [20:48:04] indeed [20:48:08] so yeah um [20:48:09] am I allowed to tunnel in? [20:48:21] ok [20:48:22] yes [20:48:23] run this [20:48:29] ssh -N tnegrin@analytics1001.wikimedia.org -L 50070:analytics1010.eqiad.wmnet:50070 [20:48:57] and then go to http://localhost:50070/ in your browser [20:49:30] where from? my terminal or analytics1010? [20:49:44] your terminal [20:49:48] on your local machine [20:49:48] ok [20:50:07] a vpn is pretty handy [20:50:12] yup [20:50:15] would be great [20:50:21] i will not start a discussion about that though :p [20:50:44] cool -- speaking with the nn [20:50:48] great! [20:51:07] yes -- thanks Andrew [20:51:18] so -- job tracker? what is this in yarn? resource manager? [20:52:31] this cluster -- it's not so big [20:52:40] we need more nodes!!!! [20:56:21] yup [20:56:40] job tracker == resource manager (+ history server) [20:56:40] task tracker == node manager [20:57:48] ok -- I have the docs; planning on reading over the break [20:58:08] cool [20:58:19] what's the ip for the RM? I'd like to look at the history [20:58:28] you want two things i think [20:58:31] then I will trouble you no more [20:58:42] ssh -N analytics1001.wikimedia.org -L 8088:analytics1010.eqiad.wmnet:8088 [20:58:55] ssh -N analytics1001.wikimedia.org -L 19888:analytics1010.eqiad.wmnet:19888 [20:58:57] ahh i will write in email [20:58:58] to you [20:59:21] awesome -- thanks [21:01:37] sent. [21:01:43] ok -- this is a little clumsy but very interesting [21:02:49] yeah [21:02:57] the ktunnel script makes it easier [21:03:05] its just a wrapper for those commands though [21:03:20] it doesn't work when you want to click on links between services though [21:03:25] exactly [21:03:29] because hadoop doesn't know it is being tunneled [21:03:46] can I wildcard the internal network? [21:04:44] to make some of that easier [21:04:50] what I have done is edited my /etc/hosts file [21:05:05] and made the hostnames that the links have resolve to localhost [21:05:16] we used to have an haproxy setup to make this easier too [21:05:23] i got in trouble for that too [21:06:46] wow -- bc this is a nightmare [21:06:56] I mean -- not a nightmare, but hard to work with [21:07:08] yup [21:07:16] yes -- I remember it now -- people said you put in a backdoor [21:07:19] not exactly fair [21:08:01] let's give leslie some task involving the web tools. she can ask for a vpn [21:08:12] yeah, i had discussed the vpn thing with her before I set it up [21:08:16] she wasn't opposed [21:08:18] oh really [21:08:26] yeah, i didn't tell her I was going to set it up [21:08:27] who was pissed? erik? [21:11:55] erik was cool