[00:59:59] <ottomata>	 phew
[01:00:07] <ottomata>	 i'm feeling better about the git submodule stuff!
[01:00:09] <ottomata>	 so far so good!
[01:00:17] <ottomata>	 and i've had a quick runthrough with faidon already about what i'm doing
[01:00:19] <ottomata>	 so that's good news
[01:00:21] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/50452/
[01:30:29] <dschoon>	 fyi, restarting all the limns on kripke
[01:30:38] <erosen>	 10-4
[01:35:10] <dschoon>	 !log Corrected supervisor runas user for all Limn instances on Kirpke to match what's listed in the deploy script. They were subsequently restarted. Effects gerrit-stats, gp, test-reportcard, dev-reportcard.
[01:35:16] <dschoon>	 ...
[01:35:17] <dschoon>	 boo!
[01:35:25] <dschoon>	 the quote?
[01:36:46] <dschoon>	 !log Corrected supervisor runas user for all Limn instances on Kirpke to match the deploy script. They were subsequently restarted. Effects gerrit-stats, gp, test-reportcard, dev-reportcard.
[01:36:51] <dschoon>	 log the goddamn message!
[01:36:56] <dschoon>	 wm-bot wtf
[01:37:05] <dschoon>	 ...i dont' think it will listen to me :(
[01:37:10] <dschoon>	 ottomata, drdee :(
[04:26:44] <travis-ci>	 [travis-ci] master/18229bc (#57 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5092919
[14:31:17] <travis-ci>	 [travis-ci] master/e66df9d (#58 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5104471
[17:10:33] <erosen>	 dschoon: you around?
[17:10:50] <drdee>	 i am
[17:11:04] <erosen>	 curious if you know of a stable limn version to run locally
[17:11:30] <erosen>	 i am on master and i'm getting a "Uncaught TypeError: Cannot read property 'length' of undefined" error
[17:12:46] <dschoon>	 not yet.
[17:12:52] <erosen>	 k
[17:12:57] <dschoon>	 i'll be around momentarily.
[17:13:05] <dschoon>	 surely there are many stable versions :)
[17:13:16] <erosen>	 hehe, just a matter of finding them
[17:13:17] <dschoon>	 we will unearth them soon. for now, i recommend lolcats
[17:13:22] <dschoon>	 or really http://femalesoftwareeng.tumblr.com/
[17:13:23] <erosen>	 i'm working from home today, though fyi
[17:13:29] <dschoon>	 k
[17:13:33] <dschoon>	 i am not yet wearing pants
[17:13:36] <kraigparkinson>	 good morning, everyone
[17:13:38] <erosen>	 hehe
[17:13:41] <dschoon>	 critical issues must be attended to
[17:26:11] <ottomata>	 hiyaaaa kraigparkinson!
[17:26:21] <ottomata>	 if i want to add todos or things i'm working on
[17:26:24] <kraigparkinson>	 heeey, ottomata!
[17:26:26] <ottomata>	 should I do this in mingle?
[17:26:48] <ottomata>	 gah adium keeps crashing
[17:26:50] <kraigparkinson>	 he asks then kleaves before he gets an answer…
[17:26:55] <kraigparkinson>	 or leaves.
[17:26:55] <ottomata>	 ha
[17:27:06] <drdee>	 he doesn't wanna know the answer :D
[17:27:07] <ottomata>	 i read that in a jewish accent, thought it was a slang
[17:27:26] <kraigparkinson>	 good q, where do you track this now? asana?
[17:27:29] <ottomata>	 ja
[17:27:33] <kraigparkinson>	 taste the soup!
[17:27:39] <ottomata>	 hah, ja
[17:27:49] <ottomata>	 i finsihed something yesterday, wanted to update it and adda  bunch of new things I nee to work on
[17:28:17] <kraigparkinson>	 hm… I'm thinking before we go all-in on mingle, we should chat about it as a team first...
[17:28:36] <ottomata>	 ok
[17:28:40] <ottomata>	 i'll keep doing asana for now then
[17:28:46] <kraigparkinson>	 If you see an opportunity to improve the way you track the work, and that's made easier through mingle, then we should add an improvement card....
[17:29:47] <kraigparkinson>	 for example, if there's a workflow that you regularly go through, or think it would be good for us to be able to visualize where there are blockers, say with ops, then mingle might make that easier to identify.
[17:30:40] <kraigparkinson>	 that is, other parts/members of ops. :)
[17:33:25] <ottomata>	 hmmmm, not sure, there are times I have todos that just get permanently blocked, that I usually just end up deleteing
[17:33:38] <ottomata>	 buuuut, right now i just want to track tasks I complete, and maintain a todo list
[17:33:44] <ottomata>	 so I don't forget about things I need to work on
[17:33:46] <ottomata>	 that's about it
[17:34:38] <kraigparkinson>	  ok, well, go for asana.  would be good to discuss what the flow is of getting infrastructure configured, setup, puppetized, approved, deployed, etc…
[17:35:16] <kraigparkinson>	 maybe we can sketch that flow quickly on paper in 10 minutes and see where that takes us
[17:35:30] <ottomata>	 hm, ok
[17:35:47] <kraigparkinson>	 and it would be great to learn what kind of stuff that gets perma-blocked.
[17:36:22] <ottomata>	 well, perma blocked isn't always bad
[17:36:32] <kraigparkinson>	 not assuming it is.
[17:36:44] <ottomata>	 for example, recently there was a desire to make the slow-parse (lgos about which articles take a long time to render) public
[17:37:14] <ottomata>	 dieds asked me to do it, I created a patch, then we got input from some people that said making it public was a security concern
[17:37:16] <ottomata>	 so we dropped it
[17:37:23] <ottomata>	 i just deleted that todo from my list
[17:37:33] <ottomata>	 which means the work that I did for it isn't tracked at all, since it never got complete
[17:37:45] <ottomata>	 the patchset is in gerrit, but I guess i'll just abandon it there
[17:39:17] <kraigparkinson>	 so, if we had a workflow around it, I think it would have revealed a few things...
[17:41:57] <kraigparkinson>	 because it's work you invested in that got stopped midway, it would be good to recognize that.  For example, it does seem to be a missed opportunity to design/validate the requirement before starting to implement it.  By putting it into a workflow in Mingle, we could introduce analysis/validation as a step before implementation…  helps see the hand-off between you and others.
[17:42:30] <ottomata>	 hmmm, aye
[17:42:31] <ottomata>	 makes sense
[17:42:33] <kraigparkinson>	 not like it's anyone's fault if the process wasn't setup for that..
[17:42:36] <ottomata>	 aye sure
[17:42:58] <ottomata>	 the work wasn't hard to do, but it did take an hourish, so I guess it would have been better to have it flushed out ahead of time
[17:43:02] <ottomata>	 but cool
[17:43:06] <ottomata>	 heya drdee!
[17:43:16] <ottomata>	 i'm trying to run the device geocode job again
[17:43:19] <ottomata>	 with your  new .jars
[17:43:23] <ottomata>	 still getting
[17:43:24] <ottomata>	 java.lang.RuntimeException: could not instantiate 'org.wikimedia.analytics.kraken.pig.GeoIpLookup' with arguments '[countryCode, GeoIPCity]'
[17:43:30] <ottomata>	 not sure how to troubleshoot….:/
[17:43:48] <drdee>	 which script and which machine?
[17:44:04] <ottomata>	 webrequest_hour_country_device_vendor.pig
[17:44:25] <ottomata>	 uhh, i'm starting an oozie coordinator from an02
[17:44:42] <ottomata>	 logs
[17:44:42] <ottomata>	 sudo -u hdfs hadoop fs -cat /var/log/hadoop-yarn/apps/stats/logs/application_1361846711130_0659/*
[17:45:13] <ottomata>	 i'm running this as the stats user
[17:45:21] <ottomata>	 I should ahve the .dat files in /user/stats/, right?
[17:45:36] <drdee>	 yes
[17:45:40] <ottomata>	 k, got that
[17:45:51] <ottomata>	 you put your latest .jars in /libs, right?
[17:45:56] <dschoon>	 pretty: http://hyperphor.com/election/scatterstates.html
[17:46:14] <dschoon>	 pretty: http://hyperphor.com/election/scatterstates.html
[17:46:35] <drdee>	 yes
[17:48:49] <ottomata>	 hm
[17:49:20] <drdee>	 we need the job server logs to help debug this
[17:53:30] <ottomata>	 i'm trying to run the pig script outside of oozie to see how this goes
[17:58:05] <drdee>	 ok
[18:01:50] <ottomata>	 hm, ok it ran when I had .jars and .dats in my cwd via pig manually
[18:01:51] <ottomata>	 hm
[18:01:52] <ottomata>	 oh
[18:01:53] <ottomata>	 is this relevant?
[18:01:56] <ottomata>	 Caused by: java.lang.NoSuchMethodException: org.wikimedia.analytics.kraken.pig.GeoIpLookup.<init>([Ljava.lang.String;)
[18:01:58] <ottomata>	 drdee^
[18:02:35] <drdee>	 let's first do scrum
[18:03:07] <drdee>	 how do you initialize the function?
[18:03:21] <ottomata>	 oh scrum ja, i'm in an ops meeting but i'll duck out
[18:03:27] <ottomata>	 be right there
[18:59:27] <drdee>	 https://raw.github.com/wikimedia/metrics/master/pageviews/webstatscollector/Pageview_definition.png
[19:00:18] <kraigparkinson>	 http://www.thoughtworks-studios.com/docs/mingle/12.4/help/formatting_help_component.html
[19:00:48] <ottomata>	 hmmmmmmmmmmmmmmmmmmmMMMmmm drdee
[19:00:55] <ottomata>	             ip4Lookup = new LookupService("./GeoIPCity.dat");
[19:00:58] <ottomata>	 you sure you want ./ here?
[19:01:09] <ottomata>	 maybe no path qualifier at all would be better?
[19:02:00] <drdee>	 that means  relative from /user/stats
[19:02:21] <drdee>	 all implementations that i have seen on the web have taht
[19:02:25] <drdee>	 but i can rip it out np
[19:03:59] <ottomata>	 hm i'm not sure, i'm just guess here
[19:04:05] <ottomata>	 i don't know why it doesn't work in oozie yet
[19:04:51] <ottomata>	 i think without ./ would mean the equivalent, but Oozie does something with libpath that should make files available, although i'm not sure if that works for anything but jars and pig scripts
[19:05:05] <drdee>	 ok let me rip it out
[19:05:06] <drdee>	  
[19:05:07] <drdee>	 1
[19:05:09] <drdee>	  sec
[19:05:56] <ottomata>	 k
[19:05:58] <ottomata>	 worth a try
[19:10:21] <drdee>	 try again
[19:13:54] <ottomata>	 uploaded to /libs?
[19:13:57] <ottomata>	 drdee?^
[19:14:05] <drdee>	 yes
[19:14:14] <ottomata>	 ok, going to try with pig cli first, then oozie
[19:16:04] <drdee>	 ok
[19:23:55] <ottomata>	 hm, works in pig, still not in oozie
[19:24:49] <ottomata>	 same error
[19:27:59] <ottomata>	 i'm thinking this is not an error with not being able to find .dat files
[19:28:14] <ottomata>	 Caused by: java.lang.RuntimeException: could not instantiate 'org.wikimedia.analytics.kraken.pig.GeoIpLookup' with arguments '[countryCode, GeoIPCity]'
[19:28:15] <ottomata>	 …
[19:28:15] <ottomata>	 Caused by: java.lang.NoSuchMethodException: org.wikimedia.analytics.kraken.pig.GeoIpLookup.<init>([Ljava.lang.String;)
[19:28:38] <ottomata>	 maybe it can't find the .jar?  or maybe the .jar that it is finding is the wrong one....
[19:31:00] <drdee>	 try putting the jars in /user/stats/ as well
[19:31:18] <ottomata>	 hmmmmwellllll, the logs show me the libpath oozie is using
[19:32:07] <ottomata>	 https://gist.github.com/ottomata/5050904
[19:32:24] <drdee>	 ok, let me get back to you in 30 minutes
[19:32:25] <ottomata>	 i'm going to remove kraken.jar and kraken_bak.jar
[19:32:30] <ottomata>	 those shouldn't be registered, but just in case
[19:32:32] <ottomata>	 s'ok?
[19:32:41] <ottomata>	 ah poo
[19:32:47] <ottomata>	 my webrequest_loss job is stil using kraken.jar
[19:32:49] <ottomata>	 don't want to remove it yet
[19:33:07] <dschoon>	 classpath conflicts are nasty
[19:34:27] <ottomata>	 HMMMMM
[19:34:38] <drdee>	 ok
[19:34:39] <ottomata>	 right because it is in place too
[19:34:43] <ottomata>	 yeah that might be it
[19:34:43] <ottomata>	 hmmm
[19:34:52] <ottomata>	 ok, let me see if I can make webrequest loss use the snapshot
[19:34:58] <ottomata>	 should be easy
[19:35:01] <ottomata>	 i betcha that is the problem
[19:38:52] <ottomata>	 fingers crossed, but actually i dunno, since the pig script specifically registers jars
[19:38:57] <ottomata>	 nope, it failed :/
[19:42:05] <travis-ci>	 [travis-ci] master/73482cd (#59 by Andrew Otto): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5113056
[19:45:02] <ottomata>	 travis-ci, you know that I ignore you, right? :)
[21:06:24] <kraigparkinson>	 https://mingle.corp.wikimedia.org/projects/analytics/wiki/Engaging_in_Continuous_Improvement
[21:08:24] <kraigparkinson>	 https://mingle.corp.wikimedia.org/projects/analytics/cards/2
[21:14:11] <dschoon>	 http://analytics-pad.wmflabs.org/p/Weekly_Reflection_02-27
[21:14:32] <dschoon>	 kraigparkinson, drdee, ottomata
[21:15:57] <dschoon>	 kraigparkinson, could you paste this into the hangout? http://analytics-pad.wmflabs.org/p/Weekly_Reflection_02-27
[21:18:36] <dschoon>	 http://etherpad.wmflabs.org/pad/p/AnalyticsWeeklyReflection_02-27
[21:18:39] <dschoon>	 switch to that
[21:18:46] <dschoon>	 ^^ kraigparkinson ottomata drdee
[21:20:51] <kraigparkinson>	 https://mingle.corp.wikimedia.org/projects/analytics/cards?favorite_id=707&view=%3EInbox+-+Improvement+Ideas
[22:27:14] <kraigparkinson>	 thanks again everyone for participating in our weekly improvement session!
[22:51:33] <kraigparkinson>	 drdee, you there?
[22:51:42] <drdee>	 yes
[23:31:50] <dschoon>	 drdee, kraigparkinson: https://docs.google.com/document/d/1ucIrwztXXA2Ps24MIb2hQiaFE0kpb4eNNijMbrrfbqw/edit?usp=sharing
[23:32:36] <kraigparkinson>	 is that the Cliff Notes to the wiki page? :)
[23:54:42] <travis-ci>	 [travis-ci] master/a5a04b8 (#60 by Andrew Otto): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5119725