[10:56:59] <drdee>	 average are you around?
[13:25:14] <average>	 hi drdee
[13:25:36] <average>	 drdee: saw your e-mail, I'm working on fixing that, but I first have to wrap up documenting the package building workflow
[13:25:42] <average>	 which I'm doing now
[13:26:05] <drdee>	 k
[13:30:08] <ottomata>	 whoaaa, drdee alex k, has a kafka .deb!
[13:30:15] <ottomata>	 i have some things to do on that now
[13:30:23] <ottomata>	 should I work on that or  on the hadoop node reinstall?
[13:31:22] <drdee>	 yEs !!!!
[13:33:09] <ottomata>	 which one?  i was in the middle of hadoop node resintall puppet stuff on friday :)
[13:36:37] <ottomata>	 drdee?
[13:36:50] <drdee>	 1 sec
[13:37:09] <ottomata>	 (I think hadoop thing, it is more doable in the short term :)
[13:46:02] <milimetric>	 hey everyone
[13:46:32] <milimetric>	 compy problems - waking up the laptop from sleep is like a 20 minute ordeal, shutdown/startup is 15 seconds.  I'm beginning to think sleep isn't worth it :)
[13:46:49] <drdee>	 morning milimetric
[13:47:05] <milimetric>	 mornin
[13:47:07] <drdee>	 ottomata: i would say help with kafka
[13:47:14] <drdee>	 and let's finish that first
[13:53:27] <ottomata>	 welll, it sounds like alex is actually still working on it
[13:53:31] <ottomata>	 i'll let him do his thing for now
[13:58:39] <average>	 ottomata: is it ok to assume that upstream tag versions have no correlation with debian tag versions ?
[14:01:21] <ottomata>	 bwer?  (good that you are asking faidon :) )
[14:01:46] <average>	 ottomata: ok :)
[14:12:25] <ottomata>	 drdee!
[14:12:26] <ottomata>	 https://blog.cloudera.com/blog/2013/05/cdh-4-3-is-released/
[14:12:49] <drdee>	 i know
[14:12:55] <drdee>	 i forgot to forward it  to you
[14:47:35] <ottomata>	 drdee, so, alex put hive,pig,sqoop, etc. in our apt, but since he just did it he did it at 4.3, which put the other hadoop stuff in at 4.3
[14:47:44] <ottomata>	 which means when I reinstall a hadoop node, it will be on 4.3 and others will be on 4.2.1
[14:47:49] <ottomata>	 which is probably fine, but to avoid potential issues
[14:47:53] <ottomata>	 i'd rather just upgrade everything now
[14:47:54] <ottomata>	 thoughts?
[14:48:37] <drdee>	 let's not mix versions; that caused issues the last time as well
[14:48:58] <drdee>	 can't we just stick at 4.2.1 ?
[14:50:56] <ottomata>	 i'll ask
[14:51:44] <ottomata>	 ergh, hm, it looks like our apt only has 4.3.0 .debs right now
[15:08:58] <ottomata>	 drdee, ^  i asked alex but he's away now.
[15:21:51] <gerrit-wm>	 New patchset: Erik Zachte; "more specific test for harvesting redirect localization from php message file" [analytics/wikistats] (master) - https://gerrit.wikimedia.org/r/66549
[15:24:55] <gerrit-wm>	 Change merged: Erik Zachte; [analytics/wikistats] (master) - https://gerrit.wikimedia.org/r/66549
[15:42:39] <drdee>	 milimetric; can you help me? i am blind
[15:46:08] <drdee>	 milimetric ^^
[15:46:14] <milimetric>	 sure
[15:46:27] <milimetric>	 I'm no ophthalmologist
[15:46:32] <milimetric>	 but what's up
[15:46:57] <milimetric>	 ugh, why the heck did this thing not beep on your first message but beeped on your second
[15:47:03] <drdee>	 exactly i need a ophthalmologist
[15:47:15] <drdee>	 a source-code ophthalmologist
[15:47:18] <milimetric>	 :)
[15:47:26] <drdee>	 i have some unit-tests failing (java)
[15:47:30] <milimetric>	 k, kraken?
[15:47:31] <drdee>	 can't figure out why
[15:47:33] <drdee>	 yup
[15:47:38] <milimetric>	 pushed?
[15:47:41] <drdee>	 1 sec
[15:48:00] <ottomata>	 (lunchtime, back in a bit!
[15:48:01] <ottomata>	 )
[15:50:12] <drdee>	 ok, pushed
[15:50:33] <milimetric>	 running
[15:51:12] <milimetric>	 ok so these 10 tests fail starting with testSampleMobileLogLine
[15:51:23] <drdee>	 yes
[15:51:36] <drdee>	 i think there are two issues at stake
[15:54:06] <drdee>	 i think one culprit is refersToSameArticle() but not sure why (in PageviewFilter.java)
[16:12:33] <milimetric>	 drdee: isn't this backwards: !pageviewFilter.refersToSameArticle(pageviewType, url, pageviewTypeReferer, referer)
[16:12:48] <milimetric>	 shouldn't that just be pageviewFilter.refersToSameArticle(pageviewType, url, pageviewTypeReferer, referer) without the "not"
[16:12:58] <milimetric>	 219 of Pageview.java
[16:14:14] <milimetric>	 oh, misread
[16:14:20] <drdee>	 i don't think so. if the referer and the url refer to the same article then that request should not be counted as a pageview
[16:14:24] <milimetric>	 "same" not "some"
[16:14:54] <milimetric>	 i need a real ophthalmologist
[16:15:20] <drdee>	 :D
[16:16:27] <drdee>	 now only 9 failing :)
[16:17:17] <milimetric>	 well, so it's just that method that's wrong (isWikistatsMobileReportPageview)
[16:17:39] <milimetric>	 it should return false if it's a mobile search for the test to work
[16:18:06] <drdee>	 8 failing
[16:19:47] <drdee>	 ok
[16:23:09] <milimetric>	 testNullRequestMethod fails predictably because isPageView requires a request method ;)
[16:24:03] <milimetric>	 drdee, line 231 PageviewFilter.java ^^
[16:24:03] <drdee>	 k
[16:24:09] <drdee>	 fixing :)
[16:24:16] <drdee>	 i will push some fixes soon
[16:24:19] <milimetric>	 k
[16:24:24] <milimetric>	 i'll look at the rest after
[16:24:34] <erosen>	 drdee: have you found anything promising that could account for the low numbers?
[16:24:41] <drdee>	 not yet
[16:25:10] <erosen>	 i turned off the check in the pig script and got roughly twice the counts, which is a bit worrisome
[16:29:57] <milimetric>	 is this where we're trying to find counts that are 10x bigger erosen?
[16:30:41] <milimetric>	 are you counting in general where the x-cs is set or looking at a specific partner?
[16:32:11] <erosen>	 milimetric: by summing the individual partners
[17:00:43] <drdee>	 erosen: scrum
[17:08:29] <ottomata>	 average: which branch?
[17:08:41] <ottomata>	 on the dclass repo?
[17:23:27] <ottomata>	 btw, drdee, erosen, milimetric:  pics!
[17:23:27] <ottomata>	 http://www.flickr.com/photos/ottomatona/sets/72157633905309558/
[17:23:37] <drdee>	 ohoho
[17:23:46] <milimetric>	 cool :)
[17:23:51] <erosen>	 awesome
[17:24:10] <drdee>	 ncie!
[17:26:34] <average>	 ottomata: newpackage
[17:26:46] <average>	 ottomata: steps to build so far are in DEBIAN.md
[17:26:56] <average>	 ottomata: please have a look
[17:27:20] <average>	 ottomata: is there a way I could get a link to the repo ?
[17:27:34] <average>	 ottomata: wanted to show it in the scrum but didn't manage to get a link
[17:28:11] <ottomata>	 ja
[17:28:21] <ottomata>	 go here
[17:28:21] <ottomata>	 https://gerrit.wikimedia.org/r/#/admin/projects/
[17:28:26] <ottomata>	 click on (gitweb)
[17:28:27] <average>	 aaah, thanks :)
[17:28:28] <ottomata>	  on the side
[17:29:52] <average>	 ottomata: you will notice the following switches in git-dch    -D --force-distribution
[17:30:03] <average>	 ottomata: these are only present for newer versions of git-buildpackage
[17:30:28] <average>	 ottomata: you may get a version which has those features here http://garage-coding.com/releases/git-buildpackage/git-buildpackage_0.6.0~git20130414_all.deb
[17:31:22] <average>	 ottomata: I've built it from the official repo (located here git://anonscm.debian.org/users/agx/git-buildpackage.git )
[17:33:15] <erosen>	 drdee: I got the pageview class working in a scala interpreter which is pretty nice for debuggin, in case you are interested
[17:50:44] <average>	 drdee: Andrew and Faidon reviewed it. There are multiple things that need to happen. A gerrit patchset will follow fixing the problems.
[17:50:51] <average>	 In the meantime I'm on #738
[17:51:34] <average>	 (I'll work on a patchset for #716 after I close #738)
[18:35:38] <drdee>	 average: awesome!
[18:35:44] <drdee>	 thanks for all your hardwork
[18:35:49] <drdee>	 much appreciated!!
[18:42:17] <erosen>	 ottomata milimetric drdee : I figured out how to use the scala interpreter for testing java UDF logic.  I haven't gotten it to work for any kraken java code (CLASSPATH issues), but it works for kraken-generic at least: https://gist.github.com/embr/11dc7af47d6356e48afe
[18:42:36] <drdee>	 thanks erosen! very cool!!
[18:42:43] <erosen>	 np
[18:42:46] <erosen>	 i love interpreters
[18:42:49] <ottomata>	 cool!
[18:43:04] <milimetric>	 ah very nice
[18:43:08] <milimetric>	 tab complete yay
[18:43:23] <erosen>	 yeah, was pretty excited to discover that
[19:35:17] <average>	 drdee: thanks
[19:35:26] <average>	 bb 1h
[19:35:38] <drdee>	 k
[19:40:34] <ottomata>	 drdee, i'm glancing at our hadoop confs, checking htings out
[19:40:44] <ottomata>	 by default we do not have mapreduce output compression turned on
[19:40:45] <drdee>	 k
[19:40:56] <ottomata>	 shoudl we turn that on?
[19:41:36] <drdee>	 maybe :)
[19:42:03] <ottomata>	 intermediate compression is on
[19:42:13] <drdee>	 that's good
[20:31:14] <erosen_>	 drdee: milimetric rounce123: have time to meet with dan?
[20:31:21] <milimetric>	 i'm in the hangout
[20:31:25] <rounce123>	 yes
[20:31:27] <rounce123>	 coming now
[22:04:21] <milimetric>	 erosen: how can I help?
[22:29:41] <erosen>	 milimetric: hey, was talking to liAnna
[22:32:56] <milimetric>	 hi
[22:33:03] <milimetric>	 yeah, i'm gonna be helping her out i think
[22:33:10] <milimetric>	 she's having trouble with the current umapi i heard
[22:33:36] <erosen>	 yeah
[22:33:39] <erosen>	 i can tell you more about it
[22:33:44] <erosen>	 as I looked into it with her briefly
[22:34:05] <erosen>	 this is the query she is using as a test: https://metrics.wikimedia.org/cohorts/wep_uscan_spring2013/bytes_added?start=20130101&end=20130601&group=input&aggregator=sum&project=enwiki
[22:34:31] <erosen>	 and the results she is getting are much lower than she would expect
[22:34:47] <erosen>	 i have a saved json output of the thing she thinks is wrong
[22:34:53] <erosen>	 and I was attempting to replicate on stat1
[22:35:00] <erosen>	 but I couldn't remember what port it is running ont
[22:35:25] <erosen>	 milimetric: I just sent you the result from metrics.wikimedia.org
[22:36:34] <erosen>	 (the results she thinks are wrong)
[22:40:15] <milimetric>	 cool, thanks erosen
[22:40:27] <milimetric>	 i gotta run some errands now but I'll take a look soon
[22:40:31] <erosen>	 cool
[22:40:45] <milimetric>	 if she asks, I'll get to it first thing tomorrow at the latest
[22:40:52] <erosen>	 cool
[22:41:31] <erosen>	 we also have userstats as a fall back
[23:16:02] <ori-l>	 hey guys
[23:16:18] <ori-l>	 how many page views do we get across all projects in a day, on average? (rough order of magnitude ok)
[23:16:55] <ori-l>	 nevermind, found it on reportcard.wmflabs.org