[00:13:25] HenriqueCrang: do you have a preferred username/ [00:31:29] [travis-ci] master/da9093c (#79 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5427677 [01:20:22] [travis-ci] master/c76dbb4 (#80 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5428405 [05:10:40] [travis-ci] master/af08eca (#81 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5431728 [14:52:17] yooooooooooooo [15:51:25] hi [15:52:48] morning average_drifter [15:52:54] or afternoon... [15:53:08] yes, kind of afternoon [15:54:52] dentist chopped off an entire tooth of mine(to insert an artificial one instead), and she's going to take another one out on friday(the wisdom one) [15:54:57] I hope it does not affect the way ladies view me [15:55:18] woah [15:55:31] [travis-ci] master/a519bcb (#113 by Dan Andreescu): The build passed. http://travis-ci.org/wikimedia/limn/builds/5443519 [15:55:32] I will try to smile less so they don't see my missing tooth [15:55:40] hehe [15:55:45] (missing for the next 3 days) [15:55:49] so the tooth is missing and not yet replaced [15:55:51] interesting [15:55:56] yes, she took a mold [15:56:00] i see [15:56:02] so she's going to make another one for me [15:56:07] she's like a tooth fairy of sorts [15:56:09] hehe [16:33:20] drdee, you there? [17:00:18] late~ [17:00:24] all of you are late! [17:00:39] drdee: kraigparkinson milimetric [17:00:40] erosen: [17:27:40] dschoon: are you on the office? [17:30:37] milimetric: hey [17:30:40] i do have stat1001 access :) [17:30:46] but it doesn't even have virtualenv :| [17:30:50] do you have sudo on it? [17:31:04] no i don't think so [17:31:22] in meeting right now, will talk after [17:31:22] ok [17:35:24] YuviPanda: can you not install virtualenv with --user? [17:35:32] hmm? [17:35:37] * YuviPanda googles [17:35:46] but I think it'd still be nice to get it installed systemwide [17:36:02] erosen: also, I'm unsure if they have python-dev installed. A fair number of packages are going to be uninstallable without ath [17:36:03] *that [17:36:20] interesting [17:36:58] it seems... very pristine? [17:37:02] hehe [17:38:09] erosen: what is the usual procedure to get more packages installed? [17:38:13] another RT ticket? :( [17:39:32] erosen: also I'm unsure, but /var/www is empty [17:39:38] and I don't have write access to it [17:40:45] YuviPanda: sorry, was afk for a sec. [17:41:07] YuviPanda: I think the usual procedure is to either ping ottomata or file an RT ticket [17:41:27] hah [17:41:33] oh, no ottomata [17:41:34] yet [17:41:51] I think he should be around though [17:41:57] he was at scrum this morning [17:42:43] yeah, okay. [17:44:28] average_drifter: regarding the prefix tree idea, I just realized that the purpose of putting it in a tree is so that you avoid duplication. [17:44:56] average_drifter: I still think a counter of subdomains / params is better, but just thought I would let you know [17:45:10] erosen: agreed [18:25:54] crashing [18:26:15] google hangout is having troubles... [18:34:56] erosen, average_drifter: wanna talk about pageviews? [18:35:05] drdee: sure [18:35:19] drdee: average drifter might be running some results, but we can probably at least chat [18:35:26] ok [18:35:30] hangout? [18:35:32] yes [18:35:47] maybe we'll wait to hear from average_drifter [18:35:54] https://plus.google.com/hangouts/_/2da993a9acec7936399e9d78d13bf7ec0c0afdbc [18:36:07] average_drifter: ^^ [18:38:25] drdee: now [19:04:46] milimetric: I'm sick (eye swelling up, vision blurry) so am going to go off now. Hopefully will be back up tomorrow but unsure. If you can, can you poke ottomata / someone and get virtualenv + python-dev installed on stat1001? [19:04:55] and a solution for /var/www? [19:04:59] thanks. [19:05:06] oh no [19:05:08] ! [19:05:24] yeah, last time this happened (4 years ago, same eye) i needed surgery [19:05:26] no problem Yuvi, I'll ask around [19:05:33] I hope it doesn't come to that [19:05:36] get plenty of rest [19:05:37] yeah, me too :| [19:05:41] will do. [19:06:02] milimetric: getting a directory in /var/www writeable by yuvipanda would also be nice :) [19:06:06] thanks a lot :) [19:06:08] ok, no prob [19:06:50] thanks [19:31:02] yeehawwwww [20:00:12] ottomata: heya! [20:00:17] HI! [20:00:18] Blurry! [20:00:19] ? [20:00:20] haha [20:00:37] BlurryPanda, I already asked ottomata what you needed [20:00:48] get some rest, we'll take care of it :) [20:00:51] ottomata: my right eye is swelling fast and badly, hence blurry [20:00:56] haha [20:00:57] milimetric: tried to sleep and failed miserably. [20:01:08] chamomile tea? [20:01:34] it's a mild disinfectant and might help with the swelling a bit [20:01:55] otto, I wrote you an email with a few things Yuvi needs [20:01:58] on stat1001 [20:02:18] yeah, i responded, kinda with an I don't know, I think we need to ask ops [20:02:29] milimetric: well, I'm in the middle of rural-ish Kolkata (came here to conduct a JS / MW workshop), do not know any of the local languages, and only thing I have is 2L of drinking water and some cigs [20:02:32] I had set up the ability for users to host whatever they wanted on stat1 [20:02:34] and they didn't like it [20:02:49] milimetric: will try tomorrow though :) Thanks :) [20:03:18] ok, I'll ping ops [20:03:33] or do you want to talk to them? [20:03:47] are you on the ops mailing list? [20:04:18] can you guys cc me as well? [20:04:19] i think so yea [20:04:22] sure [20:04:24] (am not on ops list) [20:07:24] brb [20:07:45] erosen: irc ? [20:08:03] erosen: G+ is messing with me, dunno why :( [20:08:44] damn [20:09:14] erosen: ? [20:09:19] hey [20:09:20] just returned [20:09:24] * BlurryPanda tries to sleep again. [20:09:24] https://plus.google.com/hangouts/_/2da993a9acec7936399e9d78d13bf7ec0c0afdbc [20:09:26] thanks milimetric :) [20:09:34] average_drifter: ^^ [20:18:44] hello! [20:19:18] milimetric, ottomata you're both still working on oozie stuff? [20:19:32] hey dschoon [20:19:38] I'm reading up on pig so we can work together [20:19:38] hola [20:19:42] okay. [20:19:56] did we resolve the oozie issues that blocked finishing off those tasks? [20:20:08] i was thinking for the first task, maybe we can both write a pig script for it? [20:20:15] yuppers [20:20:24] i'm just getting into it actually, but will be on for many hours today [20:21:04] dschoon, basically, right now, the zero oozie coordinator isn't working (don't know why yet), and sometimes the device oozie coordinator fails [20:21:06] but not always [20:21:07] so there are 240, 244, 61, and 78 [20:21:15] interesting. [20:21:29] okay, and judging from email, there's dissent on the "easiest" way to debug this [20:21:29] we think it is a jar/path issue, although i don't know why that would cause device to only sometimes fail [20:21:30] otto's working on 61 and 78 [20:21:34] you're in charge of 244 and 240 [20:21:45] can we start by picking a place to start collecting information? [20:21:55] an etherpad with log snippets or something/ [20:22:02] too huge [20:22:06] haha, but maybe [20:22:10] well [20:22:10] ...a folder? [20:22:16] on an02? [20:22:18] or whatever? [20:22:19] i thought we were supposed to work on pig [20:22:27] and help if otto needs to brain bounce [20:22:48] i don't know what "work on pig" means. [20:22:53] i know we have four cards that need closing [20:22:55] 244 and 240 don't have pig scripts yet [20:22:59] and atm, there are unknown problems. [20:23:06] 244 does [20:23:08] doesn't it? [20:23:19] oh user adoption? [20:23:21] that's not just pageviews? [20:23:32] diederik asked this morning if david was working on 244 and 240 and david said yes [20:23:33] we have a zero pig script already [20:23:37] okay. [20:23:48] we don't have mobile beta/alpha [20:23:49] my understanding is that i'm working on all of them [20:23:50] that's 240 [20:23:54] in the sense that they all need work [20:24:02] and we need to figure out what the work is, and divide it between us [20:24:11] ok, so there's 240 pig script [20:24:11] (if we've done that, just tell me :)) [20:24:19] and oozie for all of the 4 cards [20:25:14] okay, so should we jump in a hangout? [20:25:21] sure [20:25:32] standup link [20:25:38] maybe in 5m when everyone finds a stopping place? [20:25:42] i want to grab water [20:26:02] nvm, taken [20:26:15] milimetric: undecided? [20:26:18] milimetric: come back [20:26:28] :P [20:26:56] haha, no i was trying to meet with dschoon [20:26:56] dschoon, jump in when ready: https://plus.google.com/hangouts/_/674fb84b74e3bc9c04e4b5a558da2e949e9cf5ab [20:27:20] 1sec [20:28:29] ottomata: you wanna join for a few? [20:31:38] http://etherpad.wmflabs.org/pad/p/AnalyticsTodo [20:58:22] YES [20:58:25] SO CONFUSED [20:58:30] brb noseblowing [20:59:44] this is a no confusion zone. Confusion lives in private messages until it becomes clarity. this is a clarity-encouraged zone. [21:10:42] okay. [21:10:49] ottomata, milimetric [21:11:02] i'm going to update cards with info i've learned [21:11:11] and compile questions in that etherpad [21:11:15] cool [21:11:22] and i will update the cards using those questions [21:11:22] but there's one super important page i didn't know about: [21:11:28] https://mingle.corp.wikimedia.org/projects/analytics/cards/hierarchy?columns=Responsible%2CFeature+Status%2CBlocked+Reason%2CProject+Name&excluded%5B%5D=Milestone&excluded%5B%5D=Deliverable&expands=240&style=hierarchy&tab=Task+Breakdown+by+Feature&tf_feature%5B%5D=%5BFeature+Status%5D%5Bis+less+than%5D%5BDone%5D&tf_feature%5B%5D=%5BFeature+Status%5D%5Bis+greater+than%5D%5BQueued+for+Dev%5D&tf_milestone%5B%5D=%5BWork+Breakdown+ [21:11:29] Structure+-+Milestone%5D%5Bis%5D%5B81%5D&tree_name=Work+Breakdown+Structure [21:11:31] gah [21:11:35] it's TOO LONG FOR IRC [21:11:37] omg [21:12:05] lol [21:12:13] 3rd or 2nd tab [21:12:29] http://j.mp/13RHE7Z [21:12:33] Nice [21:12:40] ottomata, milimetric: that's the link to the task-tree view [21:12:46] a lot of the magic missing cards ended up as tasks [21:12:49] (which is good!) [21:13:09] you should both feel free to add task cards to features you're wroking on [21:14:07] drdee: can you repeat what you said about the changes to X-CS for ottomata and milimetric? [21:14:15] it's in the card but yes [21:14:27] X-CS is current a single value field [21:14:46] 1sec [21:14:52] once mobile merges https://gerrit.wikimedia.org/r/#/c/52605/ [21:14:53] and [21:15:10] https://gerrit.wikimedia.org/r/#/c/52606/ [21:15:29] it becomes a semicolon separated key/value pair list [21:15:38] the Zero Pig UDF can handle both cases [21:15:43] and has unit-tests covering both cases [21:15:54] so you can oozily this job and it won't break [21:16:04] ok? [21:16:45] that sounds good. [21:16:50] so shouldn't 240 and 244 be blocked then? [21:16:56] no [21:16:56] ^^ agree [21:16:57] on mobile deploying [21:17:13] 244 we can do because the X-CS header is being set [21:17:16] it will just cahnge [21:17:37] hm, okay. yeah. [21:17:38] i buy that. [21:17:43] 240 is indeed waiting for the mobile stuff to bedeployed [21:17:44] but 240 should def be blocked [21:17:48] it's impossible right now. [21:17:51] right [21:17:59] check with mobile team if they wanna deploy tomorrow [21:18:27] well, if we know the format after the change, we can still code it and put the card into testing [21:18:33] then wait for the deployment in order to test it [21:20:29] if 244 works [21:20:36] then 240 will also work [21:20:50] same script [21:20:53] sligthly different data [21:21:02] and where is this script? [21:21:32] this one: https://github.com/wikimedia/kraken/blob/master/pig/webrequest_zero_hour_carrier_country.pig [21:21:50] drdee? ^^ [21:21:58] yup, that's the zero that i'm going to oozify as soon as the device one works [21:22:12] brb another meeting [21:22:55] thanks to dschoon for bringing the problem re: complete and accurate info on cards to the attention of drdee and me. we'll use the sprint planning part of our meeting tomorrow to add acceptance criteria and technical info to the cards in progress, and the weekly grooming meeting as a way to do that for all cards going forward... [21:23:13] sounds good kraigparkinson [21:23:16] i'll try to do some grooming tonight as i missed the meeting today (stupid doctor) [21:23:27] also, i don't think zero can go out tomorrow. [21:23:47] i'm looking at the patchset, and they don't encode the separator in MobileFrontend [21:24:56] please feel free to grab drdee for clarification on cards as needed. :) Just make sure not to bite him; he [21:25:03] he's still catching up. [21:30:14] word. [21:35:29] yes please ask questions but also understand that all of us are in a transformation state, so if a card is unclear ping me, and i will clarify, or put questions on the card and let me know [21:39:35] iiinteresting... [21:39:36] 2013-03-12 21:36:26,771 INFO [Low Memory Detector] org.apache.pig.impl.util.SpillableMemoryManager: first memory handler call - Collection threshold init = 139853824(136576K) used = 135269312(132098K) committed = 139853824(136576K) max = 139853824(136576K) [21:39:36] 2013-03-12 21:36:27,357 FATAL [Low Memory Detector] org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[Low Memory Detector,9,system] threw an Error. Shutting down now... [21:39:36] java.lang.InternalError: Error in invoking listener [21:39:48] yes [21:39:54] this is a big issue [21:40:01] you got to talk with david about this [21:40:08] once the datasets get too big [21:40:18] there are low memory / OOM errors [21:40:47] dschoon ^^ [21:40:48] hmm [21:44:52] maybe change this setting [21:44:52] [21:44:53] mapreduce.job.reuse.jvm.num.tasks [21:44:54] -1 [21:44:55] [21:45:17] this disable reloading the jvm, this is not default behavior [21:45:24] i belief default behavior is 1 [21:45:36] this is in mapred-site..xml [21:45:49] maybe this needs to be upped: [21:45:50] [21:45:51] mapreduce.child.java.opts [21:45:52] -Xmx512M [21:45:53] [21:51:21] milimetric: can you point to an example of a limn graph that is stacked? [21:51:31] sure, one sec [21:51:34] ty [21:51:42] http://reportcard.wmflabs.org/graphs/pageviews_mobile_target_stacked [21:51:47] that's stacked bars [21:51:50] with a target line [21:52:10] stacked area is not yet supported, though it should be easy after the viewport unification [21:52:17] gotcha [21:52:31] but what about stacked with no area [21:52:34] if that makes sense [21:52:44] just stacked but the integral isn't shaded [21:58:09] right, not supported either [21:58:24] but should be fairly straight-forward [22:02:57] COOL [22:03:19] i'm not entirely sure why some of the previous device runs failed, but I know why the most recent ones did (missing Geo IP files…again :/ ) [22:03:28] but, I have scheduled all the failed ones to rerun [22:03:32] and I think they are trucking along... [22:03:41] now I wonder if zero will run… :) [22:08:00] reading [22:08:18] ottomata: you about? [22:10:19] yup hey [22:10:52] you changed Xmx in mapred-site? [22:17:01] no, that error was from a webreqeust loss job, i was experimenting with reruning oozie jobs with t hat and got the error [22:17:33] i'm reading on how oozie interacts with the normal hadoop conf files [22:17:40] any pointers i should look at? [22:22:07] hm [22:22:13] no i'm not sure either [22:23:26] drdee [22:23:33] did the output of the device pig script change? [22:26:27] 2013-02-02_06 VU Android true 720 31 [22:26:27] vs [22:26:28] 2013-03-07_06 CG Apple iPhone OS 3 [22:26:37] HMm, or maybe that was only for a short period of time [22:43:11] i think new metrics have been added, ottomata [22:43:17] the requirements in #61 are different [22:43:27] https://mingle.corp.wikimedia.org/projects/analytics/cards/61 [22:44:21] i think it now includes JS enabled, and screen size [22:50:45] i think the zero stuff is failing because maybe hdfs doesn't have drdee's latest jars for the Zero udf [23:11:37] put new jars in /libs/ [23:14:19] danke! [23:19:36] [travis-ci] master/d97365f (#82 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5456329 [23:20:48] drdee, dschoon [23:20:51] you both think I should cahnge −1 to 1 [23:20:55] right? [23:21:15] yes i think os [23:21:19] yes [23:21:24] mapreduce.job.reuse.jvm.num.tasks [23:21:25] for that [23:21:52] tho i don't see it in a non-site config [23:22:02] which probably means it 1 by default [23:22:05] and being shadowed [23:22:13] iirc it's 1 by default [23:22:32] hmm, so 1 will make it spawn up new JVMs for each mapper? [23:22:48] yes [23:22:48] so that is slower [23:22:53] ok, but it would avoid OOM [23:23:31] perf can become a concern when "working" is not. [23:23:32] ok, i'm going to change this to 1 everywhere and restart hadoop before I try to launch this big pig zero task [23:23:36] hha, yeah [23:23:47] mapred-default.xml [23:23:52] is where it should be [23:23:52] alrightie, i am off to dinner [23:23:56] no, we modify site [23:23:58] defaults just have defaults [23:24:02] it is being set in -site [23:24:05] ok drdee [23:24:07] isn't site per-job? [23:24:18] site is where you put custom configs [23:24:24] my bad. [23:24:26] truth. [23:24:27] /etc/hadoop/conf/mapred-site.xml [23:24:37] yep. [23:24:43] waaahhhh, i want to change this with puppet :( [23:29:31] !log restarting hadoop with mapreduce.job.reuse.jvm.num.tasks set to 1 [23:29:34] Logged the message, Master [23:29:45] yay logbot is back! [23:32:49] yay! [23:33:07] i know! let's set up our own puppet and... [23:33:11] * dschoon ducks [23:33:48] diederik already went to dinner, right? [23:33:54] and milimetric is out? [23:34:09] if so, i'll eat and head home (to my bigger monitor) before continuing [23:34:19] you already eat, ottomata? [23:35:22] btw, i dumped the conf into a more readable columnar format [23:35:22] https://gist.github.com/dsc/5148082 [23:35:23] naw, not yet, but had a lunch at about 3pm [23:35:39] ha, cool [23:36:01] so i'm in mexico with friend who is working remotely too, we'll probably get dinner in an hour or two [23:36:08] [travis-ci] master/8426642 (#83 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5456751 [23:36:34] lol [23:36:41] okay, good to know, ottomata [23:36:49] i desire a big screen [23:36:51] so i'm gonna bolt. [23:36:55] back in 15. [23:37:04] mmk latas [23:43:43] I AM RUNNING SO MANY PIG MAPPERS RIGHT NOW [23:43:50] 33,835 of them! [23:46:46] brb