[00:10:49] dschoon - this is kind of bad ^^ [00:11:06] bundled deployment's not working AFAICT [00:11:30] milimetric: I don't see anything new about that [00:11:41] okay. [00:11:44] i'll look into it. [00:11:55] yeah, lemme know if you wanna brain bounce [00:12:06] but master and develop are the same [00:12:23] [travis-ci] master/485742e (#118 by milimetric): The build passed. http://travis-ci.org/wikimedia/limn/builds/6002414 [00:17:51] [travis-ci] develop/485742e (#117 by milimetric): The build passed. http://travis-ci.org/wikimedia/limn/builds/6002412 [00:52:49] ah-haha it worked [00:52:52] oh man [00:54:19] i am very excited to say that https://github.com/wikimedia/kraken/blob/master/pig/rollup.pig works [00:54:22] bwa-ha-ha [00:54:29] i will explain tomorrow [00:54:31] night! [01:07:18] yeah, i'm thinking that we break out the metric defs into a separate card and don't push them out this time around. Something (probably independent of, but coinciding with metric defs) broke deployment [13:38:01] morning [13:58:13] mooooorning! [13:59:22] mooorning! [13:59:54] bam bam bam bam bam bam bam bam [14:06:17] * YuviPanda hands drdee http://arstechnica.com/gadgets/2013/03/bullseye-from-1000-yards-shooting-the-17000-linux-powered-rifle/ [14:06:39] i was just practicing :D [14:07:04] but i got kernel panic so i had to retry [14:07:10] with their http://arstechnica.com/gadgets/2013/03/bullseye-from-1000-yards-shooting-the-17000-linux-powered-rifle/3/ ipad app? [14:07:12] :P [14:40:40] yo guys, are all the mingle cards in the right place? [14:43:29] milimetric: have you got a second to help with some limn permissions issues? [14:48:43] hey erosen [14:48:47] gotta restart real quick [14:48:49] and yes [14:48:54] cool [14:48:57] not urgent [14:49:03] just ping me when you get a sec [14:50:03] ok erosen [14:50:10] what seems to be the problem? [14:50:11] :) [14:50:43] basically something weird with the permissions that keeps me from pulling the data repo [14:51:00] drwxrwxr-x 8 www-data www 4.0K 2013-03-21 22:44 limn-data.bak/ [14:51:13] on gp? [14:51:15] yeah [14:51:18] why not just use fabric? [14:51:25] fab gp deploy.only_data [14:51:32] i suspected you might suggest that ;) [14:51:42] oh, no good? [14:51:51] no, i just didn't remember the command [14:52:10] do you have the limn-deploy project cloned? [14:52:17] pull 'cause there might've been updates [14:52:35] and if you haven't, pip install fabric [14:52:40] * erosen pulling [14:53:49] running into that username problem again [14:53:55] do you remember teh command line flag? [14:54:01] i'm reading the help now [14:54:10] you're doing this locally on your machine right? [14:54:19] -u [14:54:20] there we go [14:54:21] yeah [14:54:23] oh right! [14:54:35] so what's the command you use? so I know to tell others [14:54:43] is that the right way to do it (locally on my machine, even though the data lives on stat1 and the site on kripke?) [14:55:05] data's in gerrit right? [14:55:06] fab -u erosen gp deploy.only_data [14:55:13] i think it is working [14:55:20] the deployer will fetch the gerrit repo [14:55:22] i also think i realize what the problem was [14:55:35] i had a screen session on kripke that is a few months old [14:55:48] and the way I was pulling data was to manually do a git pull as user www [14:56:01] but you probably set things up to operate as user www-data [14:56:09] which meant that www didn't have permission [14:56:29] oh, yeah, www-data is the way all the sites work on kripke [14:56:34] Disconnecting from kripke.pmtpa.wmflabs... done. [14:56:35] yay [14:56:37] appears to work [14:56:38] the user is actually configurable so you can update stages.py [14:57:52] echo "fab -u erosen gp deploy.only_data" > somewhere :) [14:58:26] it worked [14:58:47] hehe [14:58:48] good call [15:21:34] ottomata: you could use git-deploy to sync all your puppetmasters! :) [15:21:46] * jeremyb_ has asked the same question before and never really got a good answer [15:23:23] yeah that'd be fine [15:23:25] or even just rsync [15:23:41] but not as cool [15:23:43] or production like [15:26:09] have a look and leave comments: http://etherpad.wikimedia.org/92 [15:28:00] MVC definition? [15:29:27] http://www.mediawiki.org/wiki/Analytics/Kraken/Meetings/ArchitectureReview#Minimal_Viable_Cluster [15:30:08] you know drdee [15:30:11] i think we could do option 2 [15:30:20] and it would be less hacky [15:30:24] if ops says its ok [15:30:34] i wouldn't have to set up a special rsync + hdfs put loop [15:31:52] ohhh, not model view controller [15:34:48] yeah i know [15:34:51] bad acronym [15:38:09] looks good to me drdee [15:38:26] i prefer 2 like ottomata [15:55:12] fyi drdee, kraigparkinson going to call into Problem Scrum today [15:56:08] ok [15:58:11] dschoon! [15:58:15] what in the world is going on [15:58:19] hm? [15:58:21] i deployed to prod and test [15:58:26] and the metric ref code is there [15:58:31] okay? [15:58:34] the datafiles and datasources and graphs are all referencing it [15:58:37] and it's not showing up! [15:58:40] okay. [15:58:44] we will look into this later today [15:58:48] in the middle of something right now. [15:59:05] http://reportcard.wmflabs.org/graphs/pageviews [16:00:15] dschoon: can you have a look at http://etherpad.wikimedia.org/92? [16:01:03] not at the moment. [16:01:04] in a bit [16:02:06] ok, drdee… ready to chat? [16:02:13] yes [16:15:57] erosen: hey! [16:16:00] helloo! [16:16:45] erosen: came here to remind you about that email. sorry for chasing you :) [16:16:58] geohacker: apologies about not yet responding to your e-mail, i was on vacation last week, and so far this week i've been in tunnel-vision-try-to-get-this-one-thing-finished [16:17:06] ah I guessed. [16:17:07] :D [16:17:33] however, I can at least respond tot he e-mail ;), even if I don't have cycles to develop stuff right now [16:17:47] erosen: cool. let's talk may be post Friday? [16:18:00] erosen: don't worry about the email :) [16:18:00] sounds good [16:18:23] erosen: awesome, I'll find you here then. :) [16:19:00] also when you have a bit do check the stuff that we made. pretty basic though. [16:19:08] sounds good [16:19:15] laters. [16:35:27] dschoon, around? [16:35:32] yum [16:35:33] yep [16:35:36] joined wrong hangout [16:41:12] New patchset: Stefan.petrea; "Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/analytics/webstatscollector" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57316 [16:43:33] New patchset: Stefan.petrea; "Fixed a segfault" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57317 [16:44:29] https://gerrit.wikimedia.org/r/#/c/57317/ [16:44:38] ottomata: please review this ^^ [16:44:43] average_drifter: thx! and could you make future summaries more descriptive? [16:45:09] drdee: yes [16:45:37] but wait, I've not finished yet [16:45:45] I still have some more files to add to the review [16:45:53] the binary search script [16:46:12] and I have to test on the big .gz files to see if there are anymore segfaults [16:49:56] New patchset: Stefan.petrea; "Fixed a segfault - checks in->project for null" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57317 [16:50:17] average_drifter, i guess so, looks ok with me [16:50:59] ottomata: cool :) thank you [16:54:19] ottomata: what's the canonic way of NULL safety check in C ? [16:54:20] I used [16:54:22] if(!x) [16:54:25] is it better to do [16:54:28] if(x==NULL) ? [16:55:07] they are equivalent. I guess it's maybe more a question of coding style then [16:55:13] mhm, theyr'e not quite equivalent [16:55:19] i don't know what would be 'canonical' [16:55:20] but [16:55:32] !x will be true if x == 0, or if x == false or i x == null [16:55:39] does NULL == '\0'? [16:55:43] oh sorry sorry [16:55:47] I meant x!=NULL [16:55:57] oh damn no, confused [16:56:07] so I just want to make sure something's not null [16:56:54] oh alright, but it can't be "false" because x is a pointer [16:56:56] forgot to mention that [16:57:30] well, false == 0 and a variable can be 0 [16:58:44] :) yes [17:01:26] New patchset: Stefan.petrea; "Fixed a segfault - checks in->project for null" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57317 [17:01:33] ottomata: had to add another patchset with the test file [17:01:37] sorry about that [17:02:04] no probbies! [17:02:18] ottomata: average_drifter: scrummy [17:02:31] 1PM is 1PM [17:02:56] ottomata ^^ [17:03:00] average_drifter ^^ [17:03:02] is it like right now ? [17:03:08] in 30m ? [17:03:10] yes [17:03:11] doh! [17:03:14] no scrum is now [17:03:20] now === now [17:03:22] ok, I'm on my way [17:03:25] to the link [17:03:51] https://plus.google.com/hangouts/_/2da993a9acec7936399e9d78d13bf7ec0c0afdbc [17:20:59] oh man [17:21:08] so i figured out how to do a generic rollup workflow [17:21:13] it's so awesome [17:25:17] oh? [17:34:49] before i go, ottomata [17:34:59] can you fire the public sync job now? [17:35:13] so the current hour/day/month rollups show up publically? [17:35:16] (for the showcase? [17:35:17] ) [17:35:18] ty [17:35:20] bbs [17:35:26] oh ja [17:57:00] ottomata , drdee I just finished running webstatscollector filter on 2013-February-01 => [today] and it didn't segfault [17:57:16] there's a high probability that the segfault is solved [17:57:25] yay ok [17:57:30] i will make new deb then [17:57:39] 1 sec [17:57:45] can you also run it against all feb data? [17:57:48] just to be sure [17:58:02] drdee: it was run on feb data also [17:58:12] no segfault [17:58:23] did you run it against 1 day of data or the entire month? [17:58:35] drdee: 1st february up to right now [17:58:40] every day [17:58:50] ok [17:58:52] perfect [17:58:57] then go ahead with debianization! [17:59:31] room is full, btw [17:59:33] kraigparkinson: [17:59:41] average_drifter, cna you send me link to gerrit change again? [17:59:47] ottomata: yes, moment [18:00:07] ottomata: https://gerrit.wikimedia.org/r/#/c/57317/ [18:02:47] heyaa, why can't I merge it? [18:02:51] jenkinsbot is all unhappy [18:04:32] Need Rebase or Has Dependency [18:04:33] ? [18:14:06] ottomata: I branched out of master [18:14:13] ottomata: should I have branched out of something else [18:14:39] ottomata: ? maybe time_travel ? [18:15:16] hm [18:15:24] i think master is right [18:15:29] we renamed it thought [18:16:57] ottomata: uh, how should we go about this? can I help with merging? [18:17:39] ottomata: should I merge with master maybe ? [18:17:48] and then make another patchset [18:18:19] you can probably push directly, if you want to try [18:18:21] i dunno [18:20:44] New patchset: Stefan.petrea; "Fixed a segfault - checks in->project for null" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57317 [18:27:07] New patchset: Stefan.petrea; "Fixed a segfault - checks in->project for null" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57317 [18:28:12] jenkins bot Y U NO LISTE TO MEH [18:28:30] jenkins-bot is drunk [18:28:38] is hashar around ? [18:29:46] ottomata: can you merge it regardless of what jenkins-bot thinks ? [18:30:50] maybe, i tried witih the old one [18:30:53] in meeting now, can't try right now [18:31:58] ok, which meeting btw ? [18:32:05] backlog ? showcase ? [18:32:18] showcase [18:32:26] https://plus.google.com/hangouts/_/5b70172d0f7418695ff6d98f3cb53bbb7097e020 [18:37:44] New patchset: Stefan.petrea; "Fixed a segfault - checks in->project for null" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57328 [18:37:47] ottomata: thanks [18:38:46] (dunno why you aren't explicitly invited to these things) [18:39:27] ottomata: I was but the link changed or whatever [18:39:46] ah [18:39:47] I wish I could have a @wikimedia.org , but not sure if that's the cause [18:39:53] ottomata: https://gerrit.wikimedia.org/r/#/c/57328/ [18:39:57] ottomata: use this please [18:40:04] jenkins does not complain on that one [18:40:12] it has the same contents as 57317 [18:40:26] I'll abandon 57317 [18:41:56] Change abandoned: Stefan.petrea; "in favor of https://gerrit.wikimedia.org/r/#/c/57328/" [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57317 [18:42:18] drdee: can you abandon this please ? https://gerrit.wikimedia.org/r/#/c/57317/ [18:42:21] I tried but could not [18:43:34] it says you abandoned it [18:43:57] this is what jenkins bot says [18:43:57] https://integration.wikimedia.org/ci/job/analytics-webstatscollector/27/console [18:43:57] i can try to ignore it [18:45:07] ottomata: please use 57328 instead of 57317 [18:45:51] ja i can [18:46:00] should I ignore jenkinsbot? [18:46:09] 18:37:40 autoreconf: `configure.ac' or `configure.in' is required [18:46:09] 18:37:40 Build step 'Execute shell' marked build as failure [18:46:09] 18:37:40 Finished: FAILURE [18:46:18] yes, jenkins is out of date on this [18:46:28] it must be outdated [18:46:38] but not sure if we have time to put it up to date right now [18:46:45] so yes, you can ignore it [18:54:00] drdee: the #92 etherpad needs a bit of work [18:54:12] (imo) [18:54:28] i think the reportcard deploy comes first [19:07:22] Change merged: Ottomata; [analytics/webstatscollector] (master) - https://gerrit.wikimedia.org/r/57328 [19:11:19] dschoon: can you be more specific regarding #92 email, i would like to send out the email today [19:11:33] i will do so by getting to it [19:11:43] but i think the reportcard deploy comes first, yes? [19:12:03] i hope to get to it by this afternoon [19:12:11] but no matter what, the reportcard deploy needs to be finished today [19:12:28] ^^ drdee [19:12:47] yes that's true as well :) [19:26:24] dschoon; can you give me modify rights to the scrum meeting in google calendar? [19:26:32] everyone should have it [19:26:37] i don't think i own the scrum any more, tho [19:26:42] kraigparkinson probably does [19:26:50] I don't either. [19:27:06] milimetric: does [19:27:10] no you are the owner [19:27:30] huh [19:27:32] hokay [19:27:33] fixing? [19:27:34] but i mean [19:27:49] Everyone has Modify already, drdee [19:28:25] well i can't modify the repeat cycle [19:28:35] I can modify it, and I did. [19:28:46] ty [19:29:09] should see it as of next week. [19:33:59] we are all still in the same hangout [19:34:17] ok [19:56:17] New patchset: Jeremyb; "cleanup duplicate content in UDP-FILTER(1)" [analytics/udp-filters] (master) - https://gerrit.wikimedia.org/r/57395 [19:58:27] New review: Diederik; "Thank you!" [analytics/udp-filters] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/57395 [19:58:27] Change merged: Diederik; [analytics/udp-filters] (master) - https://gerrit.wikimedia.org/r/57395 [20:00:42] drdee: do you know anything about the NARA filter? [20:00:50] who asked for it, who uses the data? [20:01:08] GLAM folks asked for the filter, it is not being analyzed by anybody ATM [20:01:26] who gets the logs? [20:01:35] which glam folks? [20:01:46] 2212 didn't seem to say who [20:02:29] nobody gets the logs [20:02:34] hrmmm [20:02:35] maarten dammers [20:02:40] ahhh [20:02:41] cool [20:05:13] drdee: saw the build failure? [20:05:25] yeah, jenkins is broken [20:06:07] k [20:07:11] pipe 10 is 10% and pipe 100 is 1%? [20:07:23] it's a hashar! right on schedule [20:07:34] kind of :D [20:07:48] hashar: build failed [20:07:57] fix your code? :-] [20:08:09] drdee: so, how do we get the data? how far back do we have? needs NDA? [20:09:42] hashar: 03 20:05:12 < jeremyb_> drdee: saw the build failure? [20:09:42] 03 20:05:25 < drdee> yeah, jenkins is broken [20:09:57] i meant our use of jenkins is broken [20:10:00] jenkins is not borken [20:10:23] ;-] [20:10:40] I would be happy to help but not tonight sorry :-( [20:13:12] why are all the europeans leaving? [20:13:32] 03 20:08:16 <+multichill> Was heading for bed when you pinged me [20:13:34] :-) [20:21:19] yeah 11pm20 winter time [20:21:25] so we are all wacked :/ [20:21:31] bed time for me as feel. *waves* [20:23:49] drdee: 03 20:08:09 < jeremyb_> drdee: so, how do we get the data? how far back do we have? needs NDA? [20:24:11] yes you need an NDA, data goes back little bit less than a year [20:25:00] not all the way to the beginning? [20:25:06] 2012-01-11 [20:25:15] how do we get the ball rolling? [20:27:14] ottomata: i reduced scope of #460 significantly and create 509, 510 and 511. Happy to brain bounce about upgrade plan for emery and oxygen [20:27:27] ok cool, wanna do that right now after I pee? [20:27:33] actually, lemme pee, and upgrade webstatscollector [20:27:35] talk in 20? [20:28:15] in any case, as long as you're talking about emery, can we leave NARA as is for now? [20:28:21] or move it somewhere? [20:28:32] leaving it is fine [20:28:41] we just wanna clean up whatever is not used [20:28:47] can you respond to email thread if you want it? [20:29:27] ottomata: well i was just asking drdee what the next step was to get the data. some people i just talked to do want to see it but didn't realize it existed until now [20:29:51] do i start with legal or RT or what? [20:30:50] oh, bwaa [20:31:12] they need to sign the data nda, and then get approval to access stat1 (and I have to sync the files to stat1, but that's easy) [20:32:39] can you send me the data nda? [20:32:41] erosen: can you update https://mingle.corp.wikimedia.org/projects/analytics/cards/21 ? [20:33:38] um, jeremyb_ i cannot, i have no idea where it is or what it looks like [20:33:41] maybe drdee can? [20:34:12] jeremyb_ on whose behalf is this data, is this for yourself? or somebody else? [20:34:43] erosen: also, created https://mingle.corp.wikimedia.org/projects/analytics/cards/512, input is appreciated [20:35:40] drdee: i don't know exactly how to answer that. i've just been discussing it with the guy that apparently asked for it and then forgot about it (maarten) and Dominic (the NARA Wikipedian until a few months ago) [20:35:58] ask them to send me an email [20:35:58] drdee: they're both interested in seeing the data as am I and probably others too [20:36:30] * jeremyb_ has been to NARA several times himself [20:36:33] ok [20:41:08] Dmcdevit: can you mail "Diederik van Liere" and CC me, maarten, and whoever else you think should have access? [20:41:36] doesn't have to be long or elaborate [20:41:46] probably requires some sort of NDA :( [20:42:54] and if it proves useful then it's not so hard to expand to other GLAMs [20:42:58] i think [20:43:23] but NARA was extra easy because of the common part of the filename. so hopefully other people have that too [20:44:01] gah [20:44:07] why am i typing that in here? [20:44:14] * jeremyb_ copies to other place [20:44:16] :-) [20:44:31] i know the answer, and the answer is "it depends" [20:45:31] bbl [20:47:25] ok, average_drifter, drdee: upgraded webstatscollector on gadolinium to 0.1-3. Let's see if we still get segfaults... [20:47:44] cool! [20:48:29] nice :) [20:48:40] drdee, ready to talk if you are [20:49:03] got 10 minutes, then meeting with kraig [20:49:08] let's do ti [20:50:12] k! [21:31:10] okay [21:31:17] found the problem with the reportcard deploy [21:31:30] it was a merge that missed a commit [21:31:34] dan is re-deploying [21:31:51] deploy to reportcard.wikimedia.org, yeah! [21:31:58] no no [21:32:01] this is all dev/test :) [21:32:14] no no, yes yes [21:32:31] dschoon is the man [21:32:33] did you guys know? [21:32:35] http://test-reportcard.wmflabs.org/graphs/pageviews [21:33:21] yay [21:33:35] kraigparkinson, drdee reportcard deployed to dev/test [21:33:39] issues resolved [21:33:46] thanks dschoon [21:59:45] man, i am SO not a fan of oozie [22:00:01] workflow EL Functions don't work in coordinator.xml files! [22:00:04] for NO reason! [22:00:19] so i can use replaceAll() on strings in workflow.xml, but not in coordinator.xml? [22:00:22] so stupid. [22:08:41] [travis-ci] master/44e855f (#119 by milimetric): The build passed. http://travis-ci.org/wikimedia/limn/builds/6030367 [22:08:56] [travis-ci] develop/44e855f (#120 by milimetric): The build passed. http://travis-ci.org/wikimedia/limn/builds/6030483 [22:38:09] brion: you about? [22:38:38]