[02:18:39] it's sunday [02:18:49] early morning [02:19:12] drdee: hi :) [02:21:33] hey [02:21:49] what's up? [02:22:12] well, I'm back on track I guess [02:22:16] solved some of the problems [02:23:01] trip to s.f. got cancelled(time to get visa got too big and over 3-9march) [02:23:35] other than that I'll be on the remote meetings next week, I mean this week(since it's passed midnight) [02:24:24] need to get back to code [02:25:24] drdee: last time we talked I know I had to work on the new mobile pageview report and also extract the logic of wikistats into a diagram [02:25:31] drdee: if anything changed please let me know [02:25:41] i finished the diagrams [02:25:44] we should talk about those [02:25:53] we must fix the 500m bump asap [02:26:02] that's nr 1 priority [02:26:08] ok [02:46:56] i pushed flowdiagram for kraken to wikimedia/metrics [02:50:20] looking [02:52:15] does kraken also have the 500M ? [02:52:40] probably not, was curious to know though [02:54:02] we never tried :) [02:55:31] ok [02:55:38] wikistats still needs a diagram too right? [02:55:50] or we can skip that? [02:56:30] wikistats used webstatscollector for regular pageviews [02:56:30] drdee: there in club quarter? [02:56:39] hashar: yes [02:56:40] drdee: chad is coming at the pub in roughly 20-25 min [02:56:45] cool! [02:56:53] drdee: I am getting my laundry downstair and will get a beer straight after [02:57:22] average_drifter: we might want to document the old mobile page view logic in wikistats (not written by you but by andre) [02:57:51] [travis-ci] master/e4bfda5 (#68 by Diederik van Liere): The build has errored. http://travis-ci.org/wikimedia/kraken/builds/5216086 [03:07:34] ok [03:07:40] I put the 500M on mingle [03:07:57] it's already there [03:08:01] oh, didn't know [03:08:06] can you assign it to me please ? [03:08:45] already done [03:09:37] this is the one I've added https://mingle.corp.wikimedia.org/projects/analytics_archive/cards/263 [03:09:44] link me with the one you made please [03:10:01] I'll delete the one I've made and use yours [03:25:33] you are working in the wrong project [03:25:42] the project is called analytics not analytics-archive [03:26:51] oh ok [03:32:08] does mingle have some video lectures I could watch? [03:32:30] I found the task [03:32:34] yes, it was assigned to me [03:33:09] ok, so Analytics Archive are the tasks from Asana migrated to Mingle maybe ? [03:33:17] i dunno actually [03:33:21] and Analytics is the project I'm working on [03:33:28] yes [03:33:35] ok [03:33:51] card #60 is yours [14:01:38] morning all! [14:29:01] hello! [14:29:12] anyone know where the source for the reportcard wmflabs limn instance is? [14:40:10] the sourcecode? [14:40:22] https://github.com/wikimedia/limn [14:40:36] ottomata: for that particular instance [14:40:45] as in, the data? [14:40:49] README of limn links to wikimedia/reportcard [14:40:51] which 404s [14:40:59] ah, yeah, it should be what I just linked to [14:41:04] its wikimedia/limn [14:41:22] ottomata: wikimedia/limn has a README [14:41:23] that says [14:41:24] Once you've done that, the source to WMF's production instance of Limn, the monthly Reportcard, might serve as a useful guide. [14:41:33] 'the source' links to https://github.com/wikimedia/reportcard [14:41:35] which is 404 [14:41:48] so it should just link to itself? [14:41:52] ohhhhhh [14:42:02] hmmmmm [14:42:06] hmm [14:42:24] I think it should like to itself... [14:42:25] weird [14:42:53] heh :) [14:43:31] filed issue [14:45:26] ok cool thanks [15:49:39] moving locations, back on in a bit [18:00:45] wee [18:00:45] https://plus.google.com/hangouts/_/2da993a9acec7936399e9d78d13bf7ec0c0afdbc [18:02:52] drdee, dschoon, milimetric [18:02:54] ^^ [18:02:57] nope. [18:03:00] nope? [18:03:01] infinite meeting today [18:03:02] 10a [18:03:08] Analytics Reboot [18:03:10] OH [18:03:12] (see invite for hangout URL) [18:03:12] that starts now? [18:03:15] 10a! [18:03:22] ah poop scoops [18:03:23] right [18:03:27] ok, be right there [18:03:56] brb [18:07:11] did you join? [18:10:52] [travis-ci] develop/95d0ea3 (#96 by milimetric): The build passed. http://travis-ci.org/wikimedia/limn/builds/5232018 [18:23:00] milimetric: http://stat1.wikimedia.org:1337/app-stats.html [18:23:15] (tiny stopgap until we get Limn up) [18:23:48] We're in a long meeting today YuviPanda, I'll ping you soon (thanks for catching that README 404, I'll fix it after this meeting) [18:24:00] haha okay milimetric :) [18:26:14] btw, milimetric, the limn puppet module works great, witha pache too [18:26:15] but [18:26:17] i didn't know this [18:26:25] you need to add a labs instance to the 'web' security group when you create it [18:26:40] in order to get HTTP traffic through whatever labs firewall labs puts in place [18:26:44] thanks otto, will take a look in a bit [18:26:45] you can't edit it after the fact :/ [18:26:46] yup [18:53:03] erosen (meeting, I know), oxygen is not happy: packet loss, any idea on what the status of disabling IP based filters in favor of X-CS soon? [18:53:41] we still haven't gotten the first batch of monthy reports with x-cs [18:53:44] ok [18:53:55] drdee knows more about the status of the relevant pig scripts [18:54:01] but it could happen this week [18:54:10] well, i mean on oxygen itself [18:54:19] we could disable all of the IP filters, and have a single one that looked for hte present of X-CS [18:54:23] presence* [18:54:27] we should do that this week [18:54:41] best to run one filter on oxygen and one on kraken [18:54:54] i understand if oxygen is really struggling, but I think it is a bit dangerous to blindly trust the x-cs stuff [18:54:59] yeah, i agree [18:55:04] agreed [18:55:04] but there is packet loss there anyway [18:55:11] during high load times [18:55:18] true... [18:55:19] i think you are the only real consumer of this stuff, so as long as you are ok with that... [18:55:37] there are only really zero filters there right now [18:55:38] erosen, we've tested the validity of X-CS, right? [18:55:42] so i don't think erik z will be othered [18:55:47] yes we didd [18:55:57] bothered* [18:56:42] we still haven't compared any final outputs of the new logs [18:56:53] i did some basic checks, but I haven't looked at a whole month [19:33:27] hey YuviPanda, http://test-reportcard.wmflabs.org/datasources is a good place to go to play with datasources right now [19:33:44] ah [19:33:44] * YuviPanda clicks [19:33:47] the dev instance is not very stable atm. [19:34:18] cool page on stat1...app-stats.html btw [19:34:30] milimetric: I guess that's pulling things from https://github.com/wikimedia/limn-data/blob/master/datasources/rc_binary_files.yaml [19:34:58] the /datasources page pulls all datasources linked to an instance of limn [19:35:10] milimetric: yeah, I mean, in that instance... [19:35:14] the rc_binary_files.yaml is just one example [19:35:18] yeah [19:35:23] well, in this case I apologize [19:35:28] I meant to link to https://github.com/wikimedia/limn-data/tree/master/datasources [19:35:35] the limn-data repository is just a copy of the datasources on reportcard [19:35:44] ah! [19:35:50] right [19:35:57] so this particular limn-instance is actually pulling from the reportcard gerrit repository (link impossible to find as always with gerrit) [19:36:03] hehe :D [19:36:20] but yes, it's the same definition. But if you're after a github repository with example definitions, here I have a better one for you [19:36:36] Dario's dashboard: http://test-reportcard.wmflabs.org/dashboards/features [19:36:53] and the github repository that supports that: https://github.com/wikimedia/limn-editor-engagement [19:37:11] ah nice [19:37:19] so one example datasource and its graph: https://github.com/wikimedia/limn-editor-engagement/blob/master/datasources/registrations_daily_live.json [19:37:27] https://github.com/wikimedia/limn-editor-engagement/blob/master/graphs/registrations_daily_live.json [19:40:25] milimetric: I can't find the registrations daily_live thing [19:42:25] as in [19:42:27] the graph [19:42:36] the graph itself [19:42:42] (on dario's dashboard) [19:44:35] oh, one sec [19:45:01] you can just put the id of the graph after /graphs/ like this: http://test-reportcard.wmflabs.org/graphs/registrations_daily_live [19:45:27] but it turns out it was on a different dashboard: http://test-reportcard.wmflabs.org/dashboards/metrics [19:46:10] aha [19:46:11] okay [19:48:44] ottomata, because you're missing the post-it board, there are about 80 stickies up on the board [19:49:15] milimetric: is there documentation on what the different 'nodeType's are? [19:50:30] ok [19:50:31] its cool [19:50:37] i posted my how might we's to hangout chat [20:22:18] is the meeting still on? [20:24:42] milimetric: hi [20:25:02] drdee: hi [20:27:32] yes [20:32:14] drdee: on g+ ? [20:32:43] yes [20:33:09] drdee: I tried this link https://plus.google.com/hangouts/_/2da993a9acec7936399e9d78d13bf7ec0c0afdbc [20:33:19] and teh hangout was empty [20:33:44] we are having lucnh [20:33:48] hold on [20:33:52] ok [21:01:03] hey YuviPanda, there's no documentation on the nodeTypes yet, but good question! The only way around that right now is to read the documentation in the source (we're currently working on compiling all that and making it available): https://github.com/wikimedia/limn/tree/develop/src/graph/node [21:01:33] ah [21:01:53] milimetric: that language is going to take a lot to get used to :| [21:07:49] YuviPanda yeah, totally, coco is very cool but has a learning curve. I meant more the short doc blurbs on classes and just the file names themselves. Most of them are pretty self explanatory, like AxisNode allows you to draw X and Y axes from the data, GridNode works the same way, etc. [21:08:09] for example, the ad-hoc graphs on the /datasources page have the following nodes: [21:09:04] AxisNode (x and y), GridNode (x and y), LegendNode, TweaksNode (with SmoothingNode and ScalingNode children), LineGroupNode (with LineNode children for each metric in the datasource) [21:09:17] and all of those are children of the CanvasNode [21:09:32] (still in a big meeting today and tomorrow but would be happy to talk after. [21:10:00] ;) [21:10:08] milimetric: thanks for taking time out :) [21:10:10] I'm installing limn [21:22:45] yuvi panda, are you using a labs instance or doing it on your own stuff? [21:22:50] YuviPanda*^ [21:23:02] ottomata: well, *right* now I'm testing on my laptop [21:23:09] aye ok cool [21:23:11] will migrate to a labs instance soon, I suppose. [21:23:20] there is a limn puppet module in for review right now [21:23:30] that makes spawning instances on labs pretty easy [21:23:32] oooh, so once that is done setting up a labs instance would be super simple? [21:23:37] yep :) [21:23:41] otto's the man [21:23:43] ETA on when that'll be done? [21:23:46] and average_drifter too [21:23:50] welp, it just needs review right now, i'm waiting on that [21:24:02] except, we've decided not to officially support the .deb package [21:24:16] so you have to deploy/put limn in the proper place on the labs instance [21:24:20] which isn't ideal, but meh [21:24:26] heh [21:24:28] but [21:24:34] puppet will set up limn as a service [21:24:35] like apache [21:25:05] hmm, i'll add some comments on how to deploy limn to the readme, the easiest way is just to clone it to /usr/lib/limn [21:25:15] if you do that, then the puppet instance will work just fine [21:25:29] ottomata: okay. [21:25:36] tl; dr; getting limn up on labs is really easy because I can do it very quickly. Soon we'll make it so anyone can do it very quickly, but for now just ask me when you're ready [21:25:41] haha :D [21:25:58] milimetric: can you get me one now? [21:26:05] timezonewise easier to have better than latter [21:26:06] *later [21:26:59] sure, do you have a domain name Yuvi? [21:27:19] something like test-reportcard.wmflabs.org [21:27:30] as in, a domainname on labs? [21:27:32] or a domain name in mind? [21:27:55] i can put it on an existing labs instance for now [21:28:11] but do you have a domain name to point to the new limn instance? [21:28:11] I can go create one now. [21:28:14] it's been a while since I've done that [21:28:30] ok, I'll make an instance on the kripke labs instance [21:28:59] sweet :) [21:29:03] oh, actually, tell me the domain name first so I can name it the same [21:29:45] YuviPanda ^ [21:29:53] mobile-reportcard? [21:30:06] sure, let's go with that for now :) [21:30:10] um [21:30:14] hm [21:30:26] I don't want people to think that the reportcard doesn't track mobile though [21:31:16] you're overthinking it :) [21:31:19] eh, it's ok, let's make it mobile-reportcard for now [21:31:22] yeah [21:31:30] :) you're probably right [21:31:33] brb, creating [21:31:42] http://data.linkedin.com/opensource/datafu [21:32:25] haha :) [21:33:56] http://data.linkedin.com/blog/2013/01/datafu-the-wd-40-of-big-data [21:41:47] http://linkedin.github.com/datafu/docs/javadoc/datafu/pig/urls/UserAgentClassify.html [21:45:38] hey YuviPanda, I've set up the limn instance and a repository for you to put graphs in: https://github.com/wikimedia/limn-mobile-data [21:45:58] woohoo sweet [21:46:00] when you've got the mobile-reportcard.wmflabs.org domain, let us know and dschoon will help us point it properly [21:47:30] milimetric: I just logged in and realized I can't actually create addresses [21:47:34] (no rights) [21:47:37] do you or should I find someone with? [21:48:04] nope, I don't have rights but dschoon might. Someone in labs might also be able to help you [21:50:30] do I have rights? [21:50:37] (never done this) [21:50:52] yes I do! [21:51:21] done! [21:51:27] oh [21:51:36] milimetric, YuviPanda, mobile-reportcard now points at kripke [21:51:45] mobile-reportcard.wmflabs.org [21:52:22] milimetric, can I delete the reportcard.pmtpa.wmflabs instance? [21:52:34] we need to recreate it in order to put it in the web security group [21:52:50] ottomata: wheee thanks :) [21:52:59] yup! [21:53:22] ottomata, yeah, that's fine [21:53:30] let me know if you want me to re-deploy to it [21:53:34] k danke [21:53:41] did you just clone to /usr/lib/limn? [21:53:44] or rather When you want me to redeploy [21:53:46] yeah [21:53:47] k [21:53:51] no I deployed with fabric [21:54:19] which is now patched to work with upstart so it should be quick [21:54:37] ok cool [21:54:47] well, which comes first, puppet or deploy :p? [21:58:17] milimetric: did you give me permission to the kripke project? [22:03:25] milimetric: what's the exact name? [22:03:29] and can you add me to the project? [22:05:30] YuviPanda, we're all in a meeting righit now [22:05:33] and they split us into groups [22:05:51] yeah, I understand. Do respond when you can :) [22:05:54] also metings, yay [22:06:05] also meetings, ____ [22:06:50] :P [22:06:52] :D [22:07:03] ottomata: do add me to the group when you can sneak some alt-tab time away. [22:07:45] are you YuviPanda on labs? [22:07:51] ottomata: yuvipanda [22:07:51] yes [22:08:37] done [22:09:51] tada, all done :) [22:09:52] thank oyu [22:47:00] yeah, YuviPanda milimetric I can add the address to kripke if that's where we're running it. is it? [22:47:16] oh, nm. looks like it's done [22:47:22] yeah, ottomata did that