[14:33:44] Hey Ironholds [14:33:49] uber analytics meeting! [15:26:26] R protip of the day: magrittr makes your code 20 times more readable. [15:32:22] hey halfak, let me grab some coffe, brb [15:32:26] Ironholds / DarTar: tnegrin suggested that we sync up a bit about the mobile dashboard stuff [15:32:28] Hokay [15:32:34] milimetric, okie-dokes! [15:32:36] * halfak runs to get a candy bar [15:32:48] just lemme know when so I can ty and get on a decent connection [15:32:57] * Ironholds may go over to the backup hacker collective for that [15:33:13] Ironholds: after lunch east coast? So like 13:00 EST? [15:33:30] lemme look at your calendars [15:33:47] oh yeah we're in the same timezone now! [15:33:49] hi from the north! [15:34:30] :) time looked clear so I sent the invite [15:34:38] feel free to reject - I'm gonna go get some food [15:35:08] cool! [15:35:10] ditto [16:03:50] hey leila [16:04:01] for some reason I can’t respond via DM [16:04:09] IRC tells me you’re not online [16:04:23] mmm [16:04:27] can you see here? [16:04:27] anyway, if you read me, feel free to respond to that thread [16:04:29] yes [16:04:39] okay. I'll respond. thanks! [16:05:01] (and I'll try to join for StandUp, if the connection cooperates. It's lunch time here, so I may be free.) [16:09:31] leila: sounds good [16:28:19] DarTar: I've got 5 minutes if you do. [16:32:21] DarTar, halfak, are you in Hangout? [16:32:30] Will be in a minute. [16:32:33] got it [16:32:37] coming [16:33:59] Ironholds: yt? [16:34:03] standup [17:37:48] darnit [17:37:58] I have an R problem in a class Leila is fricking great at and she's away :( [17:58:14] ping yuvipanda [17:58:42] * yuvipanda pings J-Mo [17:59:09] :) how close are you to implementing CSV download in Quarry? I was hoping to use that functionality in my webinar tomorrow, if it was available. [17:59:18] 'sup [17:59:50] if not, I can use Wikimetrics instead. [18:00:34] J-Mo: oh, can do by tomorrow, sure [18:00:36] basically, I want to guide users through the process of grabbing a CSV dataset from the slaveDB, and manipulating those data with Python, + adding in some related data from the API [18:00:38] J-Mo: let me hook it up [18:00:50] should be done in a couple of hours [18:01:01] yuvipanda: let me know if you get stuck on anything with CSV downloads [18:01:04] that okay? I don't want to throw an emergency deadline at you! [18:01:06] I can help [18:01:24] * J-Mo thanks milimetric and yuvipanda [18:01:25] J-Mo: nah, it's trivial. [18:01:35] milimetric: \o/ will poke if needed [18:02:06] J-Mo: there's a recurring bug I can't track down yet tho - if a query is 'queued' for more than 10s, please ask people to hit 'submit' again [18:02:10] cool cool! ping me if you need my help (for some reason), or if you have questions, etc [18:02:21] will do, Yuvi [18:03:03] J-Mo: cool :) also note that with python's default csv module, it barfs on Unicode CSVs, and you need to install the unicodecsv module [18:03:15] that'll trip people up if they're not using englishwiki for stats [18:05:01] good catch. I'll work with that module (I'm making test scripts for people to manipulate, rather than having them write Python from scratch). [18:05:23] cool [18:44:17] Ironholds, yt? [18:44:43] DarTar, yep! [18:45:01] so I had a thought about the referral stuff while I was in the shower [18:45:07] (be very afraid) [18:46:20] in the context of referred traffic we’ve been talking about PVs and deprioritized UC/UV because the sampled data is not the best source to answer that question [18:46:31] DarTar, yup [18:46:33] but I thought, what about unique articles? [18:46:38] * Ironholds thinks [18:47:02] that would be programmatically problematic to implement as part of the same dataset. But I could do it alongside. [18:47:04] I think it would be fascinating to determine the breadth of traffic we get from referred vs organic traffic [18:47:21] I could even build it out as an inequality coefficient or something [18:47:22] yeah, so it’s not as high a priority as simple PV counts [18:47:35] "here is the coefficient for referred traffic over time, here is the coefficient for organic" [18:47:42] I can make it into a nice little animated visualisation [18:47:47] yeah, so I was imagining that probably some segments of traffic by referral are really focused on a small subset of articles [18:47:49] I've wanted an excuse to play with ggvis for a while. [18:47:56] oh god, I take that back [18:47:59] :p [18:48:07] what? [18:48:14] * DarTar kidding [18:48:15] y u no liek animated gifs? [18:48:37] I just read in the changelog for the latest Dropbox iOS app: [18:48:51] “better support for high resolution animated GIFs” [18:48:56] WTF [18:49:38] anyway, what do you think about unique articles as a secondary metric to look into on a longitudinal basis? I think we would discover a lot of interesting things [18:50:11] sure. Do you want the specific articles, or the number, or the inequality? [18:50:21] like I said, I could have it save a list of coordinates for coefficients. [18:50:54] brb [18:59:06] oh sorry, by longitudinal I meant “historical”, not by geography [18:59:40] (unless I misunderstand what you mean by coordinates) [19:00:11] hey, I have to find Abbey for lunch, bbl [19:15:03] DarTar, nono, I know ;p [19:15:10] I meant coordinates in the sense of coordinate plotting [19:15:25] i.e., I give you a list, each element of which is a set of coordinates R can interpret as forming a geni coefficient in plot() [22:34:54] yuvipanda: you should really use my “fleur de yuvi” screenshot as your Trello avatar [22:35:02] DarTar: :D [22:35:05] DarTar: I could [22:35:29] I can’t talk to people who show up as YP on a grey background on Trello [22:35:32] DarTar: I hung out with qchris today, and he was surprised to see me in the same outfit ;) [22:35:33] hehe [22:36:58] :D [22:36:59] it’s the famous scottish draight of 2014 [22:36:59] draught [22:37:00] aaarg drought [22:37:00] seriously [22:46:10] DarTar: :D [22:46:37] Tuesday afternoon dyslexia :-/ [22:47:53] J-Mo: CSV and TSV download implemented, I just need to add a button now [22:48:06] sweet [22:48:11] thanks dude [22:49:58] J-Mo: \o/. When's the webinar? [22:51:21] 1500 UTC Wednesday. Same day/time as last week. [22:54:15] J-Mo: cool, I should be around [22:54:51] double sweet. I'll be directing people to this chan if they have questions, as usual. [22:56:08] J-Mo: cool [22:56:22] now to add buttons [23:18:52] J-Mo: so the download buttons themselves aren't done yet, will happen tomorrow [23:18:59] I've most of the code done but am being dragged to sleep [23:20:00] J-Mo: csv output does work tho http://quarry.wmflabs.org/run/988/output/0/csv (or http://quarry.wmflabs.org/run/988/output/0/tsv since Ironholds hates CSV) [23:20:02] no problem, YuviPanda. Send me a quick email if for some reason you can't finish them [23:20:06] I'll just need to add buttons [23:20:08] J-Mo: will do [23:20:08] oh@ good. [23:20:11] thanks! [23:20:18] nice to have a backup. Goodnight! [23:20:25] J-Mo: you can also download JSON, btw http://quarry.wmflabs.org/run/988/output/0/json [23:21:03] anyway, am off! cya [23:25:10] Ironholds: I was reviewing https://meta.wikimedia.org/wiki/Research:Mobile_trends and I noticed that you added a note on filtering bots for edits, but there’s no mention of filtering crawlers from the traffic dataset [23:25:34] I know you’ve done extensive filtering of bots, I think we should have this in the Data section of that report [23:25:49] DarTar, yes! shall do. [23:25:56] danke [23:27:07] DarTar, done! [23:27:23] sweet thanks [23:29:43] DarTar, also, JFYI, I don't know how around I'll be tomorrow [23:29:58] on account of I'm staying up all night to monitor the referer scripts and work on the apps stuff [23:30:13] because I live on a couch and don't exactly have anything to do until the 31st [23:30:15] ;p [23:30:41] sounds good to me [23:31:11] make sure you let da boss know [23:35:59] J-Mo: yay, got a few mins back. [23:36:04] J-Mo: and download buttons in place! http://quarry.wmflabs.org/query/354 [23:36:39] DarTar, totes [23:36:44] okay: noms! [23:36:51] I'm being taken to something called tasty burger(?) [23:37:39] this being Boston, I wouldn’t get worried [23:37:48] YuviPanda: nice stuff [23:38:30] YuviPanda: hmm, I'm not seeing the download link? [23:38:53] where on the page is it 'sposed to show up? [23:38:54] J-Mo: right above the table to the right. do a hard refresh (ctrl+f5 or cmd+shift+r) [23:39:31] there it is! [23:39:57] THAT IS JUST BEAUTIFUL [23:40:02] J-Mo: :D [23:40:05] * J-Mo weeps from happiness [23:40:19] future feature request: add the full query to a title element of each row in the Recent Queries table so you can quickly preview it on hovering [23:40:21] thanks for coming through in the clinch once again [23:40:28] J-Mo: \o/ [23:40:41] DarTar: https://trello.com/b/fdwhYLns/quarry [23:40:50] (for your feature requests) [23:40:58] kewl [23:42:29] done: https://trello.com/c/HyZSUsmU [23:42:54] J-Mo: I'll add a feature tomorrow that always downloads the latest successful run results, and then CORS headers, so people can preview things [23:42:58] err [23:43:04] people can write scripts and shit [23:43:06] I meant :) [23:43:10] I'm off now! [23:43:16] good night [23:44:02] night! [23:55:05] J-Mo: oh well, looks like I've some more time :) anything else you want? [23:55:19] nope. I'm good! thanks, though [23:55:21] J-Mo: also if you're teaching people python, I highly reccomend you ask them to use JSON than CSV/TSV. [23:55:22] GO TO SLEEP!!! [23:55:32] It's only midnight! [23:55:40] hehe. touche. [23:55:43] J-Mo: JSON also has no unicode problems and doesn't need an extra library [23:55:48] oh, it's actuall 1 AM [23:56:04] well, my girlfriend got sucked into a wordpress loop, so I've time until she realizes it's way past 5mins [23:56:10] re: JSON: I'm going to teach people what JSON is, but will also have thme manipulating CSVs, since a lot of people are more comfortable exploring data in spreadsheets [23:58:48] J-Mo: ah, cool. don't forget unicodecsv then, since the error message it gives otherwise is super confusing - "'ascii' codec can not decode ordinal '0xfe' at position 7' or something like that [23:59:08] yeah, I'm intimately familiar with that cryptic message :) [23:59:31] J-Mo: :D [23:59:54] * YuviPanda|zzz channels inner halfak [23:59:58] WE SHOULD ALL USE PYTHON3