[04:04:28] YuviPanda, do you plan on dedicating much time in the near future to #17. I ask because I was looking into the details of it, but would rather not dedicate too much effort if you are already on it. [05:01:07] kjschiroo: yes, this week / next for sure [15:55:51] Hi everybody :) [16:15:52] o/ [16:16:02] Hi halfak :) [16:17:57] \o [16:33:41] o/ diligent13 [17:08:14] Empty agenda for the Research group meeting; looks like we're all getting an hour of our time back today! [17:11:39] Is there a place where I can get the move Log for English Wikipedia for all of 2014? [17:11:56] I mean, besides https://en.wikipedia.org/wiki/Special:Log/move [17:12:12] CristianCantoro: Yeah, you can get it from the dump [17:12:13] s [17:12:15] "Log events to all pages and users." [17:12:30] https://dumps.wikimedia.org/enwiki/20151201/enwiki-20151201-pages-logging.xml.gz [17:12:36] * CristianCantoro checks [17:12:37] 2.4GB [17:12:37] 2015-12-05 06:31:46 done Log events to all pages and users. [17:12:37] This contains the log of actions performed on pages and users. [17:12:37] enwiki-20151201-pages-logging.xml.gz 2.4 GB [17:12:43] It'll be ALL of the logs [17:13:34] Sweet [17:13:39] Thanks Reedy [17:13:52] Note, it will extract to a larger XML file :P [17:15:45] Thanks, should not be a problem. [17:16:38] http://i.imgur.com/PTpoTGv.jpg [17:19:14] guillom: eh eh [17:21:02] guillom: https://i.imgur.com/VGdpbpB.jpg [17:22:24] Emufarmers: why so snerk :p [17:22:31] :v [17:29:53] halfak: wow, I've to pay for the conference and specifically for the workshop too? [17:30:05] * YuviPanda does [17:31:06] it's like almost as expensive as a flight to europe [17:31:08] sigh [17:31:14] it'll be the most expensive conference I've gone to [17:34:53] YuviPanda, yeah :( [17:35:05] halfak: I got approval to expense it though [17:35:16] but that means I can't go to any other tech conferences this year [17:35:18] oh wlel [17:35:20] *well [17:35:50] * guillom needs to figure out which conferences to attend this semester. [17:36:46] halfak: are you sure we shouldn't be doing any notebook stuff at all? I somehow feel that's the way of the future, and Quarry a way of the past [17:53:53] YuviPanda, not sure for our workshop [17:54:14] ok [17:54:17] Quarry is still the future for our users. It's just that notebook is probably the next step. [17:54:26] Also, Quarry is proven (grew a community) [17:54:31] hmm, maybe I'm totally underestimating what you can just do with SQL :) [17:54:46] in which case it'll be a super nice learning experience for me [17:54:52] :D When I first got SQL access, I basically just ran queries for several months. [17:55:07] Then again, I exported my data and processed it in python often. [17:55:13] yeah [17:55:19] so notebooks would have the db extension [17:55:22] so you can just do [17:55:28] %%sql [17:55:30] your sql query here [17:55:32] and get output [17:55:34] and include in it [17:55:36] parameters [17:55:38] like [17:55:40] :somevariablename [17:55:42] and it'll expand [17:56:07] +1 I think that would work better. I'm just not sure I'd come to that conclusion if I didn't see Quarry first. [17:56:36] can you expand what you mean by 'that' in that sentence? [17:56:39] *what exactly [17:57:10] that = ipython + dbaccess. [17:58:07] halfak: did you have the chance of reading my email? [18:00:25] YuviPanda: I think that what we need is many examples and recipes (whether for Quarry or notebooks) for documentation and education. I personally feel that notebooks would be better suited for this purpose because you can mix documentation with code etc. I think if we do that people will naturally default to notebooks at some point, except for the simplest cases where you just want to run a quick SQL query. [18:00:47] * YuviPanda agrees with guillom [18:00:57] halfak: +1. I was going to work on dbaccess the coming week [18:01:03] (And doing this; yes, I would volunteer to be part of those who write those recipes.) [18:01:32] guillom: you can already do that kindof - notebooks are public already: http://tools.wmflabs.org/paws/public/YuviPanda/TeaHouse.ipynb [18:01:36] by default even [18:01:44] brb meeting [18:01:50] that's my TeaHouse analysis notebook using kjschiroo's awesome WikiChatter library [18:02:14] YuviPanda: Yep; but this quarter I won't have much time to work on this :/ [18:02:16] guillom: and you can kindof 'fork' it by downloading http://tools.wmflabs.org/paws/public/YuviPanda/TeaHouse.ipynb?format=raw into your own paws homedir [18:02:21] morning [18:02:24] guillom: yeah, and this needs more polish and stuff [18:02:29] _o/ Ironholds [18:02:29] ooh, WikiChatter is working and done? YAY! [18:02:37] I've been looking for a good codebase for that for this paper [18:02:43] guillom: my poking is mostly to make sure this is in people's heads all the time :D [18:02:49] marination of possibilities breeds ideas! [18:03:06] Ironholds: no we're still finding and fixing bugs - found one in our parsing library (mwpfh) [18:03:19] Ironholds: and also hoping to convince kjschiroo to not use regexes to parse wikitext, even a subset :D [18:03:20] "I just put a virus in your head. It's called an idea. Good luck getting rid of it." [18:03:31] YuviPanda, did you link them to the relevant StackOverflow? [18:03:46] Ironholds: that one is for html, right? [18:03:50] guillom: :D [18:03:53] * YuviPanda infects everyone [18:04:14] YuviPanda, yep! [18:04:23] but still, awesome to hear it's going so well [18:04:24] heh haven't [18:04:31] Ironholds: yeah, I'm using it to parse TeaHouse [18:40:57] YuviPanda, for the record you have convinced me not to use regexes [18:41:13] \o/ [18:42:26] kjschiroo: \o/ [18:42:28] <3 [18:45:10] YuviPanda, I'll probably be poking around over the next couple days to see about switching over to wikicode. I'm not sure how big of a change that is going to be. Although after doing some timing I'm thinking I could cut runtime by around 25% maybe. [18:45:42] woot [18:57:47] kjschiroo: halfak btw, I can parse all of teahouse with WikiChatter now [18:58:03] I edited the archives to fix broken markup after reporting them to mwpfh [18:58:14] \o/ [20:48:16] milimetric: hey, I hope you're around. Can you format values in y axis in this tool? https://analytics.wmflabs.org/demo/pageview-api/ [20:48:30] this would be helpful: http://stackoverflow.com/questions/2901102/how-to-print-a-number-with-commas-as-thousands-separators-in-javascript [20:48:58] I wanted to make a PR or something but I couldn't find source code directly available [20:49:02] Amir1: I could easily do that but we're not updating that demo [20:49:13] it's not even in a repo, it just lives in a gist [20:49:36] we want it to purposefully be flawed, since a few separate people are working on a permanent visual client to the API [20:49:48] and making this better would just cause people to think that we're maintaining it [20:49:56] hmm, Okay. It's a pretty cool tool why not making permanent [20:50:00] oh I got it [20:50:02] :) [20:50:07] the permanent tool will be much cooler [20:50:19] Please tell me if there is a beta to test [20:50:19] and if others fail at making such a tool, I will :) [20:50:34] np, there's nothing yet, just a few different people have plans [20:50:51] Tell me if I can help [20:51:29] btw: I'm building a simple GUI for ORES based on Semantic UI: https://tools.wmflabs.org/dexbot/tools/ORES.php [20:52:29] Amir1: you should probbaly put it in its own tool :) [20:52:49] It's not completed yet [20:52:55] I know that [20:53:46] YuviPanda: https://phabricator.wikimedia.org/T123348 [20:53:50] see this :)