[15:29:30] Hello, world of science. [15:30:17] hello guillom [15:30:26] _o/ harej [15:30:28] Having replaced my bullshit with a proper multiclass classifier, I am now up to 58% accuracy [15:30:40] harej: yay! [15:30:43] harej, nice! [15:30:51] Counter({3: 1378, 2: 238, 1: 7}) [15:30:58] Is that better than your former guesstimate? [15:31:10] It predicts zero articles top importance, seven articles high importance, 238 articles mid importance, and 1,378 as low importance [15:31:13] o/ hey folks [15:31:19] halfak: Welcome back! [15:31:21] guillom: prior guesstimate was 53-55% [15:31:35] Thanks guillom [15:31:46] * halfak goes through email backlog and VE experiment questions. [15:31:53] I am wondering if the n is too low to be able to properly predict top or high importance. I am going to test it out now with football, where *all* the numbers are big. [15:32:07] harej, what kind of model are you using. [15:32:14] What's the N of each class you are training on? [15:32:25] halfak: There hasn't been a day since you left on vacation where someone didn't say "Where the hell is halfak when you need him" in this channel. [15:32:32] ha! [15:32:41] I had 27 notifications and not enough scrollback to find them all. [15:32:59] halfak: https://www.irccloud.com/pastebin/5YEevluo/ [15:33:00] thats cute [15:33:02] halfak: :) [15:33:07] halfak still kayaking? [15:33:20] halfak: there was a major multi day labs NFS outage. ORES was untouched throughout, labels went down though. I brought labels backup today. [15:33:22] ircing while kayaking would be hardcore. :D [15:33:41] White_Cat: What could go wrong? [15:33:49] YuviPanda, thank you for bringing up labels. [15:33:57] That pastebin is reality; predictions were 0, 7, 238, and 1378 respectively [15:34:21] guillom umm typos probably [15:34:28] YuviPanda, do you have some docs I can follow to convert Wikilabels into something that'll bring itself up/something you'll be able to work more easily with? [15:34:56] halfak: not at the moment - we hadn't quite decided if we wanted to move it to tools or keep it on its own instanc.e.. [15:35:06] White_Cat, canoe :P A canoe is sort of like a water truck. A kayak is more like a personal transportation device. [15:35:08] and I'm still up to my neck in cleaning out all damage from last week's outage... [15:35:25] YuviPanda, I'll follow your lead. :) [15:35:49] I can imagine. I saw your "NFSNFSNFS" nick switch in the scrollback ;) [15:36:07] hehehe :) [15:36:48] halfak: It is a one-vs-rest classification system [15:36:50] scikit learn! [15:37:36] harej, but what model? [15:37:41] random forrest? [15:38:07] Linear SVC? [15:38:38] Gotcha. Seems like that should work OK :) [15:38:47] What kinda predictors do you have? [15:38:58] Behold! https://github.com/harej/wikiproject_scripts/blob/master/predictor.py [15:39:17] I would want two measures of pageviews. One for a long-term average and another for the tallest spike. [15:39:36] harej, I'ma convert you to revscoring :) [15:39:45] Ohhhh? [15:40:47] It would be good if we could stand up a PV stats service after the PV service comes online that will cache and give you these numbers. [15:41:19] Right. The plan is to throw out my current page view cruncher in favor of the new API [15:41:29] halfak IS BACK [15:41:31] YAAAAAY [15:41:33] WE MISSED YOU [15:41:42] o/ Ironholds [15:41:42] halfak, you missed so many things! [15:41:43] :) [15:41:49] What things?!? [15:41:50] :D [15:41:53] I turned into a Python user, we hired me an assistant [15:41:58] lotsa things [15:42:06] also I was Backup Halfak when you were out JFYI [15:42:09] (Though for testing purposes, I've written all the page view numbers to file once and have read from that file; not necessarily timely data but fine for my purposes [15:42:14] I went around being sensible and kind and freaking people out [15:42:38] Ironholds, lol woot! CONVERTS [15:42:41] Who is this assistant? [15:42:52] Mikhail! [15:42:59] henceforth "backup oliver" [15:43:10] except in machine learning where I am "backup mikhail" [15:43:17] Cool! :) [15:43:22] How long until he starts? [15:43:27] 27th July! [15:43:32] which is good because I will be back by then [15:43:42] https://github.com/Ironholds/floccus - seriously, I'm writing an actual module [15:44:00] did someone say machine learning?????? [15:44:26] harej, yeah we're getting an ML nerd [15:44:29] and by we I mean search [15:47:26] * harej waves Nettrom [15:48:01] * Nettrom waves harej [15:48:19] o/ Nettrom [15:48:24] harej: I plan to carve in a couple of hours of programming today so WP X can get some suggestions [15:48:29] hurrah! [15:48:32] not sure I'll get it working, but there'll be progress [15:48:36] halfak's back! [15:48:54] Nettrom: the only thing I really need is for the bot to be able to accept requests from outside user space? [15:48:59] it works really well otherwise! [15:49:40] harej: that's basically what I'm doing, sub-classing my existing code, grabbing suggestions from the template or from your config URL, then posting with it's own template [15:50:44] Sounds delightful [15:51:09] it'll also be Python 3, and on GitHub :) [15:51:15] WHOOOAAAA [15:51:28] I'm sure halfak is happy I'm starting to write Python 3 code and ditching 2.7 :D [15:51:38] I am very happy too. I am the zealous Python 3 enforcer. [15:51:58] Nettrom, welcome to a decade ago :P [15:52:14] halfak: I'm old and curmudgeonly, it's fitting :P [15:52:18] Python 3 revolution is slow, but it will come just in time to screw over python 4. [15:52:19] :) [15:52:41] * halfak is not aware of plans for python 4. [15:53:47] I... need to add python3 uwsgi support to toollabs at some point [15:54:42] YuviPanda: thanks again for getting scipy installed yesterday, it's working beautifully and I got something checked off my todo list :) [15:55:10] * harej managed to get around scipy dependency hell by installing the Anaconda stack [15:55:40] Nettrom: :) feel free to open bugs / poke me for new python3- installs [15:56:28] YuviPanda: thanks, will do if there's something else I come across :) [15:56:36] cool :) [16:05:40] signs you're Jewish: [16:05:53] you go out for drinks with a friend and realise halfway through the conversation that you're related [16:06:41] Why would that be a sign of Jewish and not a sign of small town? [16:06:50] Ironholds, ^ [16:07:13] halfak, she's New Jersey born and bred and I'm from the UK? [16:07:26] but we're both Katzenellenbogen, turns out [16:07:44] That's a word. [16:07:54] https://en.wikipedia.org/wiki/Katzenellenbogen [16:10:33] STAHP SENDING ME EMAIL [16:10:41] halfak, okay, I will stop [16:10:43] It's accruing faster than I can clear out the old stuff. [16:12:08] * guillom has been strategically waiting for halfa.k to work through the initial backlog before sending the emails. [16:12:53] +1 guillom [16:13:00] <3 guillom [16:13:13] halfak: It's completely self-serving :p [16:13:15] I'm going to wait week or so :) [16:13:15] Great way to make sure I don't just say "Cool story" in response to your email. [16:13:48] "Hey halfak, can you analyze Foo for me? I think it might be Baring." "Cool story. See ya." [16:13:55] Halfak what ratio is from cranky Wikipedians complaining about the VE study? [16:14:07] Not many surprisingly. [16:14:32] For all the trouble I got into the last time I ran this study, the response for this one has been surprisingly good. [16:14:56] halfak, I think that reflects on "the software does not blow quite so many chunks this time" [16:14:58] but we're both Katzenellenbogen, turns out < Pardon my lack of culture, but for a moment I thought that was a German word to say something along the lines of "allergic to cats". [16:15:04] I've got a few methodological questions. Nothing terribly bad. More "why did you choose this strategy" and less "I think this is wrong" [16:15:14] although Jason Quinn spent an extensive amount of time explaining how research worked to me [16:15:28] and then asking about my work on the VE, because obviously I could only like it if I had a COI from developing it or researching it [16:15:48] guillom, it's a clan of German-Polish Jews [16:15:55] Ironholds, sounds good. Does Jason do courses I could sign up for. ;) [16:16:00] Ironholds: Yup, I saw the article :) [16:16:02] many, many famous rabbis and also the only Jewish King of Poland! [16:16:09] He was King for a day [16:16:11] Thus "for a moment". [16:16:37] halfak, I don't think so, but you'll be happy to know that he can explain what you meant to you [16:17:11] "it's no worse than wikitext for new editors" ACTUALLY means "at no point in a editor's life cycle is it better and there are no possible explanations for why we're seeing a lack of a significant improvement in the 2-week period" [16:17:32] he's actually really more proud of your research than you are, insofar as he refuses to believe alternate hypotheses could exist [16:18:04] Ironholds, oh great. Next time I finish the preliminary analysis, I'll have him dictate the write-up to me. That would save a lot of time. [16:18:22] totally! [16:18:39] halfak, actually, can I run an alternate hypothesis by you? [16:18:47] it seems obvious but I wanna make sure I'm not missing something [16:18:56] Also, it's nice to know that my hypotheses carry so much weight. I hypothesize that world hunger has ended and that the large governments have suddenly started getting it Right(tm). [16:19:07] Ironholds, totally [16:19:11] I hypothesise free pizza! [16:19:23] hangovers make one's life goals smaller and more immediate [16:19:55] halfak, "one reason for why we don't see a substantial improvement in the early time is that a certain proportion of newly registered editors are switching from other accounts or from anonymous editing, meaning there is a degree of re-acclimatising going on" [16:20:13] +1 [16:20:17] I think that's right. [16:20:31] It's one I threw at Abbey. I really want to get a better understanding of anons. [16:20:41] nothing complex or controversial, but it seems very plausible and while we can't do it to say "okay we have z-test results but NO HYPOTHETICAL" we can actually say "it's no worse, and actually might be a lot better, it's just hindered by the lack of uniform exposure" [16:20:53] I think that fiddling with VE is mostly done by people who have some experience in Wikitext [16:21:13] yeah, which is great (everyone except me should eventually switch over) but doesn't tell us how genuinely new people handle it [16:21:50] +1. I tried a few different strategies for weeding out the not-genuinely-new people, but there didn't seem to be a nice way. [16:22:02] E.g. looking at editors who edited in the Wikipedia namespace on their first day. [16:23:23] yeah [16:23:37] but there's no good way of doing it because what we'd need to be able to do is well, look at off-wiki stuff [16:23:41] we cannot yet see inside users' heads [16:23:58] (and as one of those users: even if you could you really don't want to. Thar be swamp dragons and brainweasels) [16:24:06] step 1: Integrate facebook login [16:24:36] step 2: unique ID dot surgically implanted in each human that is required for computers to let them internet [16:24:40] step 3: MJ12 [16:24:50] step 4: ... [16:24:56] step 5: PROFIT [16:25:31] Seriously though, we can run some short-term observations by assigning anons a unique cookie and purging it quickly. We've done that before. [16:25:45] I wish I had more time to work with that old dataset... but we purged it :) [16:26:45] Ironholds: we already do a variant of step 2? [16:26:46] :P [16:27:52] 1) log out of accounts. 2) get yuvi's unique ID implant 3) trick Wikipedia into thinking I am a newcomer 4) write a parser function and throw off all the metrics [16:50:47] o/ J-Mo [16:51:00] welcome back halfak! [16:51:30] Thanks dude. [17:04:21] an inherent problem with trying to fit into pre-existing importance categories: [17:04:28] the "top" category is always going to be small, even on a large project [17:05:26] 40 articles of top importance to wikiproject football (n=137,920) [17:05:56] 19 articles of top importance to wikiproject women scientists (n=1,639) [17:06:31] football has 84 times the articles but only twice as many top importance articles [17:24:22] hello halfak. question about Q1 goals as I'll be filling in for Dario in a meeting today. [17:24:37] o/ leila [17:24:40] cool what's up? [17:24:44] is it fair to say that the work on productizing revscoring and wiki labels has already started? [17:24:51] or are you planning to do that in Q1? [17:26:13] halfak, ^ [17:27:52] It has already started and will continue full-steam into Q1. [17:28:00] leila, ^ [17:28:32] and about measuring value added (stretch): is it fair to say that you have a way for assessing contribution value, and you want to see if you can characterize the editor population based on the value they've added to Wikipedia over time? [17:28:40] makes sense, halfak. [17:29:26] leila, indeed. That sounds accurate. There are many facets of that project, but for a blurb, that'll suffice. [17:29:48] ooki. great. and one last thing in case that comes up: is the latter project across languages? [17:29:54] (I understand that the former can be) [17:30:43] halfak, ^ [17:33:14] leila, yes that's right. The method is not concerned with semantics of language -- just simple syntactic properties. E.g., I think we'll have to work out some tokenization issues for languages that do not use spaces for atomization of "words" -- e.g. Mandarin. [17:33:30] But all western languages will work out of the box. [17:33:55] I don't expect to have issues with RTL ones either, but we may need to adjust some parameters in the diff algorithm. [17:39:46] harej, indeed. this will be an issue, but it is reality, so you classifier will need to reflect it well. [17:40:17] In the revert modeling work, I've re-balanced the classes when using the linear SVC so that could get more realistic probability estimates. [17:40:57] And what did you do to rebalance? [17:42:12] harej, I experimented with downsampling the larger class (not reverted) and oversampling with replacement in the smaller class (reverted) and got similar results. [17:42:29] I chose to oversample the smaller class. [17:42:48] That's an interesting idea. [17:42:55] For it to work on my end, I would actually have to do... sampling [17:43:01] I'm working with entire WikiProject populations at the moment. [17:44:11] No problem once you have a datastructure that contains and