[15:37:21] o/ guillom [15:37:27] still looking for tags? [15:40:55] https://github.com/halfak/mwrefs [15:44:02] hey halfak [15:44:06] wow [15:44:20] I'll start a run right now if you give me the word. :) [15:45:33] halfak: I just told James_F he was crazy, because he and his team had official work meetings in the office yesterday night. I'm very appreciative of what you've done but I feel the need to tell you that you're crazy too ;) [15:45:46] Thank you so much [15:46:06] Yes, I'd love to see the results! [15:46:30] guillom, no worries. I was working on something related anyway, so it wasn't any trouble to do what you needed too. [15:46:43] OK, /me kicks off script. ETA 3 hours. [15:47:28] Well, I appreciate it. It would probably have taken me a few days to do something similar, and it would have been ugly, so thank you! [15:49:16] guillom, did I see some feedback from Erik on a talkpage somewhere? My email has lost the notification [15:49:51] https://meta.wikimedia.org/w/index.php?title=Research:Wikidata_gap_analysis&diff=next&oldid=11801271 wait no [15:50:24] hey Ironholds; I didn't see anything. [15:52:28] yeah, my early morning brain fooled me [15:52:28] ta! [16:17:16] morning J-Mo :) [16:17:43] afternoon, Ironholds! [19:02:25] Ironholds: [19:03:27] hellloooo? [19:12:16] Ironholds is busy celebrating Patriots' Day, I assume [19:24:24] Emufarmers, off by one error? [20:08:34] halfak: for the 'reverted' model of rsaas, does it just operate on the diff, or it is based on the whole article before and after? Also, does it assume mainspace, or is it also trained for predicting reversion of talk page edits? [20:19:16] halfak: you should set up a cage fight between the 'reverted' model and the wp1.0 model: find edits that simultaneously have a high 'reverted' score and also substantially increase the predicted quality. [20:19:58] ragesoss, predicts reverts cross namespace. Most features used focus on the diff [20:20:24] re. comparing "quality changes" with revert, I think we'd find the hairy corners of the decision tree, but it would be fun :) [20:20:44] e.g. someone copy-pastes a "" tag 100 times. [20:21:22] halfak: exactly. I'd expect to find a lot of cases where the model just doesn't really capture quality properly, but maybe it would also surface niches of productive edits that are overly likely to be reverted. [20:24:24] halfak: I was suprised to see my first edit (to a talk page) get 95% true for reverted: http://ores.wmflabs.org/scores/enwiki?revids=18154574&models=reverted [20:24:26] +1 Should be easy and I look forward to critiquing both algorithms :) [20:25:28] I guess lack of a real signature is probably what gives it such a high revert score: https://en.wikipedia.org/w/index.php?title=Talk:Modern_evolutionary_synthesis&diff=prev&oldid=18154574 [20:25:33] ragesoss, interesting. It was only trained on recent data (which might be related), but I don't see a good reason why this would be picked up. [20:26:59] ragesoss, I think that we're also catching things that look like reverts, but are really just talk page archiving activities. [20:27:15] We'll get those cleared up when we have a manual hand-coded dataset to train on :) [20:27:39] halfak: my later, similar posts to talk pages that are signed properly have like 20-30% reverted. [20:28:14] hey! notconfusing is Max Klein, right? [20:28:20] does anyone know what tz he’s in [20:28:27] YuviPanda: yes, that's Max. [20:28:37] ragesoss, we can make some changes and ask the classifier to try again. It requires a little bit of setup, but we could ask "what if the diff included a signature". [20:28:42] YuviPanda: he's normally in Pacific. [20:28:53] ah ok [20:28:57] he broke tools-redis :P [20:28:58] * YuviPanda emails [20:29:29] ragesoss, it might be fun to stand up a web UI that allows you to request a revscore and then play with the input like that. [20:29:48] It would be good for critique. :) [20:29:57] One of the little things on my todo list is to come up with ideas for a large-group icebreaker activity for Wikiconference USA. I think it would be fun to have people find their earliest edit, and check the reverted score. then we can have people raise hands if they had over 50% 'reverted' score (and 70%, and 90%). It would be maybe a fun way to get people thinking about the newcomer experience of 2015. [20:31:36] ragesoss, that would be great! I'd love to hear people's thoughts on why they got scored highly and whether the algorithm was "just plain wrong", "wrong for a good reason" or "mostly right". [20:32:23] These classifiers are kind of bug and dumb for being an "advanced AI" The academic side of me wants to explore the implications for that. [20:32:34] none of my early edits are under 50%. [20:32:52] Well I guess you were just a big vandal when you started :P [20:33:07] indeed. and I turned out okay. [20:33:15] and then you became a visigoth? [20:33:17] Really, I think that the editor experience features might be overly weighted. I plan to remove that. [20:33:29] EVen if it pushed AUC, the bias is too strong. [20:33:39] AUC == solid fitness metric for the classifier. [20:33:58] AUC? [20:36:39] Area Under the Curve (For the Receiver Operating Characteristic) [20:36:52] https://en.wikipedia.org/wiki/Receiver_operating_characteristic [20:38:01] as in, AUC should match up with the observed reversion rate? [20:38:19] Rather, AUC should be as close to 1 as possible. [20:38:43] That would mean that we we label all the True's True and all the False's False [20:39:05] okay, right. [20:39:07] False positives and True negatives lower the curve [20:56:27] halfak, FYI: AbuseFilter has a special page with that kind of UI for playing with the settings of a filter until we get its code right... [20:56:35] it is at pages such as https://pt.wikipedia.org/wiki/Special:AbuseFilter/examine/log/2103235?uselang=en [20:56:41] (restricted to some users) [21:03:35] Regretfully, my "WMF researcher" right will let me do an aweful lot of things, but viewing that page is not one of them [21:04:00] Oh wait.. I take it back :) [21:07:12] Helder, I'm still working on the ores refactoring. [21:07:26] I have to rebuild the models in order to re-deploy the server :( [21:07:48] I'm generating reverts and features for ptwiki now. [21:08:29] ok [21:11:17] Sorry for the delay. :\ I should have this ready by tomorrow. [21:11:40] We'll want to look at automating rebuilding all the models later so that we don't have to do this manually. [21:12:13] We can do a lot with a makefile. I'll look into setting that up. [21:20:48] halfak, if you could only point someone at one open data org to learn about why it's important and what they should do, what would it be? [21:20:56] *what they can do to help, [21:21:51] Who is the "someone" [21:21:52] ? [21:22:16] AOL [21:22:31] AOLOL [21:27:03] halfak, random twitterati [21:27:05] YuviPanda, groooan [21:27:49] Ironholds, are they researchers who are open a dataset? [21:28:00] *considering opening a dataset [21:28:35] more, people who don't get why open data is a BFD [21:29:15] Oh... Hmm. I dunno if I know of one good website for this. [21:29:23] Or really any good website. [21:30:46] * Ironholds nods [21:31:00] I'll use http://opendatahandbook.org/en/what-is-open-data/ [22:36:43] halfak: ragesoss do you guys have any out of band methods of getting in touch with notconfusing? [22:37:37] YuviPanda, you did email, right? [22:37:42] yup [22:37:55] https://phabricator.wikimedia.org/T91979#1218211 is fairly high priority since it means nobody else can use tools redis [22:38:06] and I’m tempted to just clear all of his keys but maybe he’ll lose data... [22:38:37] YuviPanda, k. Looking for other ways to contact him. [22:39:31] * halfak found a phone number that might be relevant. [22:42:18] YuviPanda, Just talked to him. OK to dump the keys [22:42:37] Confirmed that he knows this will drop all of his data from redis [22:42:40] halfak: w00t, you’re awesome :D [22:42:47] * halfak flexes muscles. [22:43:00] I found an email from 2013 with his phone number in it :D [22:44:05] hahah [22:44:18] oh, halfak; http://ironholds.org/lsaas/ [22:44:23] It's in Python, too! [22:45:50] http://ironholds.org/lsaas/data/ "Method not allowed" [22:46:07] yep [22:46:11] it only accepts POST requests [22:46:32] we got some Oracle consultants to help us build the open data API and they told me get requests weren't ISO-compliant [22:46:39] then they charged me $80,000 and left [22:46:53] ;) [22:46:58] (/papers/ accepts GET) [22:47:40] * YuviPanda is slightly tired of all the puppet, is writing some nodejs on the side now [22:47:53] guillom, http://datasets.wikimedia.org/public-datasets/enwiki/etc/ref_diffs.20150304.tsv.bz2 [22:48:01] Should be self-explanatory. [22:48:27] Each row is a reference in the most recent revision the 20150304 dump. [22:48:30] halfak: neat! Thank you. I'll launch the download now. (I'm leaving the office so I'll dive into it tomorrow). [22:48:41] Have a good night! [22:48:42] o/ [22:48:47] You too :) [22:51:43] halfak, no laugh at the POST request? aw ;p [22:52:19] LOL Ironholds [22:52:20] :) [22:52:31] * halfak was doing too much multitasking :S [22:52:35] It was fun to build and twitter is amused [22:52:41] also I got to build Python web apps! \o/ [22:52:51] Flask is really fun. R is better at Markdown, though. [22:53:02] +1 at Flask is fun :) [22:53:17] * YuviPanda is building a version of github streaks but for wiki contributions [22:53:21] neat! [22:53:35] OK. I'm off too. Have a good night, folks! [22:53:36] o/ [22:53:51] I was gonna do that in scala [22:53:57] * YuviPanda but realized I should move a bit slower. so nodejs first :) [23:32:39] halfak: we moved it because we decided there weren't enough three-day weekends