[06:16:35] legoktm: hey, around? [15:35:18] * halfak is now an admin of Meta [15:35:21] \o/ [15:53:06] o/ ToAruShiroiNeko [15:53:07] Around? [15:53:27] If so, can you make sure that we're all set for https://meta.wikimedia.org/wiki/Talk:Wiki_labels#Wiki_labels_is_translated_into_Norwegian ? [15:53:50] halfak: congrats. :) [15:54:00] Thanks Amir1 :) [15:54:01] I've got to go for a while [15:54:14] please read those messages [15:54:16] OK no worries. I like your plan for getting the sample. [15:54:23] I'll follow & extend it. [15:54:39] basically everything is set once we have those rev ids [15:54:46] Amir1, I think so. [15:54:46] thanks [15:54:48] :) [15:55:10] Once I'm done let's do some other tasks [15:55:28] Reviewing the paper [15:55:29] etc. [15:55:31] o/ [15:55:47] +1 [15:55:56] o/ [16:23:17] OK. New query is ready and running. [16:23:41] * halfak dumps into a worklog. [16:26:09] wiki-ai/wb-vandalism#144 (sample_subsets - af84262 : halfak): The build has errored. https://travis-ci.org/wiki-ai/wb-vandalism/builds/107602148 [16:26:17] Shuddup travis [16:27:37] wiki-ai/wb-vandalism#143 (sample_subsets - a4cb331 : Aaron Halfaker): The build was broken. https://travis-ci.org/wiki-ai/wb-vandalism/builds/107602096 [16:53:02] FYI: updates being posted here: https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service/Work_log/2016-02-07 [17:26:49] awesome [17:27:04] what else is needed to do? [17:27:16] can you merge the PR halfak ? [17:27:41] Amir1, working on an update to it. Then will ask for your review [17:28:01] sure [17:30:19] halfak: Can you deploy the new ores to server? [17:30:25] it would be amazing [17:30:53] Amir1, sure. As soon as I kick off the next script, I'll have some downtime and will do it then. [17:31:07] thanks [17:45:55] * halfak adds some debugging to the script [17:46:05] Otherwise, it looks like it is doing what it is supposed to. [17:48:53] Amir1, please check out https://github.com/wiki-ai/editquality/pull/14 [17:49:16] Woops. Looks like I added some typos [17:51:04] Fixed [17:51:08] {{fixed}} [17:51:08] 10[1] 10https://meta.wikimedia.org/wiki/Template:fixed [17:51:13] {{done}} [17:51:14] How cool, halfak! [17:51:16] yay! [18:00:13] I merged [18:01:03] It's better to exclude 'comment' in rvprop unless needed [18:01:49] Amir1, I was thinking about that, but it seems like the performance hit will be minimal. [18:01:58] because it would take much ram, network traffic, etc. [18:01:58] let's improve it later [18:02:03] If you feel strongly, we can put it back. [18:02:04] halfak: [18:02:04] ^ [18:02:30] Amir1, still tiny amounts of RAM and network traffic. But I agree that it is avoidable. [18:02:32] It's ok for me [18:02:34] kk [18:18:24] Amir1, just finished the big query against wikidatawiki. [18:18:27] It took 61 minutes [18:18:35] So looks like quarry couldn't do it. [18:25:51] 22.6k of the 500k edits need review [18:30:08] * halfak fires off a script to label reverted edits [18:30:28] And we're getting about 15 revisions/second [18:30:52] So, this should take about 25 minutes :) [18:35:37] OK. Now to look at deploying an updated ORES [18:45:53] OK. Looks like all of the VMs are in a good state. [18:46:05] Time to update the master config and push to staging. [19:03:52] halfak: hey, IRCCloud is down again [19:03:58] Awesome [19:03:59] I was afk for dinner [19:04:50] * halfak moves to telegram [19:15:33] Amir1_: a little bit [19:15:58] legoktm: can we schedule a deploy ? [19:16:02] everything is ready [19:16:06] for fa.wp [19:16:21] that would be amazing [19:19:44] did we already do a security review? [19:20:29] yes [19:20:33] we passed [19:20:58] legoktm: https://phabricator.wikimedia.org/T120922 [19:21:52] woot [19:22:12] legoktm: we changed the extension a little bit [19:22:19] is it okay? [19:22:20] ok, I'll take a look after I'm awake [19:22:24] I think so [19:22:33] awesome [19:23:25] irccloud is back [19:23:58] \o/ [19:24:21] So yeah, Amir1, can you walk me through the feature subset work you did in wb-vandalism? [19:24:58] halfak: they are five models [19:25:07] one is only general metrics [19:25:45] the second one general metrics + specific to some kind of vandalism [19:26:12] Amir1, I don't see a PR or a branch. [19:26:26] let me find it [19:27:30] I think I pushed it [19:27:35] in another file [19:27:51] I've trouble opening github right now [19:28:37] I've problem right now on internet [19:29:47] maybe I haven't pushed it [19:30:18] branch sample_subsets [19:30:56] Oh!!! The one I've been working from. Great [19:30:58] * halfak pulls [19:31:32] OK. So our 500k sample has 698 reverted edits that are probably for vandalism [19:33:54] We're going to have an interesting time testing this. [19:34:02] I think I'll do a 50/50 train/test split [19:34:12] And see how that goes [19:35:06] awesome [19:37:00] halfak: I've got some suggestions we can work on later. Let's do an article quality models [19:37:11] For wikidata? [19:37:16] like wp10 but binary [19:37:22] not for Wikipedia [19:37:42] why? One of the biggest example is [19:38:35] we can sort list of articles based on their quality [19:39:23] halfak: e.g. someone wants to improve articles of a category and show the articles which needs it the most (lowest quality) [19:40:13] Amir1, seems do-able [19:40:25] Just so long as we can confirm that every article in a sample was reviewed. [19:40:37] But in that case, why not have a more granular scale. [19:41:36] we can get all of the FA + GA articles as 1 and some stubs (randomly sampled) as 0 [19:42:37] Why stubs? [19:43:14] low quality [19:43:23] the article which need love the most [19:43:23] The decision boundary might be placed immediately between stubs and FA such that a B article will be predicted as FA with high confidence. [19:44:27] yeah, I know. I don't want those numbers alone [19:44:31] I want to compare [19:44:56] Gotcha. Seems worth a try [19:44:57] it's like size of article but a better metric [19:45:09] Could do it on a wiki where we have a more cganular classifier already [19:45:15] To check how it works at scale. [19:45:40] +1 [19:46:08] So, how do we find stubs? [19:46:17] If we don't have a classifier already? [19:46:39] we have a category called "All of stub articles" or something like that [19:54:35] https://fa.wikipedia.org/wiki/%D8%B1%D8%AF%D9%87:%D9%87%D9%85%D9%87_%D9%85%D9%82%D8%A7%D9%84%D9%87%E2%80%8C%D9%87%D8%A7%DB%8C_%D8%AE%D8%B1%D8%AF [19:55:01] halfak: https://en.wikipedia.org/wiki/Category:All_stub_articles [20:03:20] we have 2M articles [20:03:21] Amir1, yeah, but what about on new wikis? [20:03:33] we can do it only some wikis [20:03:38] Gotcha. [20:03:50] So, fawiki has a FA review process? [20:03:53] and if someone wants it, we say give us this category [20:03:55] But now middle ratings? [20:04:02] *not [20:04:12] halfak: most fierce one for both FA and GA [20:04:29] but not middle one, we have it but in practice no one uses it [20:33:02] halfak: I need to sleep [20:33:10] assing something to me [20:33:13] *assign [20:33:28] for me to do in tomorrow [20:34:32] (email me if you see this too late) [20:50:16] Amir1, will do [22:17:46] Bummer. Found a blocking issue in pywikibase. Hmm. [22:18:08] https://github.com/wiki-ai/revscoring/issues/239 [22:18:17] Looks like this happens while formatting a coordinate. [22:31:43] wiki-ai/revscoring#514 (caught_error - e371114 : halfak): The build failed. https://travis-ci.org/wiki-ai/revscoring/builds/107658882 [22:31:50] Shuddup travis