[00:33:52] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review: Deprecate flower - https://phabricator.wikimedia.org/T137003#2354955 (10Ladsgroup) https://github.com/wiki-ai/ores-wikimedia-config/pull/61/files [01:39:05] 10Revision-Scoring-As-A-Service-Backlog, 10MediaWiki-extensions-ORES: Set up periodic maintenance jobs for ORES cache - https://phabricator.wikimedia.org/T115785#2355013 (10Ladsgroup) >Proposed job schedule: >* CheckModelVersions should run something like hourly. This period determines how much data will be i... [01:39:27] 10Revision-Scoring-As-A-Service-Backlog, 10MediaWiki-extensions-ORES: Set up periodic maintenance jobs for ORES cache - https://phabricator.wikimedia.org/T115785#2355014 (10Ladsgroup) 05Open>03declined [01:39:29] 06Revision-Scoring-As-A-Service, 06Research-and-Data, 05MW-1.27-release-notes, 13Patch-For-Review, 03RD-2016Q2: Write prototype MediaWiki extension to surface ORES scores - https://phabricator.wikimedia.org/T112856#2355015 (10Ladsgroup) [01:44:46] 06Revision-Scoring-As-A-Service, 10Wikilabels: Sometimes, tasks get more labels than they should - https://phabricator.wikimedia.org/T131997#2355016 (10Ladsgroup) a:03Ladsgroup [03:39:07] (03CR) 10Ladsgroup: [C: 032] "Tested and it works just fine. I have waited for three days." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/292163 (owner: 10Ladsgroup) [03:40:34] (03Merged) 10jenkins-bot: Add extra_param to pass to the ORES service and use precaching [extensions/ORES] - 10https://gerrit.wikimedia.org/r/292163 (owner: 10Ladsgroup) [03:40:36] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES: ORES extension should send a precache=true to the service when an edit is made - https://phabricator.wikimedia.org/T137007#2355042 (10Ladsgroup) [14:14:31] halfak: o/ [14:15:55] Hey! [14:16:09] Strangely, I can't find the original pre-labeled 20k dataset [14:16:17] On my machine or on ores-compute-01 [14:16:31] I must have used ores-compute-02 temporarily or something like that. [14:17:21] halfak: you added link to the 20K dataset in the makefile [14:17:25] you can simply make it [14:19:36] Yeah. Working on that now. [14:20:04] So, in the worklog, I find that 7.3k/100k need review. [14:20:39] But. That suggests that in a 20k set, I should find about 1.5k that need review. [14:20:48] Somehow I found 2.5k to combine into the 5k set [14:20:58] This is the inconsistency I want to look into. [14:21:51] yeah, you're right [14:24:50] Sure enough, i found 1.44k needing review. [14:24:52] So... crap [14:26:31] halfak: I also made a PR in ores-wikimedia-config and a commit in operations/puppet to deprecate flower. Once it's done we won't have any flowers anymore :) [14:26:51] Sounds good :) [14:27:20] once you get a clear sense of what's going on. I can fix all of it for you :) [14:27:48] so you work on the ores refactor [14:28:03] Would be nice to get some tests together today :) [14:28:10] (within ORES) [14:28:22] Been testing a lot in my demo repo :) [14:29:29] neat [14:34:39] OK. As far as I can tell, this set of "revisions_for_review" were not generated from the 20k sample we know and love. [14:35:01] So, here's my guess: I accidentally renamed the 100k sample to "...20k..." [14:35:06] And then we ran with that. [14:35:14] but I put the link for the 20k sample in the Makefile [14:35:25] So, I'm going to try to find the 100k query. [14:36:08] let me test that [14:38:08] let's see if True cases can be found in the 5k sample [14:39:46] I just checked. they can't [14:40:01] I am running prelabel on the old 100k sample now. [14:40:02] :) [14:43:51] nice :) [14:45:21] halfak: can you give me link to the 100K query? I will do the rev_reverted and other stuff on it (from 40% of it actually, I don't want to wait for hours to extract features and bring WMF servers to their knees :D) [14:48:34] OK.. So this isn't brilliant. It's still obviously not the same sample, but I think that is OK. [14:48:42] We'll just keep the original 5k sample checked in. [14:48:53] And try not to write over it ^_^ [14:49:18] sure :) [14:51:11] Amir1, https://github.com/wiki-ai/editquality/pull/33 [14:51:52] halfak: {{merged}} [14:51:53] 10[3] 04https://meta.wikimedia.org/wiki/Template:merged [14:53:58] halfak: Is it okay to work on a sample 40K revs instead of 100K? it would be too big [14:54:16] (I just sample it from the 100K sample) [14:54:59] Sure, that's fine IMO [14:55:16] kk [14:55:24] you'll have the PR very soon [15:22:34] Amir1, see like 68 of https://etherpad.wikimedia.org/p/ores_refactor [15:22:57] This describes the generalized scoring process and how some interesting "ScoringSystems" tie into it [15:23:11] brb [15:25:45] I have a question, what do you register in 2.5? revid and model pair? or just revid? i think that was our biggest issue in the sync meeting [15:37:19] revid/model pair. [15:37:28] Because they represent a "score" [15:37:48] Essentially, we are registering "these scores are in-progress -- no reason to generate the, again. [15:44:36] nice [16:08:12] 06Revision-Scoring-As-A-Service, 10rsaas-editquality: Train/test `reverted` model for nowiki - https://phabricator.wikimedia.org/T131856#2355402 (10Ladsgroup) Extracting features for 40K revisions right now in ores-compute-01 [17:01:41] halfak: the nowiki is still extracting features, I need to go to airport to pick up Amir, so I would be afk for a while [17:05:03] Hokay [17:05:14] o/ [23:50:09] 06Revision-Scoring-As-A-Service, 10rsaas-editquality: Train/test `reverted` model for nowiki - https://phabricator.wikimedia.org/T131856#2355957 (10Ladsgroup) ``` 2016-06-04 23:27:38,260 INFO:revscoring.utilities.train_test -- Training model... 2016-06-04 23:28:27,327 INFO:revscoring.utilities.train_test -- Te... [23:56:20] 06Revision-Scoring-As-A-Service, 10rsaas-editquality: Train/test `reverted` model for nowiki - https://phabricator.wikimedia.org/T131856#2355965 (10Ladsgroup) https://github.com/wiki-ai/editquality/pull/34 [23:59:06] going to sleep [23:59:09] o/