[00:35:35] (03PS3) 10Cenarium: Defer changes matching specified thresholds [WIP] [extensions/ORES] - 10https://gerrit.wikimedia.org/r/326156 [00:36:44] (03CR) 10Cenarium: [] Defer changes matching specified thresholds [WIP] (032 comments) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/326156 (owner: 10Cenarium) [00:36:49] (03CR) 10jenkins-bot: [V: 04-1] Defer changes matching specified thresholds [WIP] [extensions/ORES] - 10https://gerrit.wikimedia.org/r/326156 (owner: 10Cenarium) [03:40:00] (03CR) 10Catrope: [C: 04-1] Defer changes matching specified thresholds [WIP] (032 comments) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/326156 (owner: 10Cenarium) [03:40:52] halfak: Do I understand correctly that ORES scores are not direct probabilities, i.e. that if an edit has a score of 0.91 it's incorrect/misleading to say it's damaging with 91% probability? [03:41:31] halfak: I ask because of https://gerrit.wikimedia.org/r/#/c/326156/3/i18n/en.json@29 , and because your precision/recall tables seemed to suggest to me that the scores are just numbers on a 0-1 scale and don't relate to probabilities or percentages except through those tables [04:13:04] (03CR) 10Catrope: [C: 04-1] Defer changes matching specified thresholds [WIP] (031 comment) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/326156 (owner: 10Cenarium) [15:23:16] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spike] Do we dare have ML assistance in meta-ORES? - https://phabricator.wikimedia.org/T153219#2876596 (10Halfak) [15:23:47] 10Revision-Scoring-As-A-Service-Backlog, 10AbuseFilter, 10ORES: [Spike] Suppression system for Meta ORES freeform text fields - https://phabricator.wikimedia.org/T153142#2876599 (10Halfak) p:05Low>03Triage [15:23:57] 10Revision-Scoring-As-A-Service-Backlog, 10AbuseFilter, 10ORES: [Spec] Suppression system for Meta ORES freeform text fields - https://phabricator.wikimedia.org/T153142#2871070 (10Halfak) [15:24:12] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spec] ORES query results should include Meta-ORES refutations - https://phabricator.wikimedia.org/T153143#2876609 (10Halfak) [15:24:44] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spike] [bike shed] Name the Meta ORES service - https://phabricator.wikimedia.org/T153146#2876611 (10Halfak) [15:24:50] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Meta ORES: API data storage and querying - https://phabricator.wikimedia.org/T153145#2876612 (10Halfak) p:05Low>03Triage [15:24:55] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spike] [bike shed] Name the Meta ORES service - https://phabricator.wikimedia.org/T153146#2871167 (10Halfak) p:05Low>03Triage [15:26:02] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spike] Should we integrate Meta ORES with Flow? - https://phabricator.wikimedia.org/T153147#2876618 (10Halfak) [15:26:05] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spike] Should we integrate Meta ORES with Flow? - https://phabricator.wikimedia.org/T153147#2871186 (10Halfak) p:05Low>03Triage [15:26:12] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Meta ORES: UI - https://phabricator.wikimedia.org/T153148#2876622 (10Halfak) p:05Low>03Triage [15:27:04] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spec] Design meta ORES general wiki entity matching schema - https://phabricator.wikimedia.org/T153149#2876623 (10Halfak) [15:28:10] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spec] Design meta ORES general wiki entity matching schema - https://phabricator.wikimedia.org/T153149#2871216 (10Halfak) p:05Low>03Triage [15:29:04] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Discuss] Could Wiki Labels backend be replaced with Meta ORES? - https://phabricator.wikimedia.org/T153150#2876628 (10Halfak) [15:29:10] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Discuss] Could Wiki Labels backend be replaced with Meta ORES? - https://phabricator.wikimedia.org/T153150#2871229 (10Halfak) p:05Low>03Triage [15:31:25] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: [Spike] Review ORES architecture - https://phabricator.wikimedia.org/T153321#2876634 (10Halfak) [17:31:48] o/ sabya [17:32:22] o/ halfak [17:32:56] How've you been? [17:34:11] great :) trying to get back to routine contribution schedule [17:35:28] what's new? Curious, Are we using the HashingVector anywhere in production? [17:36:49] sabya, not yet. I've been focusing on getting the PCFG work to a good place. [17:37:01] I'm just about settled on that. I'll have a writeup to show off soon. [17:37:47] But the hash vectors are on the horizon. I've been looking at them very carefully around the detox models [17:38:05] (predicting which user-to-user comments are personal attacks) [17:38:32] cool! [17:39:03] I think the hash vectors will be *huge* there. [17:41:41] makes sense. because 'angry' comments will have lots ngrams in common :-) people can't get too creative here [17:42:19] sabya, I've got high hopes for PCFGs too. Big problem there is that the sentence parser take 1.5GB of memory [17:42:45] For reference, our workers for extracting features currently have a 1GB footprint [17:42:48] O_O [17:42:55] Hash vectors should have a much better footprint. [17:44:21] where are we using PCFG? for flagging of article edits? [17:45:53] or user-to-user comments? [18:00:08] halfak: is there any plans you know of to build on the WikiWho project, or anything similar for providing authorship data for articles? [18:01:42] I'm trying to add something like WhoColor to the dashboard. I just rolled out showing articles from the dashboard: https://outreachdashboard.wmflabs.org/courses/University_of_Warwick/Human_Sexuality_%28Autumn_2016%29/articles [18:01:50] (click a row then click 'Show Current Version') [18:02:24] but I want to figure out how to show which content in the current article was authored by the participants. [18:03:01] stuck at the moment with CORS, and the wikiwho server is also pretty slow. [18:03:35] was hoping maybe there were plans to run it on wmflabs and build it out. [19:47:19] ragesoss, nothing right now. We'll need some serious funding for hardware resources in order to make such a thing tractable. [19:48:19] halfak: thanks. I'll keep hacking on it with wikiwho's api then. [19:48:25] I have code. There are serious performance issues with doing it at a large scale. We could hit a middle-ground with an on-demand service. I've experimented with that before. Still some hardware limitations (disk/ram/CPU) [19:48:49] halfak: one thing that could make it much easier is that I only care about revisions since time X [19:48:51] Been talking to the wikiwho folks aboutmoving to labs. [19:49:06] Regretfully, they don't believe in the vision enough to do the work. [19:49:30] :( [19:49:52] I also think that wikiwho is *distributable*, but not with how it is implemented. [19:49:54] I was emailing with Fabian this week, and it sounds like they are still planning *some* continued feature building. [19:50:05] i.e. we could use many machines to increase performance/availability. [19:50:21] yeah, totes, since articles are independent of each other. [19:50:22] If you push on him re. labs, I'd be interested in working with him. [19:50:29] okay. [19:51:38] I'll try to get it working as a user experience, then push on that for performance reasons. [19:54:39] ragesoss, keep me informed. I'll help as I can. [20:17:17] Arg. I just saw some questions from sabya that I missed due to lunch [20:17:19] :( [22:53:37] 06Revision-Scoring-As-A-Service, 06Design-Research, 10Research Ideas, 06Research-and-Data, 10Wikimedia-Developer-Summit (2017): Evaluating the user experience of AI systems - https://phabricator.wikimedia.org/T149373#2750321 (10srishakatux) @Capt_Swing Hey! As developer summit is less than four weeks fro... [22:57:47] 06Revision-Scoring-As-A-Service, 10Research Ideas, 10Wikimedia-Developer-Summit (2017): Where to surface AI in Wikimedia Projects - https://phabricator.wikimedia.org/T148690#2730132 (10srishakatux) @Halfak Hey! As developer summit is less than four weeks from now, we are working on a plan to incorporate the... [23:20:42] 06Revision-Scoring-As-A-Service, 10Research Ideas, 10Wikimedia-Developer-Summit (2017): Where to surface AI in Wikimedia Projects - https://phabricator.wikimedia.org/T148690#2878858 (10Halfak) @Fjalapeno and @jmatazzoni, any interest in taking the lead here? I've got a lot of other obligations for the dev s... [23:28:36] 06Revision-Scoring-As-A-Service, 10Research Ideas, 10Wikimedia-Developer-Summit (2017): Where to surface AI in Wikimedia Projects - https://phabricator.wikimedia.org/T148690#2878908 (10JMinor) ^ also potentially @dr0ptp4kt who is also digging in these areas. If none of them want to, I'd be willing to be a ba... [23:30:19] 06Revision-Scoring-As-A-Service, 10Research Ideas, 10Wikimedia-Developer-Summit (2017): Where to surface AI in Wikimedia Projects - https://phabricator.wikimedia.org/T148690#2878909 (10Halfak) Oh yeah! Of course! Thanks @JMinor.