[09:10:52] 10Scoring-platform-team, 10ORES: Reimage deployment-ores01 as Stretch - https://phabricator.wikimedia.org/T189790#4055744 (10akosiaris) I am guessing this was resolved and I am no longer needed. [13:42:32] o/ [17:13:46] halfak: i added 100 estimators to the tuning list but i should mention that it takes a lot of time to generate tuning report with that, its been running for 12 hrs and still on... [17:14:34] 10Scoring-platform-team (Current), 10drafttopic-modeling: Checklist for drafttopic repo - https://phabricator.wikimedia.org/T189797#4057094 (10Sumit) [17:16:40] codezee, gotcha. Still seems like it would be worth a test, I guess. [17:17:52] If it doesn't finish today, then I could attempt it on the big stats machines that have a little more than twice the power. [17:19:58] ok [17:21:09] * halfak just created a goddamn python package for the stupid dict merge utility we seem to need everywhere [17:22:50] https://github.com/halfak/deep_merge [17:56:41] Nice one! [18:00:14] awight, I'll make a revscoring PR that'll add a new utility and simplify some others. [18:00:21] The new utility will be important for huwiki [18:01:43] ok neat [18:24:08] (03PS1) 10Ladsgroup: Remove already done TODO [extensions/ORES] - 10https://gerrit.wikimedia.org/r/420091 [18:26:00] (03PS2) 10Ladsgroup: Remove already done TODO [extensions/ORES] - 10https://gerrit.wikimedia.org/r/420091 [19:21:05] (03CR) 10Awight: [C: 032] Remove already done TODO [extensions/ORES] - 10https://gerrit.wikimedia.org/r/420091 (owner: 10Ladsgroup) [19:21:30] Amir1: btw, https://gerrit.wikimedia.org/r/#/c/419856/ [19:21:59] The conclusory patch should probably be deployed separately, https://gerrit.wikimedia.org/r/#/c/419857/ [19:23:08] yeah [19:23:12] (03CR) 10Ladsgroup: [C: 032] Remove old thresholds syntax parser [extensions/ORES] - 10https://gerrit.wikimedia.org/r/419857 (https://phabricator.wikimedia.org/T181159) (owner: 10Awight) [19:23:21] (03CR) 10Ladsgroup: [C: 032] Update default thresholds to new syntax [extensions/ORES] - 10https://gerrit.wikimedia.org/r/419856 (https://phabricator.wikimedia.org/T181159) (owner: 10Awight) [19:23:47] awight: https://gerrit.wikimedia.org/r/#/c/420091/ https://gerrit.wikimedia.org/r/#/c/419798/ :D [19:23:47] oh um... I was trying to say, they should be deployed separately. [19:23:54] (03Merged) 10jenkins-bot: Remove already done TODO [extensions/ORES] - 10https://gerrit.wikimedia.org/r/420091 (owner: 10Ladsgroup) [19:24:06] awight: I +2'd the wrong one, and removed it :D [19:24:16] :L hehe [19:24:21] oh you already merged one, thanks! [19:25:19] (03CR) 10jenkins-bot: Remove already done TODO [extensions/ORES] - 10https://gerrit.wikimedia.org/r/420091 (owner: 10Ladsgroup) [19:27:00] (03CR) 10Awight: [C: 032] "It's scary that we're not actually testing all 3 migration phases, but the changes look good." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/418982 (https://phabricator.wikimedia.org/T167246) (owner: 10Anomie) [19:27:52] (03Merged) 10jenkins-bot: Update default thresholds to new syntax [extensions/ORES] - 10https://gerrit.wikimedia.org/r/419856 (https://phabricator.wikimedia.org/T181159) (owner: 10Awight) [19:31:35] (03CR) 10jenkins-bot: Update default thresholds to new syntax [extensions/ORES] - 10https://gerrit.wikimedia.org/r/419856 (https://phabricator.wikimedia.org/T181159) (owner: 10Awight) [19:33:58] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10Patch-For-Review, 10User-Ladsgroup: Store wp10 predictions in the MediaWiki database. - https://phabricator.wikimedia.org/T175757#4057498 (10awight) [19:35:57] (03CR) 10Awight: [C: 032] "Cool. It has potential to be abused, e.g. we probably never want to configure any of the editquality models to self-clean." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/418877 (https://phabricator.wikimedia.org/T166427) (owner: 10Ladsgroup) [19:41:29] (03Merged) 10jenkins-bot: Update for the actor table change [extensions/ORES] - 10https://gerrit.wikimedia.org/r/418982 (https://phabricator.wikimedia.org/T167246) (owner: 10Anomie) [19:47:01] (03Merged) 10jenkins-bot: Build a system that allows deleting old scores when new ones have arrived [extensions/ORES] - 10https://gerrit.wikimedia.org/r/418877 (https://phabricator.wikimedia.org/T166427) (owner: 10Ladsgroup) [19:48:47] halfak: Not much to review so far, but this is the direction I'm going in: https://www.mediawiki.org/wiki/JADE/Wikimania_2018_presentation [19:49:08] Also, more abstractly: https://docs.google.com/presentation/d/1DJdp98jVg7BRKhfmy-Qfv-eyjhSIhc0p4KXbYhK1QSc/edit [19:50:17] One thought: your graphics are hard to follow at a glance. You might take the graphic from mw:JADE instead. [19:52:08] awight, ^ [19:52:23] Good idea [19:52:32] (03CR) 10jenkins-bot: Update for the actor table change [extensions/ORES] - 10https://gerrit.wikimedia.org/r/418982 (https://phabricator.wikimedia.org/T167246) (owner: 10Anomie) [19:53:18] I would get some additional mileage from using pictures for each actor and data store... [19:54:40] I don't think that a wikimedia audience cares about the data-store. I think they want to hear a story about how they can use it and also maybe something about why auditing matters. [19:54:54] *wikimania [19:55:27] (03CR) 10jenkins-bot: Build a system that allows deleting old scores when new ones have arrived [extensions/ORES] - 10https://gerrit.wikimedia.org/r/418877 (https://phabricator.wikimedia.org/T166427) (owner: 10Ladsgroup) [19:55:42] That's helpful, yeah these drawings should be simplified. The thing I'm really trying to emphasize is the multiple feedback cycles, and how what we're doing is in the same spirit. [19:56:58] halfak: You think the audience will care about these usage details, though?: https://docs.google.com/presentation/d/1DJdp98jVg7BRKhfmy-Qfv-eyjhSIhc0p4KXbYhK1QSc/edit#slide=id.g3590a4447f_0_37 [19:58:06] Oh yeah. Definitely. I think demoing the data is useful. [19:58:37] Seeing it in a wiki will get people thinking [19:59:02] awight, I'd recommend talking about feedback without the details of how the data will flow. [19:59:57] I'll see how that looks. I'm sure I can dig around in the hippie literature :-) [20:42:54] wiki-ai/revscoring#1461 (observation_set_utils - 593675c : halfak): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/354502828 [20:44:03] 10Scoring-platform-team (Current), 10editquality-modeling, 10User-Ladsgroup, 10User-Tgr, 10artificial-intelligence: Train/test damaging and goodfaith model for Hungarian Wikipedia - https://phabricator.wikimedia.org/T185903#4057647 (10Halfak) https://github.com/wiki-ai/revscoring/pull/395 I think we'll n... [21:02:38] awight, what are we going to do for April 1st? [21:02:55] Yass. I'll be in the SF office, if that helps. [21:03:07] I want to deploy a model that does something ridiculous [21:03:20] We can return sores rather than scores [21:04:31] hrm [21:04:51] * halfak tries his hand at template fu again [21:06:40] I could leave unsigned union membership cards on everyone's desks [21:13:08] awight, this templating stuff is really driving me crazy :( [21:13:23] It's like untangling spaghetti. [21:13:26] Don't get too far into the weeds, is all I can suggest. [21:13:37] IMO we already have way too much logic in the template. [21:13:51] If you just rough out what it should look like, I can clean up. [21:14:05] OK I can do that. I'll get you something quick. [21:14:27] I should document all the crazy flags we're introducing... [21:17:43] For posterity, I want to state that spaghetti is very easy to untangle [21:17:49] https://gist.github.com/halfak/f00ea4efb2b158fe3d1924b3dfb10d58 [21:17:52] Something like this. [21:17:59] awight, :P [21:19:04] The reason this intersection is necessary is due to our old patterns with wikilabels. E.g. http://labels.wmflabs.org/campaigns/huwiki/33/?tasks does not contain "needs_review" fields. [21:19:24] So the data from wikilabels must be merged with autolabel before the merges can take place. [21:19:56] That's not a "union"? [21:20:09] Na. union would have all of the rows. [21:20:22] We'd need to do some sort of check to find out which ones are labeled [21:20:28] We have no logic for that [21:20:36] kk [21:20:48] Is there a branch I can push to? [21:21:04] Oh yeah. I'm working off of tgr's work. [21:21:12] It's in his repo, but we could pull it into a branch [21:21:43] cool [21:21:47] either way [21:21:51] I'll just make a patch you can use [21:21:54] https://github.com/wiki-ai/editquality/pull/142 [21:22:06] +1 it's incredibly annoying to work with the template at the moment. [21:22:18] I think partials might help... [21:22:36] and a more systematic way of dealing with different sampling approaches [21:29:09] 10Scoring-platform-team, 10Wikilabels: Complete edit quality campaign in English Wiktionary - https://phabricator.wikimedia.org/T166910#4057711 (10jberkel) @Halfak ok, so it's separate from a user's normal patrolling activity, meaning you work on an older sample instead of labeling recent changes? I think the... [21:44:05] halfak: if there is a description of what needs to be done I can try my hand at it over the weekend [21:44:24] tgr: Careful, it's turning everyone's brain into spaghetti [21:44:34] I had very little time for this project over the last few weeks but I'm hoping to wrap up now [21:45:03] tgr: Feel like mucking around in some light code generation? [21:45:05] I'm a PHP developer, I breathe spaghetti code :) [21:45:08] lol [21:45:25] It's like having gills, but for tomato sauce [21:45:47] I would love an "outside" opinion on what's happening with our code generation. [21:47:04] There are a few things happening. * templates/Makefile.j2 is accumulating way too much logic, * code_generation/config.py has editquality-specific logic which should be extracted into a separate module, * huwiki requires a new special case, which halfak roughed out in this gist: https://gist.github.com/halfak/f00ea4efb2b158fe3d1924b3dfb10d58 [21:48:31] Here's the code generation package, https://github.com/wiki-ai/editquality/tree/master/editquality/codegen [21:48:54] Call it like, pip install -e . && editquality generate_make > Makefile [21:49:19] >:( We're not putting Wikimania proposals on the wiki anymore [21:49:23] :'( [21:49:25] INO [21:49:34] * halfak wants to edit a wikipage. [21:49:45] But the submission form only allows for 300 words, so you have to put your actual proposal on a wiki, it seems. [21:50:34] 10Scoring-platform-team, 10Wikilabels: Complete edit quality campaign in English Wiktionary - https://phabricator.wikimedia.org/T166910#4057753 (10Halfak) @jberkel, when looking at recent changes, you are not reviewing a pure random sample of activity. When we're training a model, it doesn't start with any kn... [21:57:53] halfak: Feel like mapping out the possible labels pipelines with me? [21:58:20] or maybe next week. [21:58:26] yeah. next week. but yes [21:58:31] I think there are fewer than it seems [21:58:41] ^_^ that's what I think, too [21:59:04] The template is bogged down with edge cases, though [21:59:58] We should also do everything possible to move logic into a preprocessing step [22:00:37] Just a tiny example, if wiki.models doesn't exist, we should set it to {}, so we can use a bare "for" loop without testing for existence first. [22:00:47] Lots we can do there. [22:01:08] And I'm imagining that the labels pipelines can be expressed in a more generic way, e.g. "merge_strategy: intersect" [22:13:43] o/ [22:15:13] * halfak clicks submit [22:15:14] o/ [22:15:18] have a good weekend awight! [23:39:50] halfak: What does it mean when an observation comes out of wikilabels with needs_review: true? e.g. https://github.com/wiki-ai/editquality/blob/master/Makefile#L382-L383