[00:25:17] (03PS1) 10Catrope: Follow-up a93bd12: also ignore RC_LOG/RC_EXTERNAL in non-RCFilters UI [extensions/ORES] - 10https://gerrit.wikimedia.org/r/368117 (https://phabricator.wikimedia.org/T168487) [07:52:22] 10Scoring-platform-team-Backlog, 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3477154 (10fgiunchedi) cc'ing #operations too [11:46:59] 10Scoring-platform-team-Backlog, 10Wikilabels: Complete edit quality campaign v2 in Finnish Wikipedia - https://phabricator.wikimedia.org/T166909#3477735 (10Zache) [11:47:12] 10Scoring-platform-team-Backlog, 10Wikilabels: Complete edit quality campaign v2 in Finnish Wikipedia - https://phabricator.wikimedia.org/T166909#3311606 (10Zache) [11:47:37] 10Scoring-platform-team-Backlog, 10Wikilabels: Complete edit quality campaign v2 in Finnish Wikipedia - https://phabricator.wikimedia.org/T166909#3311606 (10Zache) First update message done - https://fi.wikipedia.org/w/index.php?title=Wikipedia%3AKahvihuone_%28tekniikka%29&type=revision&diff=16646108&oldid=166... [13:35:54] 10Scoring-platform-team, 10ORES, 10Epic: [Epic] Structured deployment of ORES - https://phabricator.wikimedia.org/T130369#3477986 (10Ladsgroup) I don't think we should call it done, I think we need to spec out the details and then implement them. What I suggested was to use wmf-like deployment. Maybe it's no... [14:00:00] o/ [14:13:00] 10Scoring-platform-team, 10ORES: Discuss OS version for stat* and new ores* machines - https://phabricator.wikimedia.org/T171851#3478146 (10Halfak) [14:42:44] 10Scoring-platform-team-Backlog, 10Wikimania-Hackathon-2017: ORES @ the Wikimania Hackathon - https://phabricator.wikimedia.org/T170015#3478271 (10Halfak) Added ORES as a feature project to https://www.mediawiki.org/wiki/New_Developers#ORES [14:49:26] 10Scoring-platform-team-Backlog: Add dressed up JSON response for browser-based Wiki Labels API requests - https://phabricator.wikimedia.org/T171770#3478274 (10Halfak) [14:50:05] 10Scoring-platform-team-Backlog: Add dressed up JSON response for browser-based Wiki Labels API requests - https://phabricator.wikimedia.org/T171770#3475804 (10Halfak) Relevant: https://etherpad.wikimedia.org/p/wikilabels_routes [14:50:46] 10Scoring-platform-team-Backlog, 10Wikilabels: Add dressed up JSON response for browser-based Wiki Labels API requests - https://phabricator.wikimedia.org/T171770#3475804 (10Halfak) p:05Triage>03Low [14:51:49] 10Scoring-platform-team-Backlog, 10Wikilabels: Allow Wiki Labels API to list inactive campaigns - https://phabricator.wikimedia.org/T171768#3478282 (10Halfak) p:05Triage>03Normal [14:52:29] 10Scoring-platform-team-Backlog, 10ORES, 10Scap, 10Release-Engineering-Team (Watching / External): Simplify git-fat support for pulling from both production and labs - https://phabricator.wikimedia.org/T171758#3475312 (10Halfak) 05Open>03stalled p:05Normal>03High [14:52:31] 10Scoring-platform-team-Backlog, 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3478291 (10Halfak) [14:52:57] 10Scoring-platform-team-Backlog, 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3470967 (10Halfak) [14:53:00] 10Scoring-platform-team-Backlog, 10ORES, 10Scap, 10Release-Engineering-Team (Watching / External): Simplify git-fat support for pulling from both production and labs - https://phabricator.wikimedia.org/T171758#3475312 (10Halfak) 05stalled>03Resolved p:05High>03Normal [14:53:10] 10Scoring-platform-team, 10ORES, 10Scap, 10Release-Engineering-Team (Watching / External): Simplify git-fat support for pulling from both production and labs - https://phabricator.wikimedia.org/T171758#3475312 (10Halfak) [14:53:18] 10Scoring-platform-team-Backlog, 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3470967 (10Halfak) [14:53:20] 10Scoring-platform-team, 10ORES, 10Scap, 10Release-Engineering-Team (Watching / External): Simplify git-fat support for pulling from both production and labs - https://phabricator.wikimedia.org/T171758#3475312 (10Halfak) 05Resolved>03Open [14:54:05] 10Scoring-platform-team-Backlog, 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): ORES should use git-fat for binaries - https://phabricator.wikimedia.org/T171619#3470967 (10Halfak) [14:54:28] 10Scoring-platform-team-Backlog, 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): ORES should use git-fat for binaries - https://phabricator.wikimedia.org/T171619#3470967 (10Halfak) 05Open>03stalled p:05Normal>03High [14:57:51] 10Scoring-platform-team-Backlog, 10ORES: Design a re-review pattern for meta ORES - https://phabricator.wikimedia.org/T171496#3478314 (10Halfak) @awight, it seems like our training sets really need a re-review pattern. @Natalia's analysis shows that some of the labels are pretty dubious. I think this has imp... [14:58:10] 10Scoring-platform-team-Backlog, 10ORES: Design a re-review pattern for meta ORES - https://phabricator.wikimedia.org/T171496#3466935 (10Halfak) p:05Low>03Normal [15:09:23] GROOMED [15:29:17] 10Scoring-platform-team, 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Unlabeled goodfaith observations are assumed "false" -- should be "true" - https://phabricator.wikimedia.org/T171491#3478450 (10Halfak) https://github.com/wiki-ai/editquality/pull/87 is merged. Now all that is left i... [15:30:15] 10Scoring-platform-team, 10ORES: Late-July 2017 ORES deploy - https://phabricator.wikimedia.org/T171505#3478452 (10Halfak) 05Open>03Resolved [15:30:17] 10Scoring-platform-team, 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Unlabeled goodfaith observations are assumed "false" -- should be "true" - https://phabricator.wikimedia.org/T171491#3478453 (10Halfak) 05Open>03Resolved [15:30:19] 10Scoring-platform-team, 10Wikilabels, 10User-Ladsgroup: linting tests for wikilabels - https://phabricator.wikimedia.org/T171084#3478454 (10Halfak) 05Open>03Resolved [15:30:21] 10Scoring-platform-team, 10Wikilabels: The great test making of Wikilabels - https://phabricator.wikimedia.org/T171080#3478455 (10Halfak) [15:30:23] 10Scoring-platform-team, 10MediaWiki-extensions-FlaggedRevs, 10ORES: Decrease FlaggedRevs backlog by using ORES predictions models - https://phabricator.wikimedia.org/T165848#3478458 (10Halfak) [15:30:26] 10Scoring-platform-team, 10editquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Flagged revs approve model to fiwiki - https://phabricator.wikimedia.org/T166235#3478457 (10Halfak) 05Open>03Resolved [15:30:28] 10Scoring-platform-team, 10MediaWiki-extensions-ORES, 10MW-1.30-release-notes (WMF-deploy-2017-06-27_(1.30.0-wmf.7)), 10Patch-For-Review, and 2 others: [Discuss] Make ORES Review Tool preferences more prominent - https://phabricator.wikimedia.org/T167910#3478456 (10Halfak) 05Open>03Resolved [15:31:17] BLAM [15:46:38] 10Scoring-platform-team, 10ORES: Reimage ores* hosts with Debian Stretch - https://phabricator.wikimedia.org/T171851#3478511 (10Halfak) [15:48:02] 10Scoring-platform-team, 10ORES: Reimage ores* hosts with Debian Stretch - https://phabricator.wikimedia.org/T171851#3478146 (10Halfak) I just talked to @fgiunchedi about this in IRC and he said that reimaging while they aren't getting traffic makes sense. @akosiaris, how difficult is this? [15:48:12] 10Scoring-platform-team, 10ORES, 10Operations: Reimage ores* hosts with Debian Stretch - https://phabricator.wikimedia.org/T171851#3478519 (10Halfak) [15:48:37] 10Scoring-platform-team, 10ORES, 10Operations: Reimage ores* hosts with Debian Stretch - https://phabricator.wikimedia.org/T171851#3478146 (10Halfak) a:05Halfak>03None [15:48:52] 10Scoring-platform-team-Backlog, 10ORES, 10Operations: Reimage ores* hosts with Debian Stretch - https://phabricator.wikimedia.org/T171851#3478146 (10Halfak) [15:49:30] 10Scoring-platform-team-Backlog, 10ORES, 10Operations: Reimage ores* hosts with Debian Stretch - https://phabricator.wikimedia.org/T171851#3478532 (10fgiunchedi) +1 to starting brand new clusters with stretch! [15:51:18] 10Scoring-platform-team-Backlog, 10ORES: Meta ORES: UI/API for reviewing/refuting how ORES classifies you and your stuff - https://phabricator.wikimedia.org/T148700#3478540 (10Halfak) [15:51:20] 10Scoring-platform-team, 10ORES: Design Meta ORES data storage schema - https://phabricator.wikimedia.org/T153152#3478539 (10Halfak) [15:51:22] 10Scoring-platform-team-Backlog, 10ORES: Design a re-review pattern for meta ORES - https://phabricator.wikimedia.org/T171496#3478538 (10Halfak) [15:51:58] 10Scoring-platform-team-Backlog, 10ORES, 10Documentation, 10Spike: [Spike] potential technical implementations of ORES meta - https://phabricator.wikimedia.org/T166053#3478544 (10Halfak) [15:52:01] 10Scoring-platform-team-Backlog, 10ORES: Meta ORES: API data storage and querying - https://phabricator.wikimedia.org/T153145#3478545 (10Halfak) [16:34:51] halfak: do you somehow use Reverted model in Goodfaith/Badfaith prediction? [16:35:02] fajne, nope [16:36:51] halfak: ok, then. let me know if you think this worth exploring: https://meta.wikimedia.org/wiki/Research_talk:Automated_classification_of_edit_quality/Work_log/2017-07-24#Can_the_Reverted_model_help_improve_the_Damaging_model.3F [16:38:01] oops, i meant not in Goodfaith, but in Damaging model [16:38:04] fajne, the reverted model uses the exact same features as the damaging model. [16:38:38] BUT I one thing that we might do is take all of the weird examples from the enwiki labeling and put them into wikilabels to get a second label. [16:38:48] What do you think of that? [16:39:36] E.g. revisions that were reverted but not marked "damaging". Revisions that were maked "bad faith" but not "damaging". And revisions that were not marked damaging, but in cross-validation, the model thought they looked damaging? [16:39:43] yep, this also works. if you know how to detect all weird examples) [16:39:56] I suppose for the last one we could do the same for revisions that were marked "damaging" but CV model though they didn't look like it. [16:40:16] fajne, I think we could spec out a process for subsampling the weird examples from a random sample. [16:40:55] E.g. Pass #1 finds the obvious cases (e.g. "reverted" but not "damaging") and pass #2 does a cross-validation to find the less obvious examples. [16:40:59] What do you think of that? [16:43:28] fetching reverted+not damaging, as well as not damaging+badfaith should be straightforward. not sure how CV work for less obvious examples... [16:46:54] fajne, I agree that the CV part is less obvious and more complicated. Essentially, I imagine that we can get output from the model during a call to `cv_train` that would output the scored revisions. [16:48:53] * awight rubs eyes and wonders what year it is... [16:51:07] is your idea that the pass #2 will just use the updated training set in CV and therefore let the model learn something new about Damaging and Badfaith? [16:53:01] fajne, hmm... not sure what you are asking. [16:53:09] o/ awight [16:53:37] \o [16:53:52] (03CR) 10Jforrester: [C: 032] "We now have this SQL in three places (and an if condition in a fourth); something to clean up later." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/368117 (https://phabricator.wikimedia.org/T168487) (owner: 10Catrope) [16:53:56] 10Scoring-platform-team-Backlog, 10ORES, 10Documentation, 10Spike: [Spike] potential technical implementations of ORES meta - https://phabricator.wikimedia.org/T166053#3478808 (10Halfak) Talked to @Ottomata about using EventBus to free us from settling on a "final" database store. If we can tie into Event... [16:54:17] halfak: the union merge PR is ready to review, btw [16:54:19] halfak, nevermind. [16:54:57] https://github.com/wiki-ai/revscoring/pull/338 [16:57:08] (03Merged) 10jenkins-bot: Follow-up a93bd12: also ignore RC_LOG/RC_EXTERNAL in non-RCFilters UI [extensions/ORES] - 10https://gerrit.wikimedia.org/r/368117 (https://phabricator.wikimedia.org/T168487) (owner: 10Catrope) [16:57:48] awight, we might need a second opinion on this. It seems you've opted to continue with a class and I just don't see that as a good option. It looks very YAGNI to me. [16:58:51] I imagine a function called union_merge(*observations, *, id="rev_id") [16:59:42] Amir1, maybe you could do the review [16:59:53] I know you won't have time until the weekend though. :/ [16:59:59] I'm around [17:02:28] halfak: I don’t really care about the class [17:02:37] happy to refactor or whatev [17:02:57] IMO, classes are always better in the long run, more testable, better encapsulation [17:03:11] but this is pretty much a one-liner [17:06:44] awight, https://gist.github.com/halfak/399ae575d99888865f9b72e90c830c6d [17:07:47] ahem…. yes I see [17:08:02] awight: as the third party opinion I can think of two things: in the whole product we built the utilities the functional way, IMO consistency is way more important than the method itself [17:08:12] +1 [17:08:50] 2- python is not very OOP-friendly (specially comparing to php), lots of time it just makes things more complicated then simple [17:09:09] I don't know if that's the case or not [17:09:26] * halfak wonders if Amir1 thinks php is more OOP or less [17:09:27] :D [17:09:41] php is way more OOP-friendly [17:10:00] interesting. I'm curious how you see that. [17:10:06] interface, abstracts, visibility, etc. [17:10:18] python has no concept of interface [17:10:19] OH! yeah, it uses explicit terms for that. [17:10:23] Interesting point, I think python’s modules are actually a mushy OO, just lacking state [17:10:47] Right, from python's point of an "interface" is just a fancy word with no meaning. Just make your interface and use it like an interface. [17:11:02] Less constraining, easier to shoot own foot. [17:11:14] but I have no formal education and everything I learned was through working in WMF products, so my assessments might be very very shallow [17:11:22] yeah you’re not supposed to check is_a in python. Duck typing all th eway [17:11:31] Amir1, :) I think I see what you mean though. [17:11:53] I came from Java land before python so I feel like I grok the highly explicit OOP practice [17:12:20] land of the boiler plates :) [17:12:21] also typehinting [17:12:31] this is bothering me a lot in python [17:12:42] that python has type hinting? [17:12:45] not having compile-time checking of anything sucks [17:13:00] it doesn't, comparing to php [17:13:17] https://www.python.org/dev/peps/pep-0484/#non-goals [17:13:32] It has for two years, I think. We just don't use them. [17:13:39] My pet theory is that python is actually optimized for incredibly bad and lazy programmers. Which is why it’s so great. Python never forces you to adhere to any one way of doing things. [17:13:42] in php when you define the method you can explicitly say, this argument must implement this interface (or instantiate this class) otherwise it fails [17:13:52] https://docs.python.org/3/library/typing.html [17:14:04] (03CR) 10Catrope: [C: 031] "I'll test this then merge" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/359364 (https://phabricator.wikimedia.org/T167908) (owner: 10Catrope) [17:14:12] awight: yes, exactly. It's one of its greatness [17:14:22] which might come at a cost [17:15:04] * awight tries to portmaneau “strength” and “weakness”. hard to mash up [17:15:26] pianoforte [17:15:36] * halfak heads to lunch [17:15:38] back soon [17:15:44] * halfak makes a quick sammich [17:17:08] awight: when do you think you guys will discuss Meta ORES today? [17:18:16] Good question, we haven’t scheduled it yet. Is there a time that works best for you? [17:18:36] any time works [17:21:10] halfak: looks like you have the busiest day, can you pick a time? [17:32:53] I’m rewriting this patch to be consistent etc. but want to highlight the changes I’ll have to make to the test. It’s much nastier to test now, since the business logic has moved up to the main() routine, and the external interface is some random string-based commandline nonsense. [17:32:56] Doesn’t matter in the case, but it does demonstrate what I’m on about. [17:33:01] *this [17:33:35] I also don’t like that we’ll be merging within a file if an id repeats, but meh [17:41:34] awight, isn't that the intention of the union part of union_and_merge [17:42:05] Also, it seems to me the only logic that remains in the main() function is parsing commandline arguments and turning them into things that union_and_merge can operate on. [17:42:25] It’s a tiny detail…. IMO the correct behavior would be to validate that an ID never recurs within a file, and complain if it does [17:42:42] the DataUnion code would drop any duplicates within a file rather than merging, which is hardly good behavior, so I’m “meh" [17:43:33] I’ll push in a second—main() took on too much. I could fix it but YAGNI this entire tool [17:45:15] Re. where which logic belongs, check out https://github.com/wiki-ai/revscoring/blob/master/revscoring/utilities/extract.py for a good example of how we usually standardize. [17:46:00] main() handles arguements from the command line, run() (if necessary) handles file operations, and () performs the business logic [17:46:31] oh wait. I suppose that run() would have the business logic [17:46:33] I get the intention, but it doesn’t break down that cleanly here [17:46:46] for example, we do read_observations in main() [17:47:07] right! We're handling a param! Opening the file and turning into something python can use. [17:47:15] It's a great way to check if the file exists and is readable. [17:47:48] et voilá [17:48:41] I’m sure we’ll run into more fruitful philosophical debates than this :D [17:49:46] halfak: I diluted the more important question, though. Do you feel like chatting about meta-ores with fajne and me today? [17:50:16] 10Scoring-platform-team-Backlog, 10ORES, 10Documentation, 10Spike: [Spike] potential technical implementations of ORES meta - https://phabricator.wikimedia.org/T166053#3479051 (10Halfak) Oh! One more thought... We'll need to have the judgement storage operate at high performance so that judgements can be... [17:50:26] ^ been taking some notes here. [17:50:35] I could talk at 3PM [17:50:46] My "Documentation Time" meeting is perfect for this. [17:50:59] It's more of a work meeting :) [17:51:05] Sorry 1PM PDT [17:51:13] 3PM halfak-time [17:51:29] Should probably pay some attention to the metrics meeting now :) [17:51:33] cool [17:52:05] I'm going to go do my usual yardwork + listen to metrics routine. [17:52:22] So I'll be AFK for the next hour. I might rush back if something exciting happens at metrics. [17:56:25] * halfak added notes to awight's update [17:56:47] ty. Also defaultdict was a nice addition. [18:04:21] 10Scoring-platform-team-Backlog, 10ORES, 10Documentation, 10Spike: [Spike] potential technical implementations of ORES meta - https://phabricator.wikimedia.org/T166053#3479102 (10Ottomata) Sometimes, it’s too bad we don’t do the avro thing! Then you could use this: http://docs.datamountaineer.com/en/lates... [18:10:45] 10Scoring-platform-team-Backlog, 10ORES, 10Documentation, 10Spike: [Spike] potential technical implementations of Meta-ORES - https://phabricator.wikimedia.org/T166053#3479117 (10awight) [18:11:12] wiki-ai/revscoring#1121 (data_utils - 06791f2 : Adam Roses Wight): The build was fixed. https://travis-ci.org/wiki-ai/revscoring/builds/258250279 [19:36:38] awight you wont have to work with gwtui soon heh :). [19:36:51] hehe don’t worry, I don’t :) [19:37:12] lol [19:37:16] But I’m happy for the world to hear that it’ll be replaced [19:37:38] :) [19:39:45] awight though the change url look like [19:39:46] https://gerrit-review.googlesource.com/#/c/gerrit/+/116091/ [19:39:51] now notice any difference? [19:40:16] I read in one of the links you sent that the change URL would include the project. That is helpful. [19:40:24] yep [19:40:45] though confusing. Luckly there's backwords compat. and in polygerrit they use the old format for now :) [19:44:14] That’s also good news, cos I’m sure it’ll take some time to shake out links to the old URLs [19:44:59] yep [19:47:42] paladox: Thanks again for helping improve gerrit, that has a big impact on our day-to-day lives! [19:47:52] Your welcome :) [19:47:54] Have you been watching any of the WMF’s discussion about Differential? [19:47:58] yes [19:48:16] The migration is currently on hold [19:48:28] I’ve been out of that loop myself, but having tried Differential for one small project last year, I hope the migration stays on hold forever. [19:49:00] New Gerrit becoming more hackable will be key to that discussion, one can dream... [19:49:14] yes [19:49:36] differential is deftly not as good as gerrit [19:53:34] Everything about it… The opaque URLs might take the cake though. [20:04:29] lol [20:06:26] awight /me attempts to do https://gerrit-review.googlesource.com/#/c/116211/ heh [20:06:34] i wonder why it's not done already [20:06:39] as upstream require one lol [21:48:25] (03CR) 10Krinkle: [C: 04-1] "The core patch was landed and I2edae3ff1f3d24 seems pretty straight forward. Shall we continue there, then?" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/341946 (https://phabricator.wikimedia.org/T159753) (owner: 10Catrope) [22:32:37] (rebase) [22:40:09] awight, we might need to have an RFC on test patterns with CLI args. It seems you're pushing for a new standard of integration testing for utilities. I think this is interesting but I have concerns. [22:41:07] Sorry to keep pushing back, but know that it's primarily coming from a "this is very new to my work" and not "I think you're being a big dummy" [22:41:55] Me too—IMO it’s very much fair game to have complicated or infrastructurey tests, but that’s why I was whinging about taking out the DataUnion function. [22:41:55] err class [22:42:06] the class was written with testability in mind [22:42:19] what's wrong with testing union_merge? [22:42:33] hold up—responding in GH [22:42:33] I will, it’s a good suggestion! [22:42:34] let’s test both. [22:42:37] :) I like that better [22:45:51] for the record, I see that your and Amir1’s functional approach is way *more* testable for the core routine, cos it’s stateless. [22:46:15] Ideally I’d like to combine all the strengths, and have a nicely encapsulated algorithm state somehow composed with this minimal core routine. [22:48:56] awight, I generally side on not hiding something unless it gets big and annoying to keep around. In this case, one dictionary that is only used in one spot is a really nice state object to manage. [22:49:29] But as you pointed out in the past, the request and response stuff in ORES was unwieldy! [22:49:36] I was so happy to encapsulate that :) [22:51:57] On the next level, I’m glad we have such diversity of perspective so far <3 [23:15:16] halfak: and thanks for all the code review! [23:15:34] :) Is good to not be blocked for forever [23:15:42] * halfak works through awights old reviews :) [23:15:50] I’m starting on https://meta.wikimedia.org/wiki/User:Adamw/Draft/fiwiki.flaggedrevs_work_log now, will spend a few hours on it [23:16:05] hehe good to have to CR comments saved up for winter, yeah [23:25:59] wiki-ai/revscoring#1128 (revscoring_2_followup - dbdf2c1 : halfak): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/258355945