[01:27:15] eyeballs are bleeding from Google Sheets [01:45:58] o/ [09:21:53] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/JADE] - 10https://gerrit.wikimedia.org/r/460246 (owner: 10L10n-bot) [13:48:22] * halfak sits on hold with Delta. [13:48:28] Oh mornings. Why are you difficult? [15:12:55] 10Scoring-platform-team (Current), 10Documentation: Document and justify JADE schema proposals - https://phabricator.wikimedia.org/T204250 (10Halfak) [15:45:21] o/ awight [15:45:28] working on a consolidated list of use-cases. [15:45:39] See T204250. [15:45:40] T204250: Document and justify JADE schema proposals - https://phabricator.wikimedia.org/T204250 [15:45:50] I'll post the linked wiki page soon. [15:48:24] halfak: I've been working on change proposals, so we have something explicit to discuss [15:48:30] https://docs.google.com/spreadsheets/d/1IxXtym0ELTANl40zzK8zix_HQ-uVPgP-uNOb_8gVJNY/edit#gid=0 [15:48:33] "change proposals"? [15:49:28] Yeah, along the lines of your suggestion yesterday that I write down what I'm proposing. [15:50:23] I see. I was thinking that we could rather tackle this as a wholistic proposal. [15:50:25] The schema we looked at yesterday presented a snapshot of what I'm thinking these days, but I rewrote as change requests from mas o menos our last consensus savepoint to now. [15:50:39] holistic didn't seem to work, so how about we go one point at a time? [15:50:44] I also have it in holistic form [15:50:47] What was the last consensus save point? [15:51:17] I really can't say for certain, so I was conservative and tried to include anything we might have not discussed fully. [15:52:51] 10Scoring-platform-team (Current), 10Documentation: Document and justify JADE schema proposals - https://phabricator.wikimedia.org/T204250 (10awight) [15:53:14] 10Scoring-platform-team (Current), 10Documentation: Document and justify JADE schema proposals - https://phabricator.wikimedia.org/T204250 (10awight) p:05Triage>03High a:03awight [16:00:22] https://www.mediawiki.org/wiki/JADE/Use_cases [16:02:06] nice work! [16:05:04] I'm missing admin actions. [16:05:07] Will get to those later. [16:05:13] I couldn't crib them from other materials :| [16:07:32] good point, we had taken those for granted but they do need to be spelled out. [16:08:05] * halfak works to add more [16:11:21] P2 "Move endorsement.comment" Where? [16:11:24] awight, ^ [16:12:39] > Move endorsement.comment, it will be replaced by an edit made to the main judgment slot or its talk page, at the editor's discretion. [16:12:50] Is text wrapping working? drive sheets are always wonky. [16:13:19] (03PS1) 10Zfilipin: Selenium: add selenium-daily NPM script [extensions/ORES] - 10https://gerrit.wikimedia.org/r/460388 (https://phabricator.wikimedia.org/T188742) [16:16:33] Right. Move to ... [16:16:46] I think you might mean to say Move to talk page or edit comment. [16:16:53] It's hard to respond when it is unclear. [16:16:56] to the main judgment slot or its talk page [16:17:09] not the edit comment [16:19:27] "Main judgment slot" is a separate proposal, isn't it? [16:20:58] P2 introduces the wikitext slot, I can expand on that for sure. [16:24:25] Yes. Please make that clear. [16:24:46] "The main judgment slot" needs its own justification. [16:25:50] (03CR) 10Zfilipin: "This change is ready for review." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/460388 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [16:26:33] Will do. P2 is the justification, so I'll roll those two together I suppose, since I don't see a way to split as you suggest. [16:26:43] But I'll make the MCR aspect more explicit. [16:37:42] kk, MCR noted. [16:41:33] halfak: Trying to understand the drawbacks for P2. [16:41:59] For (2), Wikipedians can still leave comments. How is that affected? [16:42:22] For (3), edit comments are not part of the proposal. [16:48:35] halfak: Thanks for taking on this review! [16:48:47] (I moved the last 2 questions into the doc, FYI) [16:52:05] harej: ^ party in the spreadsheet [17:10:42] (03CR) 10Hashar: [C: 032] Selenium: add selenium-daily NPM script [extensions/ORES] - 10https://gerrit.wikimedia.org/r/460388 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [17:18:52] Really great, I think we're making progress... hopefully other people feel the same :-) [17:24:43] halfak: I think I left out a detail in P1, that endorsements are still nested under judgments. I thought that was useful, too. [17:32:14] (03Merged) 10jenkins-bot: Selenium: add selenium-daily NPM script [extensions/ORES] - 10https://gerrit.wikimedia.org/r/460388 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [17:33:56] Ohhhhh meetings. [17:34:07] my condolences! [17:34:08] So, I made some adjustment to clarify re. P2 [17:34:25] (03CR) 10jenkins-bot: Selenium: add selenium-daily NPM script [extensions/ORES] - 10https://gerrit.wikimedia.org/r/460388 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [17:34:36] Darn, it's hard to see what changed [17:35:48] Specifically, 1: endorsement comments are *next to* their endorsement data. [17:35:56] Woops. That was for 2; [17:35:58] I think there might be some confusion about what's happening with endorsements btw. They're still a subkey under a judgment with given value fyi. [17:36:15] How would that work? [17:36:25] If it's just randomly placed somewhere on a content page? [17:36:39] https://phabricator.wikimedia.org/T202596#4575825 [17:36:44] Also, how would an endorsement comment be keyed under judgment? [17:37:16] Ah I see. yes endorsement comment moved, but the endorsement metadata is available in the same data path as before. [17:37:46] endorsement comments are associated with judgment value just as revisions, not as the static final text. [17:37:53] OK sure. So the relationship between endorsement and proposition in duplicated between proposition_rev and nesting? [17:38:19] no, proposition_rev cannot be calculated unless it's explicitly in the data. At least, it's really hard and will still be a guess. [17:38:41] But you are proposing it. [17:38:47] proposition_rev is a revid for the entire judgment, so it's a snapshot of the wikitext and the JSON, does not index into the data at all. [17:39:06] awight, it does via nesting [17:39:15] yes [17:39:18] what's the duplication? [17:39:24] So the same information is duplicated. [17:39:31] Why have propostion_rev at all? [17:40:01] I understand what you mean about nesting, but I don't see how proposition_rev duplicates anything. Think of it as previous_rev when editing the judgment. [17:40:07] The goal of proposition_rev is to catch the revision of the thing being judged at the point of judgment, right? [17:40:08] o/ mzanotto [17:40:18] Welcome! [17:40:39] Thanks! Happy to join [17:41:00] mzanotto, so you'd asking me in PM how is best to get an overview of our work? [17:41:20] harej: Moving noisy discussion to #wikimedia-ai-ephemeral, pls jump in when you can! [17:41:57] https://www.mediawiki.org/wiki/Wikimedia_Scoring_Platform_team [17:43:09] mzanotto, This is a good place to start ^ [17:43:27] Our primary product is https://www.mediawiki.org/wiki/ORES [17:43:36] It's a machine prediction service for Wikipedians. [17:43:56] We build AIs to detect vandalism, assess article quality, route new page creations by topic, etc. [17:44:15] I had an overview of what's in there. I'll have a deep dive in the different project pages and repos to get a better picture [17:48:51] I'll try to figure out where I could be of help. My background is in machine learning, have you already got an idea of where you could use a couple more hands? [17:51:22] There are some models that we are regularly building for various wikis. E.g. we have article quality models for English, Turkish, French, Russian, and a few others. [17:51:31] But we don't have one for Galician, so I'm working on that now. [17:51:41] https://phabricator.wikimedia.org/T201146 [17:52:11] A lot of the work is building an adequate training set. We do that in collaboration with the relevant wiki communities. [17:56:44] Interesting. Great, I'll have a look around the pages and come back with clearer ideas then [17:58:26] Cool. I can direct you towards a good place to get started if you just want to make your first contrib ASAP. :) [17:58:40] Alternatively, we have some big ticket modeling problems that you could look at. [17:59:09] e.g. notconfusing is looking at modeling newcomer quality (e.g. is this newcomer trying to contribute productively or are they some type of vandal?) [17:59:19] I'm sure there's some work to bite off there. [18:01:50] sure, feel free to direct me to a couple of open problems and I'll look into those to see where I'd be able to help more [18:03:53] mzanotto: o/ Hi, I'm a teammate of halfak. We add a so-called "easy" tag to tasks in our work tracker and you can filter to see what's available at the moment: https://phabricator.wikimedia.org/project/board/1901/query/rCNFr9ipmEnt/ [18:04:31] This doesn't actually mean the work will be easy, just that it won't take a huge amount of knowledge transfer and coordination with us, so are reasonable tasks to pick up as a volunteer. [18:05:14] mzanotto: Maybe this is interesting to you? https://phabricator.wikimedia.org/T155843 [18:26:20] That looks interesting thanks. I'll have a look at the other "easy" tasks and decide. Should I notify on this channel what I'd pick or elsewhere? [18:29:25] mzanotto: This channel is a good place during U.S. business hours, or the most durable place is to comment on the tasks directly. [18:34:36] awright: Perfect [18:40:12] mzanotto: FYI, here's some documentation on how to create an account in Phabricator: https://www.mediawiki.org/wiki/Phabricator/Help [20:07:23] o/ [20:08:38] awight, I'm digging into the spreadsheet you started. I've got to say, this is really messy. I have concerns about how proposals are being made. I also have concerns about how these are not tied to use cases. [20:09:22] headed out for a bit, thanks for looking at my mess. [20:12:11] * awight throws a few legos into the doorway for good measure [20:13:38] FWIW, I actually thought this was a pretty great format and that we were making more progress than we have with any of the other styles we've tried so far. [20:15:57] I'm not as concerned about use cases because IMO this and all of the previous design proposals have satisfied almost every use case, and what I'm doing now is trying to optimize for both human and machine interactions. [20:17:19] For example, judgment.notes and the wikitext proposal both satisfy "edit judgment reasoning", but in the first schema you're editing a single field for the reason, and in the second schema you're editing your reason as part of a larger narrative. [20:19:35] back for our meeting [20:36:19] Honestly, I am finding it intensely frustrating. [20:50:53] 10Scoring-platform-team (Current), 10Documentation: Document and justify JADE schema proposals - https://phabricator.wikimedia.org/T204250 (10Halfak) a:05awight>03Halfak [20:52:27] 10Scoring-platform-team (Current), 10Documentation: Document and justify JADE schema proposals - https://phabricator.wikimedia.org/T204250 (10Halfak) [20:52:36] 10Scoring-platform-team (Current), 10Documentation: Document and justify JADE schema proposals - https://phabricator.wikimedia.org/T204250 (10Halfak) [21:16:02] halfak: How can I help you? [21:16:27] Happy to clean this up or whatever is going to make it better. [21:16:55] It's already quite a bit more paperwork than the understanding I was working with until yesterday, that I had the authority to implement as I saw fit. [21:17:06] I'd like you to consider data consumers more seriously or just take my word for it. [21:17:28] I'm taking it very seriously, and we're hammering out the details... [21:19:32] Halfak generously compiled some of my notes into a concisely presented use cases page and I am currently working on describing the features of JADE in terms of use cases. I think that will help guide the discussion, because along the way I am finding gaps. [21:20:08] Nice. Thanks harej. Also half of this was from my own notes. You'll notice there are new use cases :P [21:20:10] Any implementation detail that doesn't affect a use case I'm not sure how strongly I should care about. [21:20:18] I hope my notes were able to contribute something! [21:20:26] rad! [21:21:21] Just finishing up a list of the schema proposals. [21:22:11] VisualEditor doesn't like this ;Weird syntax: thing. [21:23:36] https://www.mediawiki.org/wiki/JADE/Content_schemas [21:23:46] That's a definition list [21:23:54] It's pretty standard syntax :| [21:23:59] ;Term: Definition [21:24:36] VE doesn't recognize it as a thing. Fascinating. I didn't think it was *that* obscure. [21:25:15] Yeah. It's ...er... very *not* obscure. [22:03:53] i think that was a helpful exercise, thank you halfak and awight [22:04:17] Indeed. It feels nice to churn through some points of disagreement. [22:05:28] Here's another minor point about why it feels wrong to completely dismiss MCR: we're going through this process because of an RFC with TechCom, and that group (daniel) actually recommended us to use MCR. [22:06:57] I think they recommend MCR in good faith, but I don't think they make that recommendation understanding the full context we're working in. [22:07:14] Agreed. [22:07:35] I don't think that adopting a technology to appease a group of technologists who are fans of it is a good idea. [22:07:48] What about our context is surprising, though? It seems that we're just implementing a type of wiki page. [22:07:59] It doesn't serve a good use case for us. [22:08:11] The proposal to use MCR reduces the usefulness of the system. [22:08:17] It's not to appease anyone, it's for the reasons I suggested. [22:08:20] It also massively complicates our project and makes us liable to be on the receiving end should shit hit the fan. [22:08:43] If we invest MCR and it isn't fully integrated in a month or two, we're stuck. [22:09:01] +1 this is all stuff I was saying a few months ago, but now that time has passed and we're closer to the milestones. [22:10:10] How much delay should we accept in return for how much benefit? [22:10:51] To understand the delay, we should put together the blocking work... [22:10:55] And what benefit, exactly? [22:11:03] I was thinking of doing that myself, for fun. [22:11:20] We're pretty severely behind on JADE as it is. [22:11:35] * awight refrains from pointing fingers [22:12:23] No fingers to point. Stuff happened. Let's do good efficient (and well-scoped) product design coupled with good, efficient engineering. [22:12:30] The benefit as I see it is that we create a single, coherent text document per entity. I understand that nobody else is seeing it that way at the moment. [22:13:43] That's not a benefit. That's a fact of the implementation [22:13:52] "Coherent" is a speculative adjective. [22:14:02] It might be a big mess. [22:14:05] :D [22:14:22] Luckily, it's intended for humans to read [22:14:28] and to structure as they see fit. [22:14:36] Right. I don't see why we want that. [22:14:51] We're purposefully implementing structure so we can all reap the benefits of being structured. [22:14:58] And we have a talk page to negotiate openly. [22:15:10] And we have design iterations to address missing aspects of our structure. [22:15:24] I'm advocating for that because the notes field is for humans, not machines. [22:15:32] No it's for machines too! [22:16:06] Edit comment parsing is a big deal. Hash tags and wikilinks are semi-structured signals that work well in tightly defined text fields. [22:16:40] I see the primary purposes as * ask the judge to think hard enough to write freeform text, * provide something for other humans to read, and only in a distant third place, * do poorly-defined machine analysis of text [22:17:40] We'll be able to associate users interacting with a judgment with their endorsement of a specific judgment.data value, so we have a way to associate the data value with their wikitext diff, FYI [22:19:08] Why are data consumers a distant 3rd place for you? Why don't you value what itwiki did with their grounded theory process? [22:19:25] wikitext diff is not a comment [22:19:35] It's also extremely complicated to compute a diff [22:19:45] And mediawiki does it very poorly [22:20:00] https://it.wikipedia.org/wiki/Progetto:Patrolling/ORES ? [22:20:06] Yes. [22:20:16] That's written for humans, unless I'm misunderstanding [22:20:22] They used comments to code and then build themes around false-positives. [22:20:27] They followed an algorithm. [22:20:41] I big value of JADE is that they could implement that algorithm with a machine! [22:20:45] oh cool! I hadn't seen the edit comments [22:20:50] Rather than having to implement it with themselves. [22:20:54] They are not using edit comments. [22:21:04] They are putting comments on their judgments. [22:21:24] ah yes. That's pretty much what I expect the entire "main slot" to consist of, in my proposal. [22:21:46] Or judgment.notes would have the same content, for these common/trivial cases. [22:22:29] awight, well that would work if there were only one type of judgment per entity. [22:22:43] And all judgments agree. [22:22:59] I feel like we keep going in a circle. [22:24:02] In the P8 scenario, we would extract that data by taking the users who judged and endorsed "not damaging", and matching their contribution on the main slot and talk pages... [22:24:19] By looking at diffs? [22:24:23] yes [22:24:37] Yeah. that is a bad way to support the use-case. [22:24:46] Why? The code exists [22:25:09] Because of all of the things I said about diffs earlier. [22:25:31] Diffs are messy, intractable, and not all edits to a page == a new comment. [22:25:40] Comments can be edited. So how do you delimit comments? [22:25:53] It's just very complex and adds no value to the use case. [23:33:50] harej: See you tomorrow! [23:33:59] o/