[00:30:08] awight: should I be at the meeting on Friday? [01:12:17] harej: Probably not necessary, it's lower-level storage design stuff [01:12:25] & also, now it's Tuesday at 6:30am :-/ [01:12:35] if that helps make the decision ;-) [08:58:28] (03CR) 10Ladsgroup: [C: 032] "Sounds good to me" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/445402 (https://phabricator.wikimedia.org/T199358) (owner: 10Sbisson) [09:01:43] (03Merged) 10jenkins-bot: Store class 0 of models with more than 2 classes [extensions/ORES] - 10https://gerrit.wikimedia.org/r/445402 (https://phabricator.wikimedia.org/T199358) (owner: 10Sbisson) [09:03:12] (03CR) 10jenkins-bot: Store class 0 of models with more than 2 classes [extensions/ORES] - 10https://gerrit.wikimedia.org/r/445402 (https://phabricator.wikimedia.org/T199358) (owner: 10Sbisson) [09:19:56] (03CR) 10Ladsgroup: [C: 04-1] "Name of the class need fix, everything else is optional. Thanks!" (036 comments) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/444252 (https://phabricator.wikimedia.org/T198748) (owner: 10Sbisson) [09:38:46] 10Scoring-platform-team (Current), 10JADE: Develop "local" documentation for wikis where JADE will be deployed - https://phabricator.wikimedia.org/T199519 (10Harej) [09:38:48] 10Scoring-platform-team (Current), 10JADE: Develop "local" documentation for wikis where JADE will be deployed - https://phabricator.wikimedia.org/T199519 (10Harej) [09:44:24] 10Scoring-platform-team, 10JADE: Determine which wikis will get JADE and when - https://phabricator.wikimedia.org/T199520 (10Harej) [09:44:37] 10Scoring-platform-team, 10JADE: Determine which wikis will get JADE and when - https://phabricator.wikimedia.org/T199520 (10Harej) a:05awight>03None [12:07:24] 10Scoring-platform-team (Current), 10ORES, 10Wikidata, 10User-Ladsgroup: new ORES labeling campaign for Wikidata - https://phabricator.wikimedia.org/T195701 (10Ladsgroup) https://github.com/wiki-ai/editquality/pull/166 [13:24:23] (03PS11) 10Sbisson: Introducing DatabaseQueryBuilder [extensions/ORES] - 10https://gerrit.wikimedia.org/r/444252 (https://phabricator.wikimedia.org/T198748) [13:25:07] (03CR) 10Sbisson: Introducing DatabaseQueryBuilder (036 comments) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/444252 (https://phabricator.wikimedia.org/T198748) (owner: 10Sbisson) [13:31:25] (03CR) 10jerkins-bot: [V: 04-1] Introducing DatabaseQueryBuilder [extensions/ORES] - 10https://gerrit.wikimedia.org/r/444252 (https://phabricator.wikimedia.org/T198748) (owner: 10Sbisson) [13:36:05] (03PS12) 10Sbisson: Introducing DatabaseQueryBuilder [extensions/ORES] - 10https://gerrit.wikimedia.org/r/444252 (https://phabricator.wikimedia.org/T198748) [13:59:19] o/ [14:04:45] halfak: on phabricator i proposed making wikilabels/jade integration a priority [14:04:54] the idea being it's easier if we are our own first customer [14:05:20] harej, interesting idea, but I don't think others will use JADE like wikilabels will. [14:05:49] I don't understand. [14:07:58] So, presumably, Wikilabels will use the API like anyone else. [14:08:08] But the patterns of use will be very different. [14:09:17] I wonder if it would be more productive to partner with, say, Huggle/Petr to make sure that it works for a QC tool. [14:09:35] Person makes a judgment, clicks a button in an interface to record a judgment, judgment page gets created based on input. What's the difference? [14:12:17] Is it that Wiki Labels is used to train ORES while JADE will be used to criticize ORES? [14:12:25] So there’s a risk of a feedback loop thing? [14:12:58] harej, the work pattern around it and the set of concerns involved. [14:13:38] E.g. Huggle can't be slowed down. Huggle users may want to take advantage of the additional information JADE affords them. [14:14:15] I don't know though. I think the wikilabels use case is more of a "oh hey this could work for us too" [14:14:33] Where as the huggle use case is more "let's figure out where ORES is failing" [14:20:23] harej, ^ [14:20:56] I guess a thing I'm worried about is how much effort it will take to re-engineer Wikilabels to use a different storage back-end. [14:21:23] We need to figure out how we'll deal with the multiple labels problem too. [14:21:44] E.g. jade can only store one judgment per thing. Wikilabels can store multiple labels per thing. [14:27:08] I wonder if we can be a better first customer by implementing some model bias tracking reports. [14:27:29] E.g. we'll pull judgements, compare against ORES models and plot some stats. [14:31:46] I think that makes sense. While I think they're both valid use cases for JADE, the Huggle one is more interesting. [14:32:40] So if we were to be our own first customer, it wouldn't help us tell if JADE is a good fit for tools like Huggle, but could help tell us how easy the JADE API is to work with in general. [14:39:46] I think that is right. [14:40:20] FWIW, we aren't implementing a novel API but we might want to make some niceties on top of the MW API ala wikibase. [14:49:23] It'd be nice to have, certainly. Especially since the alternative is a lot of boilerplate code. [14:56:59] Oh? How so? [14:57:03] harej, ^ [14:57:29] Or wait. Maybe it wouldn't. [14:57:32] All I was imagining was a way to query within a page. E.g. "only get me the 'damaging' judgement" [14:57:48] Which wouldn't be more performant but it would reduce network traffic. [14:59:31] What could happen is, in the absence of dedicated JADE API methods, would be people copying and pasting the same piece of code for "record this judgment to the wiki". But if we have proper libraries abstracting that away we wouldn't have to worry about that. [15:05:21] harej, like mwapi? [15:05:33] Right. [15:05:39] OK done [15:05:39] :) [15:06:13] But more seriously, I think it would make sense to have an API that allowed someone to store a specific judgment (e.g. "damaging": "false") [15:10:52] That would be good too. [15:11:07] Do we know if Petr Bena is going to Wikimania? [15:23:35] Happy Friday the 13th! [15:23:44] (and I don't know about anyone's whereabouts) [15:24:51] halfak: on phabricator i proposed making wikilabels/jade integration a priority [15:25:09] +1 this is a *great* way to defang the scalability concerns [15:26:39] but also +1 with halfak that the use cases are wildly different, and we're not really proofing the storage concept since wikilabels is not growing [15:27:09] awight, I don't have any good ideas for replicating some types of Wiki labels functionality in JADE unless we have endorsements. [15:27:55] The other unfortunate reason we probably shouldn't elevate priority is that wikilabels is not sore for a storage abstraction [15:29:17] Either way, I'm intrigued that some of the behaviors we want to see in the "wild" might be present in wikilabels. [15:30:49] And also on second thought, it's a benefit that the storage requirements aren't growing, it makes this a safe test. [15:31:04] * awight continues following prodigious backscroll [15:34:47] I think I have a killer use case, in favor of wikilabels: this is work we would be doing anyway in order to feedback JADE labels into ML training. [15:35:17] In fact, it gives the users the fullest possible control over the process per se of feedback [15:35:23] awight, how will we deal with the N labels per task issue? [15:35:35] Hmm? [15:35:51] We keep the wikilabels UI, obviously, so I think I'm missing the question. [15:36:07] In wikilabels, we can have more than one label per task -- to be rolled together after the fact [15:36:18] We used this in the edit types campaign for lots of reasons. [15:36:38] Right now, we don't have a way to store multiple judgments in JADE. [15:36:41] is a "task" an observation (diff)? [15:36:47] Yes. [15:36:50] cos I did add multiple judgments. [15:36:56] I'm just waiting (forever) for CR [15:37:20] https://gerrit.wikimedia.org/r/#/projects/mediawiki/extensions/JADE,dashboards/default [15:37:24] Just +1 if you're feeling shy. [15:37:29] harej: ^ CR requested. [15:38:57] halfak: To your point, I'm thinking there are two ways to integrate: * wikilabels only writes to JADE, appending new judgments, or * wikilabels presents previous judgments as new users go through tasks. [15:40:07] awight, one of the reasons that wikilabels captures multiple judgments per task is so that we can do inter-rater reliability checks. [15:40:30] If at any point a labeler knows what another labeler thought about an observation, it invalidates that ' [15:41:26] awight, multiple judgments without endorsements!? -1 [15:42:09] Which change is for multiple judgments? [15:42:13] halfak: We talked about this, it's a hacky workaround [15:42:22] multiple judgments can emulate endorsements, but in a sort of better way. [15:42:23] Hmm. I don't remember this. [15:42:34] it's okay, it was a side note [15:43:37] Aha: something we haven't thought of yet. Endorsements would have been incorrect if the underlying judgment were changed. The other schema actually fixes this. [15:43:55] by endorsements coming with a permanent record of what is being endorsed. [15:43:58] Underlying judgments should never be changed. [15:44:03] eh [15:44:05] that is B-R-D [15:44:40] See straw polls [15:44:49] The options in the straw polls are not open to be changed. [15:44:52] New options can be added. [15:45:03] If a person adds a judgment and wants to change their judgment, they change it. [15:45:28] it shouldn't make a difference whether there were endorsements or not. [15:45:31] The judgment doesn't belong to a person when we have endorsements. [15:45:49] right, I'm arguing that this was a flaw we hadn't considered. [15:45:49] I don't see how multiple judgments fixes anything. [15:46:00] I don't think it is a flaw. This is the shape of a straw poll. [15:46:02] Multiple judgments make endorsements atomic. [15:46:15] You technically *can* change the options but you'll get reverted and told not to do that. [15:46:22] that is incredibly annoying [15:46:40] Well, I don't see how multiple judgments helps a consensus process. [15:46:52] Anyway, the reason I (maybe unilaterally) decided to leave multiple judgments for now is that we can emulate many different processes using that schema. [15:46:55] Which judgment is the *right* one? [15:47:04] halfak: preferred=1 [15:47:17] What if many judgments agree? [15:47:23] ty by the way for beating on the schema [15:47:35] What would the meaning of setting preferred=1 on one of them (owned by one user) mean? [15:47:39] halfak: then it should be easy for a human to set the preferred bit on one of them [15:47:59] Maybe ownership of judgments was a mistake. But let's not touch that for now? [15:48:13] I don't know, man. I don't know how we went in this direction. [15:48:28] I do know how, happy to explain more [15:48:34] We had a long discussion about simplifying the schema with sign-off [15:48:45] "with sign-off" meaning we both agreed? yes [15:48:49] yeah [15:49:04] I didn't agree to multiple judgments. I would have raised these concerns. [15:49:05] I'm pretty certain we also both agreed during that conversation to leave multiple judgment support, as a proxy for other stuff [15:49:29] I did not. If it seems like I did, I'm sorry. [15:49:29] OK I'm much more interested in solving fresh than referring to the last convo in that case [15:49:35] And I'm totally happy to do this btw [15:49:58] Well, I think in this case, we're just repairing a misunderstanding [15:50:27] One thing to keep in mind: multiple judgments is an optional implementation, we can act like it doesn't exist when implementing the first phase. [15:50:50] See https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/442885/ [15:50:52] Right, but I don't think multiple judgments without endorsements will ever make sense. [15:51:07] * halfak looks for a schema [15:51:08] That code corrently appends judgments, but we can change it to overwrite. [15:51:52] https://phabricator.wikimedia.org/diffusion/EJAD/browse/master/jsonschema/judgment/v1.json [15:51:55] halfak: ^ [15:52:22] I still see "entity" in here. Didn't we agree to remove that too? [15:52:53] in CR [15:52:57] https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/442884/ [15:53:06] Also on the schema: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/442990/ [15:53:06] Gotcha. [15:54:44] So, I thought we were also going to remove guid entirely [15:54:58] * halfak tries to find our notes. [15:56:10] halfak: Where is guid? [15:56:19] I think it's gone already [15:56:24] Aha! Good q. [15:56:43] So multiple-judgment-wise. What differentiates two judgments with the same data? [15:56:49] yeah referring back to above, that solves the "ownership" question. [15:56:59] halfak: Good point! [15:57:10] um, comment perhaps [15:57:27] ah there's where "preferred" makes sense. [15:57:50] It can be applied to the judgment with the consensed best "notes" field [15:58:09] (03CR) 10Halfak: [C: 032] Drop entity from the schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/442884 (https://phabricator.wikimedia.org/T179301) (owner: 10Awight) [15:58:10] This is also nice for discouraging empty "+1ism" [15:58:13] :D [15:58:21] Feel free to go on a rampage. [15:58:31] (03CR) 10Halfak: [C: 032] Fix schema: syntax of boolean data [extensions/JADE] - 10https://gerrit.wikimedia.org/r/442990 (https://phabricator.wikimedia.org/T179301) (owner: 10Awight) [15:58:34] * awight pokes at harej's /away flag [15:58:53] * harej is reading scrollback [15:58:57] :) [15:59:07] What goes around comes around. [15:59:44] halfak: Seems that judgment.notes is an excellent argument for allowing multiple judgments? [16:00:11] Actually, I would expect notes to be collaboratively edited -- like wikidata descriptions. [16:00:46] It's not wikitext--or is it? [16:00:53] Perhaps it is. [16:00:59] Maybe wikitext. I think so. [16:01:07] * awight adds to specification [16:02:30] (03PS1) 10Awight: Document that judgment.notes supports wikitext. [extensions/JADE] - 10https://gerrit.wikimedia.org/r/445637 [16:03:07] OK, we can leave collaborative vs. disjoint notes as an open question... [16:03:26] Thank you for linking to the JSON Schema, by the way [16:03:40] yes. It helps as a focus of discussion :) [16:04:27] ;-) [16:04:39] (03Merged) 10jenkins-bot: Fix schema: syntax of boolean data [extensions/JADE] - 10https://gerrit.wikimedia.org/r/442990 (https://phabricator.wikimedia.org/T179301) (owner: 10Awight) [16:04:41] (03Merged) 10jenkins-bot: Drop entity from the schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/442884 (https://phabricator.wikimedia.org/T179301) (owner: 10Awight) [16:05:17] Point in favor of mixed collaborative and non-collaborative judgment.notes: With some workflows such as wikilabels, it is invalid to show previous comments. [16:05:56] A new wikilabels judgment is an independent row, not fit to be merged with anything else. [16:06:12] (03CR) 10jenkins-bot: Fix schema: syntax of boolean data [extensions/JADE] - 10https://gerrit.wikimedia.org/r/442990 (https://phabricator.wikimedia.org/T179301) (owner: 10Awight) [16:06:20] I'm glad we're having this discussion in any case; it helps improve the robustness of the final product [16:06:20] I'm not sure we should design for wikilabels primarily. [16:06:43] What if we have a primary judgment and then have additional ones as appendices? [16:06:45] no, but we should support a diversity of uses [16:07:00] awight, or keep our scope focused. [16:07:02] i.e. deliberately not treating them as equal (except where an end user wants to?) [16:07:02] harej: We do, by using the "preferred" bit on a judgment [16:07:07] I see. [16:07:30] But that has the nice property of defaulting to equal weight for all judgments. [16:07:38] (03CR) 10jenkins-bot: Drop entity from the schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/442884 (https://phabricator.wikimedia.org/T179301) (owner: 10Awight) [16:08:58] One thing I like about judgement + endorsement is that there's a clear way to signal agreement without duplicating data. [16:09:54] Is there any reason Wikilabels couldn't ride off of judgement + endorsement? A repeated judgement would be added as an endorsement, and a contradicting one would just be a second judgement. [16:10:04] I think that could work just fine. [16:10:18] That's kind of what I had in mind. [16:10:53] Also I'm assuming provenance is captured in the edit summary? [16:11:10] We used to have a field for that in "endorsement" [16:11:29] I don't consider comments to be structured data [16:11:38] halfak: How do we handle changing judgments? Strict "straw poll" style doesn't feel good to me. [16:11:42] because we can only really handle them in post-hoc analysis. [16:12:05] awight, what do you mean by "changing judgment"? [16:12:43] halfak: I see what you're saying though, the multiple judgment schema doesn't have an affordance for "endorse an existing judgment and its .notes, without making it preferred" [16:12:56] Idea: what if all the possible judgements could be encoded on a judgement page, the initial person to make a judgement in a certain direction would be endorser #1, and no endorsements is to meant that the judgement is effectively not made? [16:13:21] halfak: we already hashed "changing judgment" avove, it's the flaw where a user can change their judgment.data by editing, but not if endorsements exist. [16:13:33] harej, all possible judgements for the draft topic model is a very large set. [16:13:54] Perhaps, but they wouldn't literally all have to be there. It could be absent, or it could be present with zero endorsements, and they would be equivalent. [16:14:19] This is just to avoid duplication of judgment.data? [16:14:22] awight, right. I don't see that as a flaw. E.g. someone could just clear our all of the judgments. That would also break things. [16:14:50] We can't prevent people from editing part of the schema they shouldn't be editing. [16:15:18] Wikibase does this by not letting humans edit the raw JSON text [16:15:27] lame :) [16:15:29] Changing judgement should not be afforded by the UI. [16:15:29] (If it even is possible, they sure don't make it obvious.) [16:15:41] harej, you can edit the raw JSON if you want. [16:15:58] +1 harej to non-obviousness [16:16:26] I know they disable the action=edit method but I think you can still do it through the API? [16:16:38] harej, yeah. [16:16:43] Some bots do it. [16:16:54] halfak: Can we make explicit why endorsements are any different than multiple judgments? So far, the only points I understand are * DRY of .data, and * something weaker than "preferred" [16:17:12] awight, they are structured like a straw poll [16:17:39] In order to achieve straw poll with multiple judgments, we have to agree to *use* multiple judgments in a specific way [16:17:50] So the business motivation is "this lets us support the straw poll process"? [16:18:05] awight, more, let's support consensus processes our users already use. [16:18:22] We do: B-R-D and !vote [16:18:34] This is why I like either the 1 judgement (the content is consensus) or the judgement + endorsement (the content is the process) [16:18:49] With one judgment, we fit into BRD [16:19:54] The current schema supports 1 judgment, obviously [16:20:00] With one judgment, the discussion/endorsement process would be on the talk page. [16:20:11] Talk pages hardly seem ideal here, but it's what we have. [16:20:19] it also supports !vote, and it happens to use exactly the same format [16:20:19] right. [16:20:33] harej: Judgment_talk: is a thing. [16:20:40] awight, I disagree. It does not support the structure of a !vote [16:20:45] halfak: why? [16:20:49] how, rather [16:20:51] The proposals are listed at the top. [16:21:01] Then !vote, reason follows referencing the proposals. [16:21:14] I think in practice you're going to want to be able to express more nuanced things, since you could have for example very political topics where people fight over the approved value. I guess the question is where the fight takes place. [16:21:28] harej, indeed. [16:21:42] The contention would be an interesting data point. Compare mundane edits with one judgement vs. one with 50 judgements. [16:21:48] endorsements don't support threading, so they fail !vote discussion patterns. [16:21:55] (or, 2 contradicting judgements with 50 endorsements total, I think you get my point) [16:21:58] I'm of half a mind to either embrace the complexity (judgment+endorsement) or KISS (1 judgment) [16:22:04] awight, that's a fair point, I agree. [16:22:05] where are proposals in the endorsements schema? Those are the root judgments? [16:22:20] That's where my mind is. [16:22:20] But the !vote itself is some important data. [16:22:30] awight, right [16:22:52] (Where are we on judgements with no endorsements being functionally null?) [16:23:03] harej, I'm with you there. I think there's meaning to highly contested judgments (probably very rare) [16:23:50] harej, I think the idea is that a standard workflow for an empty page would be to submit a judgment, endorsement, and preferred flag in the first edit per BOLD. [16:24:10] harej: IMO those are just an individual judgment with no consensus, so !=null and a degenerate form of consensus. [16:24:10] Or to just submit the one judgment in the 1-judgment case [16:24:31] halfak: IMO we don't recommend "preferred" for single judgments. [16:24:50] but that's a workflow detail that can be experimented on without schema changes. [16:24:53] Right. There's no preferred for single judgments. What would it even mean? [16:25:03] ok cool, I misunderstood you is all. [16:25:22] Oh wait. in judgment+endorsement, I think preferred should totally be set on the first judgment. [16:25:34] mmm [16:25:47] I don't like it, this should only happen in a consensus process [16:25:59] The first edit is a consensus. [16:26:02] but I'm a wiki noob so will leave it to organicness [16:26:11] WP:BOLD [16:26:24] I don't think this is done in wikidata, fwiw [16:26:30] Consensus changes with disagreement and discussion. [16:26:39] on Wikipedia, silence is consensus, pretty much [16:26:42] Default ranking is used. [16:26:52] hehe in life, silence is the voice of complicity for sure [16:26:54] if I make an edit to a page and it's so boring no one cares, so long as no one contests it the consensus is that it remains [16:31:31] on Wikibase, the first statement for a given property is treated as normal, not preferred, as preferred has a very particular meaning [16:31:36] and not just "whoever showed up first" [16:32:01] Right. I think that in this case, we only need 2 levels. [16:32:01] i imagine the procedural parallel for endorsed here is an admin coming along and closing the discussion saying "X is the winner" [16:32:01] +1 is my understanding. [16:32:07] Wikidata has more than 2 [16:32:24] harej, in our case, we seek to preserve judgments that run counter to consensus. [16:32:29] So like wikidata, but not the same. [16:33:05] and Wikidata does the same. In this case, a judgement being "preferred" just means it's certified as the consensus winner, but we would still preserve the dissenting voices [16:33:08] E.g., Chelsey Manning is both female and male but primarily female. [16:33:17] Not a great example but I get your point. [16:33:21] An edit is not both damaging and not damaging. [16:33:25] halfak: I think that BRD is a specific process and we don't need to enforce it, but supporting it is great. [16:33:33] harej, what is wrong with the example. [16:33:49] hehe I also have been losing points for using the Chelsey Manning example [16:33:56] awight, ? [16:34:07] ^ re BRD [16:35:01] What I mean about BRD is that asking users to set preferred=1 when making the first edit is encouraging the BRD process, so wiki communities can choose their own norms [16:35:24] awight, I disagree. The RD part of BRD is a process. [16:35:30] But publish first is what we do in wikis [16:35:30] I don't think we should care how the schema is abused, right up to the limit where we enforce strict validation. [16:35:47] awight, again I disagree and cite Wikibase [16:36:14] If people do weird things with an entity's judgments, then it's a rich source of data that broadens our understanding somehow... [16:36:26] They did that for a reason, in other words. [16:36:37] awight, just like talk pages. [16:37:06] Talk pages also show us weird things, yeah [16:37:34] But it would be better if a UI directed people to reply and post correctly. [16:37:45] And maybe put those replies and posts into structured data. [16:37:57] I'm a fan of leaving it all editable within the structure though. [16:37:58] Of course. We're currently at the stage of creating a schema which best supports every potential workflow we're considering, though [16:38:24] +1 that a UI should have built-in paradigms which are best for specific workflows [16:38:28] Just that, if you thin you want to edit a judgment data, you probably actually just want to create a new judgement. [16:38:34] e.g. the wikilabels UI cannot show existing judgments... [16:39:04] Or in some case, delete an old judgment. [16:42:25] I think we're at a solid place with the schema, the last simplifications especially got rid of everything that bothered me, and the boundary between our structured schema and MediaWiki metadata feels right, e.g. no usernames in the content. [16:43:02] If there's anything we can do to expand the schema to better support something you're thinking of wrt. endorsements, let's do that. [16:43:25] But I'm still not convinced that anything is lacking... [16:44:56] A UI is welcome to display the judgments in sequence, grouped by data value, which makes the first judgment with each value the de facto proposal, and users can massage that first judgment or whatever. [16:45:45] Alternatively, the proposal can live on the Judgment_talk page and so on [16:50:40] I'm sorry if I'm not following this back-and-forth all that well, but would this have particular implications for Wiki Labels integration? It sounds like having multiple judgements with endorsements, with duplicative judgements serving as endorsements, sounds workable. Is the hangup when people want to go back and edit their original judgements? [16:51:52] harej: I could be wrong, but it seems like wikilabels requires multiple judgments, which we do currently support. [16:59:02] Would we have to change JADE in any meaningful way to support Wiki Labels integration, even if the actual integration is a while off? [16:59:06] harej: Point of interest, do you like to CR these days? [16:59:19] I wouldn't say I'm terribly good at it [17:00:09] Hehe I would take that. I'll leave it up to whether you find it fun: https://gerrit.wikimedia.org/r/#/projects/mediawiki/extensions/JADE,dashboards/default [17:01:14] (03CR) 10Harej: [C: 031] Document that judgment.notes supports wikitext. [extensions/JADE] - 10https://gerrit.wikimedia.org/r/445637 (owner: 10Awight) [17:01:35] :) ty [17:02:05] (03CR) 10Harej: [C: 031] "Better semantics: namespaces are named for the things they are, not the extensions supplying them." [extensions/JADE] - 10https://gerrit.wikimedia.org/r/443649 (owner: 10Awight) [17:10:21] I'm not following the conclusion to stick with multi-judgments. Was there a counter-argument that I missed? [17:11:11] awight, I see that you've summarized our discussion but I'm not sure I agree with the summary. [17:11:53] In the multi-judgment set, we'd want to include guids with each judgment and that would put "usernames" in the content space. [17:12:06] If we don't have usernames, why even have multiple judgments? [17:12:36] multiple judgments so that wikilabels is supported, if nothing else. [17:13:14] We don't need usernames for wikilabels, but they're available in "blame" for analysis. [17:13:31] awight, that's kind of gross IMO [17:14:09] The multi-judgment seems to be a denormalized judgment+endorsement to me. [17:14:45] Are we not presumed to interpret multiple matching judgments as endorsement of one conceptual judgment? [17:15:17] That's just one interpretation, and sort of Platonic. In the wikilabels case, they are explicitly not endorsements. [17:15:41] awight, right. In the wikilabels case, there is no consensus. [17:16:04] Which is why wikilabels might not be compatible with JADE without a serious redesign. [17:16:08] Well, editors can come along and create a consensus after the fact, which is cool. [17:16:15] Without guid, how would a user remember whether or not they have supplied a judgment before? [17:16:34] They can look at the page history... [17:16:35] re. platonic-ness I don't see that as a counter-argument. [17:16:50] awight, that's not going to work [17:17:08] What if someone deleted their judgment in the meantime? What a mess! [17:18:17] This seems like an edge case, I can't come up with a UI where this would be a legit concern [17:18:50] e.g. for patrolling, you know that the diff is already patrolled, so you wouldn't be prompted to re-patrol regardless of who judged. [17:19:01] When someone considers the judgment of something more than once. [17:19:13] But how does that come about? [17:19:20] Any time there is a discussion and multiple judgments. [17:19:28] You know. The consensus process we are talking about. [17:19:49] If there's a discussion, then presumably they'll remember their judgment.notes and recognize them. [17:20:11] heh. I think we'll have people signing their judgement notes then. [17:20:28] Anyway, !vote is only going to be perfectly rendered on the talk page, all of the structured solutions we've considered are an imperfect reflection of the process. [17:21:02] What do you mean by "!vote is only going to be rendered on the talk page"? [17:21:21] !vote can only be done correctly on the talk page, cos we're not supporting threaded discussion for starters... [17:21:22] 04Error: Command “vote” not recognized. Please review and correct what you’ve written. [17:21:31] !self-destruct [17:21:35] Right. But I think the multiple-judgements version is particularly bad. The only counter-argument that I [17:21:44] ve heard is that it can be crammed into other shapes. [17:22:28] awight, indeed. But the (~vote, reason) tuple is very valuable as structure data. [17:23:11] I don't think we're progressing... Donno if it would help if I summarize the points so far in an etherpad, or how to unblock us. [17:23:29] +1 that , is great [17:23:38] awight, I think in order to progress we should go back to old consensus and then changes to that should be proposals. [17:23:40] Also will mention that doesn't really matter [17:23:45] urgh [17:23:54] we disagreed on what the consensus was, so that's not an option. [17:25:18] Do you believe that I ever supported multi-judgment without endorsements? [17:25:47] That is my recollection [17:26:08] Like I said, it was a passing few sentences in a longer discussion so I'm not married to it being "consensus" [17:26:30] I can dig it up but really don't think that's the right way to proceed? [17:27:40] Well, please accept my -1 to multi-judgment without endorsements. I think the right way to proceed is to either make the case or dial it back. [17:28:06] https://etherpad.wikimedia.org/p/JADE_multiple_judgments [17:28:29] It's a good idea to make the case, so I'll do that for now. [17:30:04] harej: You are invited to ^ etherpad epic proposal tango [17:31:09] halfak: Q: !vote in your proposal requires multiple judgments + endorsements, right? [17:35:05] Also, are endorsements only "support" or can there be "oppose", etc? [17:35:59] Right. Support and oppose only make sense when there's only one proposal. [17:36:13] kk that works for me. [18:45:40] halfak: Does it make sense to you if I grab harej to discuss the schema with me and see if that generates new ideas, or would you rather be there? [18:48:11] Also, I'm going to try to take notes from our chat and present them in a neutral way, i.e. not as pros and cons. [19:43:58] o/ [19:44:00] Just got back online at the U. Working on Wikimania stuff :) [19:56:28] right on! [19:56:51] lmk if you want me to beat at the CSCW paper with a blunt object. [20:33:36] CSCW paper is submitted. Was due on Wednesday :) [20:34:09] awight, ^ [20:34:27] Was side-tracked in the lab. An old labmate is back in town. needed to catch up quick ^_^ [20:35:12] Nice, congrats on the paper! [20:42:03] To us all! The email thread captures most of the work I was doing and what I had Stuart and Jonathan pushing on :) [20:42:22] Oh! I updated the commons file so you can see the final product. It's also in the repo [20:42:32] https://github.com/halfak/ores-paper [20:42:42] https://commons.wikimedia.org/wiki/File:ORES_-_Facilitating_re-mediation_of_Wikipedia%27s_socio-technical_problems.pdf [20:47:51] "vagrant hat" [20:47:57] that did not work, somehow. [20:48:24] Do not hat when you intend to halt... [20:50:15] 10Scoring-platform-team, 10Fundraising-Backlog: Machine Learning for Fraud Detection - https://phabricator.wikimedia.org/T190523 (10saurabhbatra96) Tracking API frontend code here - https://github.com/saurabhbatra96/wmf-fd-api [21:06:37] harej: Mind if I book you for 30 min on Monday? [21:07:39] That should be fine. Keep in mind I am in UTC+2 at the moment [21:07:59] perfect :) [21:08:19] oh. wait that's crazy, thx [21:08:21] mathing. [21:10:42] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/445695 (owner: 10L10n-bot) [21:29:04] * awight braces for IRC interestingness [22:01:32] heh [22:02:08] I'm heading out. I hope to be online a bit tomorrow morning (~UTC 1500ish for me) to work on Wikimania presentation. Otherwise, I'll be around for a half-day on Monday before I fly away [22:02:12] have a good one folks! [22:02:12] o/ [23:02:41] harej: Are you already UTC+2? [23:02:42] And, do you think you might be occupied with Wikimania at the time I chose Monday evening? We can do this another way, async, IRC, if so... [23:16:59] harej: lmk if this is readable, https://etherpad.wikimedia.org/p/JADE_multiple_judgments [23:28:53] Signing off... [23:29:01] * awight shuffles out of the empty room