[11:03:59] o/ [11:22:15] harej: hey, I made this edit [11:22:16] https://www.mediawiki.org/w/index.php?title=JADE/Implementations&diff=2854363&oldid=2847471 [11:22:25] It's an amazing document [11:22:34] Thank you! [11:59:23] 10Scoring-platform-team (Current), 10ORES, 10User-Ladsgroup: Implement PoolCounter support in ORES - https://phabricator.wikimedia.org/T201823 (10Ladsgroup) p:05Triage>03High [12:04:39] 10Scoring-platform-team (Current), 10ORES, 10Operations: Spin up a new poolcounter node for ores - https://phabricator.wikimedia.org/T201824 (10Ladsgroup) p:05Triage>03High [12:06:04] 10Scoring-platform-team (Current), 10ORES: Test poolcounter support for ores in beta cluster - https://phabricator.wikimedia.org/T201825 (10Ladsgroup) p:05Triage>03High [12:08:15] 10Scoring-platform-team, 10ORES: Implement support for whitelisting and proxy requests for poolcounter in ORES - https://phabricator.wikimedia.org/T201826 (10Ladsgroup) p:05Triage>03High [13:52:04] afk for lunch, be back soon [14:04:12] o/ [15:22:37] back now [15:22:57] okay, now it's time to do react stuff [15:23:50] Amir1, welcome back [15:24:00] :D [15:24:40] They are making a cafe in the backyard of WMDE, if that happens, it would save lots of my lunch break time [15:24:49] right now, it's only noise [15:24:58] construction noise [15:36:57] I'm not feeling great today. I'm not sure what's up. I might need to bail on our meetings. [15:37:08] The heat has been pretty oppressive here too :| [15:37:34] I got the agenda set up so it should be easy to chair for whoever can take over for me. [15:39:10] halfak: we can switch places [15:39:21] thanks, harej [15:39:47] I added two non-recurring agenda items that should be self-explanatory [15:40:27] I plan to be mostly AFK on wednesday for the WMF holiday. I can use the time to get some errands done, but I'll be working a half-day on paper writing. [15:40:49] My plan is to adapt the ORES paper for FAT*. The due date is in two weeks. [15:40:53] [15:50:27] halfak: to give you an answer regarding https://phabricator.wikimedia.org/T197979 , I started thinking about how to implement it but I haven't gotten around to actually doing it yet [15:51:02] Just one note: It's in our quarterly goals :) [15:51:13] And we've burned 33% of our time. [15:52:35] Amir1, FYI: I mad a lot of progress on the mwbase PR. I'll push something when I next get to sit down with it. [15:52:49] nice [15:53:04] halfak: remind me where our list of quarterly goals is? [15:54:52] https://www.mediawiki.org/w/index.php?title=Wikimedia_Technology/Annual_Plans/FY2019/TEC5:_Scoring_Platform/Goals [15:55:04] Debt just did some re-formatting [15:55:44] See: Outcome 2 / Output 4 [15:57:07] btw. I'm working on poolcounter, it will take quite some time [16:01:59] halfak: It’s that time :-) [16:02:23] AFK for a bit. Not feeling well [16:02:31] harej switchd with me [16:03:30] ono! [16:05:19] halfak: please update https://etherpad.wikimedia.org/p/scoring_current_work if you are feeling up to it [16:28:34] * awight jumps at the sound of a whip cracking [16:58:01] hi [16:59:40] hello Fajne [17:12:47] harej: hey, can you elaborate on https://www.mediawiki.org/wiki/JADE/Implementations#Per-wiki_%E2%80%93_log_entries "However, this may result in a slightly more complicated user experience and poor AbuseFilter integration. " [17:12:59] AbuseFilter is obvious [17:13:49] but combination of custom table (integrated to users using a special page) + log entries seems practically the same UX [17:14:10] complicated because they won’t be editing as a wiki page [17:14:14] I mean we can have Special:Judgment/diff/666 [17:14:16] it will be some other random interface to learn [17:14:28] I think it would be more complicated in that we would need to build out an interface for people to perform the log action, as opposed to just interacting with edits [17:14:30] (which is the plan anyway, so I’m wrong :) [17:15:11] harej: yeah, that can be addressed by javascript and API [17:15:29] it's not trivial but not hard and we were planning to do it anyway [17:15:33] What will “judgment history” look like… [17:16:05] awight: that is very valid: One way: another special page to query the logging table [17:16:18] it can be just a link to logs [17:20:29] I think you’re right to question “complicated UX”, more accurately it seems to be: For our initial integration, the judgment-as-page schema will give us ordinary MediaWiki editing for free, which everyone is familiar with for better or worse. Other schemas won’t give us anything, or if they do it will be partial and a custom UI. In the far future, there will be multiple, specialized UIs which are nearly equivalent regardless of our schema. [17:23:24] Amir1: ^ [17:24:08] awight: yeah, my point here is that we need to move away from json editing anyway: https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Judgment:Diff/376901&action=edit [17:24:13] This is not user-friendly [17:24:20] oh of course [17:24:49] why just having the API to do the edit [17:25:06] like wikibase, every time you edit something it calls an API [17:25:07] +1 that it’s “complicated” in every case, until we have specialized UIs [17:25:35] Amir1: That’s what I did in https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/442885/ [17:25:41] * awight coughs [17:25:59] Yup I know but I mean utilizing it in the special page :D [17:26:17] Exactly, I think we agree. [17:26:31] awight: Amir1, i made edits to the page [17:26:39] so we have a page like Special:Judgment/Diff/376901 [17:27:00] We can’t *only* go through the API, because tools should be allowed to do more sophisticated stuff, but generally the API will be the right place to integrate. [17:27:02] and we call API to do the logging and changes to the custom table, etc. etc. [17:27:05] s/API/JADE API/ [17:27:34] more powerful APIs IMO [17:27:51] wikibase has two types of APIs: 1- wbsetclaim, etc. 2-wbeditentity [17:28:10] But wikibase doesn’t prevent api.php?action=edit, right? [17:28:21] it does [17:28:29] nooo. what? [17:29:17] you can still edit the raw JSON via API [17:29:20] this is what tools and bots do [17:29:51] Weird, https://www.wikidata.org/wiki/Wikidata:Data_access#MediaWiki_API “For entity pages, that raw page content is not guaranteed to use any documented format or follow any standard structure.” [17:29:56] wheee [17:31:16] I swear that contradicts their stable interface policy [17:31:42] “The JSON binding of the Wikibase Data Model as used in JSON dumps, with the web API, and with the Linked Data Interface, is considered a stable data format." [17:31:57] The reason is for me, logging has waaaay less overhead than pages. From elastic, to parser cache, to revision table, etc. [17:31:58] iono [17:32:20] Amir1: why are we trying to optimize? [17:32:22] and I cleaned logging table to a very small size [17:32:27] hehe [17:33:09] The log table alternative is fine for an append-only store, but terrible for collaboration. [17:33:37] It's not because of SRE, it's basically scalability in long run. I want jade to be powerful, I don't want to tell users to limit themselves [17:34:09] Totally fair. [17:34:27] How are we telling them to limit themselves, though? AFAIK it’s only “don’t spam with bots" [17:34:48] That’s the same rule we have for the rest of the wiki... [17:35:16] stopping recursive judgements, also IIRC the plan was to have the limit that SREs gave and don't let users to make more (like rate limit) [17:35:32] what’s wrong with judgments of judgments? [17:35:51] awight: I heard we don't want to let that happen [17:35:53] We’re not rate limiting though cos it’s already 5x more than our planned maximum traffic [17:36:01] naw “recursive” judgments are key [17:36:03] we want them, [17:36:06] all good [17:36:09] and also spamming for bots I think it makes sense, people make pages with bots [17:36:20] They scare DBAs but not on a rational level [17:36:30] haha :D [17:36:58] It makes no sense to have bots add judgments though, so we’re not exactly hobbling people by asking them to not do that [17:37:02] For example [17:37:16] ORES could be considered a data source for JADE, if we allowed automated judgments [17:37:41] ORES’s infrastructure is pretty ideal for what it does, score everything, cache everything... [17:37:55] ORES piped into JADE would be a hot mess, regardless of what our backend was [17:38:38] When we were first discussing the project, it made sense to me that we would basically capture ORES and any related AI in the system [17:39:04] hmm, if ORES is more confident than a certain threshold and then we pipe it to JADE and then re-train ORES based on that is a very good case of self-training (semi-supervised learning) [17:39:07] But now I’ve realized that we should have AIs pipe their data through ORES, and humans enter their data into JADE [17:39:45] but that's for a proportion of edits not all [17:40:03] Amir1: it’s also an example of why this is a bad idea. Consider that we might have a bad model version. That data would gum up JADE, and be very difficult to remove, we’d have to hunt down exactly which entries were made by that version x model. [17:40:10] when it's sure it's good or bad [17:40:27] hmm, valid point [17:40:54] IMO ORES is very much downstream of JADE, I’m thinking of JADE as just a generalized+collaborative wikilabels [17:41:14] ORES will be able to serve JADE judgments, plus any AI scores that we have available [17:41:33] Just finished meeting with Pchelolo and ottomata. We came to what I think is a good agreement about ChangeProp, ORES scores --> EventStream. awight and Amir1, you should expect to hear from them soon re. ORES-Ext -- where we think we can do the transformations they want. [17:41:42] +1 on general points made re. JADE/ORES, awight [17:41:47] I'm going back to lay down. [17:42:19] awight: https://adamwight.github.io/ores-reference-ui/dist/ [17:42:20] But the only type of feedback that goes from ORES back into JADE are: when ORES is uncertain about something, we can active-learn that observation back into the wikilabels pool, but *without* the ORES score. [17:42:30] halfak: geez, sorry to hear it [17:42:39] click on one the different inputs and notice the border [17:43:43] Amir1: I love that page :). Ah the blue border? [17:43:45] awight, I think I did too much in the heat this weekend. I took all day yesterday to try to recover. Still on the mend. [17:44:04] yikes, that’s dangerous [17:44:42] awight: the blue border is different in different inputs [17:44:58] compare server url and model input [17:45:18] I want to fix it but upstream is a mess [17:45:22] weird, I probably botched the styles [17:45:40] jdlrobson’s library? [17:46:00] awight: nope, you're using react-select for some and jon's for the other which makes sense [17:46:06] hehe [17:46:12] because Jon's don't have everything [17:46:22] I wonder, maybe we should just override styles? [17:46:38] I want to add them but oojs ui is pretty entangled to take out things [17:46:41] Probably in the direction of, coercing react-select to look like jdl [17:47:25] Amir1: BTW, thanks for chatting about the JADE schema and implementation, it’s great to have your mind on the problem, let’s do this more often! [17:47:35] There are several ways to fix it: 1- Ideally, make oojs ui work with jdl and load less files 2- add some to jdl's 3- change css [17:47:49] A lot has changed in the last few months, it’s helpful to do another round of review [17:47:57] Amir1: so before I create some tickets, could you explain why we have both ORESFetchScoreJob and change-prop competing for hitting ORES oon every edit? [17:47:58] awight: Thank you for listening to me :) [17:48:12] haha my pleasure, any time. [17:48:33] Pchelolo: because the models ores ext. and ores service have overlap but it's not exactly the same [17:48:55] Amir1: Like I was saying earlier, I’d much rather find out that it’s all stupid and broken, now rather than in a year. [17:49:37] mmm... Amir1 is one the subset of another or they do different things? [17:50:01] Amir1: Your point about bad scaling when revsion -> page was really a breakthrough for me, that’s the only reason I finally came around to seeing that AIs have no business feeding into JADE. [17:50:05] awight: it's not stupid, it's in my opinion have too much overhead and migrating away will be hard :) [17:50:15] Pchelolo: the ores ext. is a subset [17:50:50] Amir1: No double that migration away would be terrible. As for the overhead, I’m conflicted about it. [17:50:58] hm... ok, I see you're in a middle of a conversation, I will write up a ticket with the proposal to simplify this all and poke you there [17:51:12] Pchelolo: Thanks! [17:51:46] Amir1: I see that a simple schema in custom tables would be much more straightforward, c.f. how I was about to pull the trigger on https://github.com/wiki-ai/jade/blob/master/schema.sql [17:51:47] awight: in the past two years I've been dealing with horrors of mediawiki in scale of wikidata. I'm like once bitten twice shy :D [17:51:56] lol that’s no joke [17:52:09] Can I ask, how are you feeling about wikidata being on top of MW these days? [17:52:47] It seems like they would have been much better off to start with an entirely different backend? [17:52:55] It's better than two years ago but it has several pain points that need to be addressed ASAP otherwise we won't scale [17:53:00] change_tag is one of them [17:53:06] kk [17:53:18] 10Scoring-platform-team, 10Analytics, 10Analytics-Kanban, 10EventBus, and 4 others: Modify revision-score schema so that model probabilities won't conflict - https://phabricator.wikimedia.org/T197000 (10Ottomata) Just had a meeting with Aaron and Petr. We talked about how there is a Mediawiki JobQueue bas... [17:53:34] IMO postgres would have been a huge win [17:53:51] awight: it's basically cost-benefits of integration with Wikipedia vs. scalability [17:54:09] cool, I agree that we’re dealing with something similar for JADE [17:54:59] Yup, IMO which can be wrong and it's very subjective jade functionality is too small to need that integration [17:55:08] My imagination is currently limited to “stuff people want to do today” that would fit into JADE, but if some new thing comes about and it makes sense for humans to add judgments for every revision, and tools exist to make that possible, then our house of cards collapses. [17:55:30] awight: That's exactly my point all along [17:55:40] It’s a great point ;-) [17:55:43] as I said I want it to be powerful [17:56:23] I actually fought halfak for a months, to keep JADE out of MediaWiki [17:56:28] *a month or so [17:57:34] I’m happy to admit that it’s probably cognitive dissonance that now keeps me attached to the MediaWiki integration. [18:13:25] so it sounds like Wikilabels judgements will never be viewable in JADE to keep it friendly to databases and humans. I'm a bit confused now about whether wikilabels is destined to be replaced by Jade though? [18:13:47] JADE will be the backend store for wikilabels [18:13:57] makes a lot of sense [18:14:07] wikilabels judgments are for sure the correct type of thing to put into JADE [18:14:35] so wikilabels will be the main example of 'gadget' interacting with jade [18:14:36] There’s some scientific stuff about not showing previous “answers” though, so for example wikilabels will never show JADE judgments, it will be write-only. [18:15:18] mmm, the wikilabels use case is actually not a main example, think more Recent Changes Patrol [18:56:36] 10Scoring-platform-team (Current), 10ORES, 10Upstream: Make ORES dependency solving upstreamable - https://phabricator.wikimedia.org/T201657 (10awight) a:05awight>03None Unlicking this interesting cookie in case anyone else wants to take a shot. [19:20:03] 10Scoring-platform-team, 10ORES, 10Services (designing): Merge ORES precaching - https://phabricator.wikimedia.org/T201868 (10Pchelolo) [19:20:08] awight: what is wikilabels judgments? is there an example somewhere? [19:20:32] 10Scoring-platform-team, 10ORES, 10Services (designing): O - https://phabricator.wikimedia.org/T201869 (10Pchelolo) [19:24:55] Fajne: My fault, that was a portmanteau of existing wikilabels labeling, plus the JADE layer we’re calling judgments which will be merged in the future [19:25:00] Let me get you the current schema… [19:29:34] 10Scoring-platform-team, 10ORES, 10Services (designing): ORES hooks - https://phabricator.wikimedia.org/T201869 (10Pchelolo) [19:32:04] Fajne: https://github.com/wiki-ai/wikilabels/blob/master/wikilabels/database/schema.sql [19:32:24] 10Scoring-platform-team, 10Analytics, 10ORES, 10Services (designing): ORES hooks - https://phabricator.wikimedia.org/T201869 (10Pchelolo) [19:32:47] And, this is *not* our finest file but just to make it more concrete https://github.com/wiki-ai/wikilabels/blob/master/wikilabels/database/schema-testdata.sql [20:14:21] 10Scoring-platform-team, 10ORES, 10Performance-Team (Radar), 10Wikimedia-log-errors: ORES Storage::SqlScoreStorage exception every 2-3 minutes: Model contains an error for [id]: TimeoutError - https://phabricator.wikimedia.org/T201412 (10Imarlier) [21:25:24] halfak: you on? [22:32:54] Vermont: halfa*k is out sick, maybe I can help with something? [22:37:47] awight: Do you know who I could contact to see about prioritizing Simple English in the “in other languages” thing on enwiki? [22:39:07] Which thing is this, the Languages sidebar for articles? [22:42:20] Is there a task? [22:43:25] I see that Simple English barely shows up in the “96 more” universal language selector under that sidebar. [22:43:29] Vermont: ^ [22:43:46] Yeah... [22:43:52] I didn’t make a task [22:44:31] Maybe that’s an on-wiki discussion, I would defer to you on that judgment call, of course. [22:44:44] Is there already consensus? [22:45:41] Nope... [22:45:44] :/ [22:45:53] Well, loose consensus. [22:46:20] hehe that seems like the first step, either way. Once it becomes a technical thing, I’d suggest a Phab task. Implementation is probably easy. [22:47:52] The languages seem to be unordered in Wikidata [22:58:40] Vermont: wow, I’m trying to at least find where the languages become ordered and it’s an insane sequence of redirections in the code. [22:59:08] if it’s not possible... [22:59:10] meh [23:00:00] It’s got to be set somewhere, and if it *isn’t* then it should be. [23:00:35] I’m starting to suspect that it’s unordered though, e.g. https://en.wikipedia.org/wiki/Business_Plot [23:00:53] fr, simple, Bulgarian, Spanish, Portuguese, Chinese [23:01:06] I don’t think that’s by wiki size nor anything about fallback language [23:01:35] oh hey! https://phabricator.wikimedia.org/rOMWCd1d2d9a0b9bee0bbd2455c58da6b13302978573a [23:01:43] There it is. [23:03:03] darn, been moved since [23:11:00] haha github can’t handle > 14k lines [23:13:15] Vermont: sorry, I would have to ask around to learn more (happy to do if you wish!), but this should get you started: https://phabricator.wikimedia.org/source/mediawiki-config/browse/master/wmf-config/InitialiseSettings.php$18549 [23:13:39] Thanks awight! :D [23:13:51] It seems that enwiki is displaying in random order [23:13:57] ah, k [23:14:09] u could compare two pages with similar translation languages to verify [23:14:36] This line doesn’t seem to be active, https://phabricator.wikimedia.org/source/mediawiki-config/browse/master/wmf-config/InitialiseSettings.php$18557 [23:14:54] (because the condtional above is false by default) [23:17:50] Thanks, that was an interesting corner of MediaWiki :-) [23:19:54] 10Scoring-platform-team (Current), 10ORES, 10Patch-For-Review: ORES deployment (Early August) - https://phabricator.wikimedia.org/T201518 (10Catrope) [23:19:58] 10Scoring-platform-team (Current), 10ORES, 10Patch-For-Review, 10User-Ladsgroup: Enable wp10 and draftquality models for testwiki - https://phabricator.wikimedia.org/T198997 (10Catrope) 05Open>03Resolved