[00:21:33] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Meta ORES: UI for reviewing how ORES classifies you - https://phabricator.wikimedia.org/T148700#2730380 (10Halfak) [00:21:47] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Meta ORES: UI for reviewing how ORES classifies you and your stuff - https://phabricator.wikimedia.org/T148700#2730395 (10Halfak) [00:32:22] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Meta ORES: UI for reviewing how ORES classifies you and your stuff - https://phabricator.wikimedia.org/T148700#2730439 (10Halfak) [09:33:26] 06Revision-Scoring-As-A-Service, 10ChangeProp: Replace ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2730864 (10awight) [09:43:55] 06Revision-Scoring-As-A-Service, 10ChangeProp: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2730885 (10awight) [10:24:15] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2730929 (10mobrovac) > Presumably, that matrix is already duplicated in several places. It'll be easy to reuse th... [10:24:36] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2730931 (10mobrovac) [13:46:55] o/ [13:47:10] I just realized that I forgot to read awight's scrollback [13:51:09] OMG figures! [13:51:12] <3 awight [13:52:14] Amir1, I'm going to be a couple minutes late for grooming [14:11:06] I'm back, but it looks like I'm digging into documentation today [14:11:14] Since no one else for grooming. [15:26:43] 06Revision-Scoring-As-A-Service, 10rsaas-articlequality : [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#2669300 (10Halfak) I talked to @chasemp in IRC yesterday and he confirmed @jcrespo's recommendation. I think we should wait until the new DB mac... [15:39:47] o/ awight [15:40:00] sorry to not get back to you last night. Thanks for making progress on ORES docs. [15:40:10] I need to think more about what documentation lives where. [15:40:21] E.g. where should the on-wiki documentation about the ORES service go? [15:41:01] Probably not on top of the Extension documentation. [15:41:10] But probably somewhere in MediaWiki. [15:41:32] halfak: thanks for the feedback! [15:41:50] I did a bit of hacking on some basic documentation that seemed to be necessary, but didn't have a clear spot to put it here: https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service/What [15:41:54] (a while ago) [15:42:04] Agreed, mw:ORES/ seems like a better home going forward [15:42:05] It seems like this documentation ought to be on mediawiki.org? [15:42:17] Or do you think we should keep it on Meta? [15:42:20] ah, fantastic [15:42:43] Great to know that there wasn't much redundant work [15:43:44] A reasonable division might be: meta) user documentation, mediawiki) implementation and technical details, wikitech) deployment info exclusively meaningful to the WMF instance [15:44:22] it's pretty hard to draw those lines, but yeah organizing the material early will make it easier to write more docs [15:44:36] (by limiting scope) [15:47:40] fwiw, I think the DFD you created for training the ML is a perfect detail diagram to explode what's inside the training process in the overview DFD [15:50:20] The inputs and outputs don't exactly match, but that's fine IMO [15:50:46] I'd like to move the majority of content from https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service to be within the docs at ores.wmflabs.org/v2/ [15:50:56] E.g. the lists of supported wikis and models. [15:51:26] That sounds perfect [15:52:08] I guess the online help can be edited via github source? [15:52:38] Yeah. That's a drawback, I think. [15:52:49] Also: translation [15:52:53] Maybe we can do some sort of hybrid re-hosting of wiki-content. [15:52:58] Can swagger (or what is doing that?) do i18n? [15:53:20] Yeah... most of the stuff I want to pull in will be stats, jargon and names. [15:53:24] s/what is doing/what source is used to generate/ [15:53:26] So they won't need translation [15:53:40] But yeah, we'll want some i18n strings. [15:53:52] And any long-form description is best left on the wiki. [15:53:54] kk, maybe you saw my bug last night about the config DRY, too? [15:54:09] that must be maddening to have so many copies of the deployment matrix [15:54:41] T148714 [15:54:41] T148714: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714 [15:54:49] * awight beams at stashbot [15:54:59] (gtg in moments) [15:55:08] * halfak cliks [15:56:01] * halfak is not a rest purist [15:56:35] meh the PUT thing was just a side note [15:57:02] & your explanation about metrics changes my opinion... [15:59:18] mostly I was saying that we should replace the redundant model deployment config with a Python filter that can read from existing ORES settings [15:59:41] And produce ChangeProp config? [15:59:46] no [16:00:11] the changeprop config would say: ores_cache.... action: OresCpFilterClass [16:00:23] minor, anyway [16:00:33] just looking for a way to eliminate that DRY nightmare [16:01:01] Yeah. Definitely is lame. This was one of my complaints about switching to Change Propagation [16:03:48] k well if you agree, I'll try to write that module [16:04:24] Yeah. I think that's a good idea. I'm not totally sure where you imagine "that module" living and how it will work into the larger system though. [16:04:27] Mebbe comment on that task with which config you'd like to see become the authoritative source [16:04:40] me neither :) but other modules exist I believe [16:05:56] gtg 4 real now [16:05:57] thanks! [16:06:23] Will do [16:06:24] i./ [16:06:26] *o/ [16:23:59] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2730864 (10Halfak) I think it is important that the primary config for precaching (throu... [16:48:35] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2730864 (10Pchelolo) >>! In T148714#2731926, @Halfak wrote: > I think it is important th... [17:25:29] 06Revision-Scoring-As-A-Service, 10Research Ideas, 10Wikimedia-Developer-Summit (2017): Where to surface AI in Wikimedia Projects - https://phabricator.wikimedia.org/T148690#2732178 (10Halfak) @leila, would you be interested in including the recommender AI in this session? I'd originally intended to include... [18:21:21] 06Revision-Scoring-As-A-Service, 10Research Ideas, 10Wikimedia-Developer-Summit (2017): Where to surface AI in Wikimedia Projects - https://phabricator.wikimedia.org/T148690#2732408 (10leila) @Halfak Can I get back to you about this by October 28? I'm thinking about it and I need to spend some cycles on it b... [18:46:31] 06Revision-Scoring-As-A-Service, 10Research Ideas, 10Wikimedia-Developer-Summit (2017): Where to surface AI in Wikimedia Projects - https://phabricator.wikimedia.org/T148690#2730132 (10Halfak) [18:54:40] So.. fun story. We can't pickle the bllipparser reranking model [18:54:57] So... building features that require it is going to be a little weird. [18:59:51] Filed https://github.com/BLLIP/bllip-parser/issues/52 [19:09:19] spacy's english parser take 27.8 MB of memory [19:09:21] When pickling [19:09:27] probably a little less in memory. [19:11:34] OK. Time to change locations. [19:11:40] I'll be back online in 45 mins [20:38:37] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732866 (10awight) Thanks for taking a look at this! Options (1) and (2) seem problemat... [20:48:09] Thinking about https://etherpad.wikimedia.org/p/ORES_docs_split for a whilke [20:48:32] awight|semi, if you're around, it would be great to try to spec out what belongs at [[:mw:ORES]] [20:48:33] 10[1] 04https://meta.wikimedia.org/wiki/:mw:ORES [20:48:39] <3 AsimovBot [20:48:42] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732886 (10Pchelolo) >>! In T148714#2732866, @awight wrote: > Thanks for taking a look a... [20:51:12] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732889 (10Halfak) We need to limit requests in some ways. E.g. there's no good reason... [20:52:54] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732890 (10Halfak) I should note as well that other models will have new precaching dyna... [20:54:58] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732892 (10Pchelolo) >>! In T148714#2732889, @Halfak wrote: > We need to limit requests... [20:59:09] halfak: hey, sure--etherpad / call? [20:59:19] oic [20:59:21] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732895 (10Halfak) Indeed. So, if we're going to have ORES figure out which requests to... [20:59:52] Na... no call. I'm already in a "Documentation Tea Time" WIth the research team. I could invite you to join, but we try to be mostly quiet anyway :S [21:00:22] So I was trying to work out what belongs in each on-wiki location and thinking about Mediawiki.org was the most difficult. [21:00:24] awight, ^ [21:00:38] I agree [21:00:55] We run into that all the time with fr-tech docs [21:01:14] It's mostly a matter of personal philosophy AFAICT [21:01:40] but my approach lately has been to aggressively push technical things onto mediawiki.org [21:01:53] In our case, that was because so much important stuff is hidden on private wikis for no good reason [21:02:20] But it makes sense here as well--IMO wikitech should only contain "how to log into WMF servers" and "why we chose a particular deployment configuration" [21:04:55] awight, "run these commands in order to deploy" [21:05:00] --> Wikitech [21:05:38] "A production ORES config has a pool of web nodes that do IO and worker nodes that do CPU" --> MediaWiki [21:05:51] "In order to get ORES support for your wiki, ..." --> Meta [21:05:59] awight, ^ does that seem right to you? [21:06:04] +3 [21:06:07] :D [21:06:15] I'm just channeling S [21:06:47] OK I'll start taking inventory and planning what content is moving shortly. [21:07:10] Yeah I'd say any *potential* configuration can be documented on mediawiki, then the *actual* config settings on wikitech, with a link to their explications on mw [21:07:37] take everything I say with salt though, I'm pretty bad at docs [21:07:40] Unrelated: I thought you might find this idea interesting -- http://socio-technologist.blogspot.com/2016/10/best-practices-for-ai-in-social-spaces.html [21:07:49] Which links to https://phabricator.wikimedia.org/T148700 [21:08:39] right, the meta-ORES! [21:08:41] thx [21:08:57] I think there's a lot to say/explore re. making "refutations" appear within query results next to predictions. It makes a false-positive report *effective* in that a tool dev can easily incorporate/account for them. [21:09:05] I was also thinking last night, your framework might be a perfect home for random decentralized human curation. [21:09:22] As it stands, false-positive reports go to some back-alley wiki page that only I see. [21:09:27] like, a new model space for collaborative score-like annotations [21:09:37] It could eventually replace the bogus "revert" paradigm [21:09:55] Interesting :) [21:10:08] I don't get much sleep lately ;) [21:10:46] I'd like to see a clear statement of the current "revert paradigm" and what you think we could change about it. [21:10:50] If you have the time. [21:11:15] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732915 (10Pchelolo) @Halfak As I remember from out conversation with @Ladsgroup on wiki... [21:11:23] I think that would be good fodder for thinking about Future of Merged Standpoints(TM). [21:12:36] halfak: yes I'd love to help merge the merging theories [21:13:39] I accept the challenge to B.S. a bit about that, later, but short story I think we need to track revisions as deltas rather than article content at a snapshot. Like in a modern VCS. & then just provide transparent tools to manage the madness. [21:15:05] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732918 (10Halfak) ORES will respond to any request a user sends. You can literally sco... [21:16:13] awight, yeah. I definitely agree with that. That would help us solve a lot of problems. Honestly, I think we should track both full versions and deltas. [21:16:47] Because deltas have context and there are many delta strategies -- some more desirable than others for different contexts. [21:18:49] +1, both deltas and full content are useful for analysis--that should be an implementation detail really. I think that e.g. git does heuristic things where it uses deltas or full content, whichever is more efficient for a revision. Something like that. [21:20:09] also on this tangent, there's probably a wide margin to play in, where we can make tools that handle "delta" revisions, without changing anything about the backend object models [21:20:29] native support is just an optimization. [21:21:42] halfak: What can I do for the doc org'ing project? Want me to help with inventory, or have other ideas? [21:24:42] awight, I think it's good to leave inventory to me. I want to ping you again once I've taken stock and made a plan to get your thoughts. Maybe we can tag team doing the updates. [21:24:45] Does that sound OK? [21:25:07] That sounds better than what I was imagining :p [21:25:21] which was major muddling on my part [21:26:19] touching base about my committment level--I have 3 more weeks of PTO, in which I'm going batshit. [21:26:28] A little side project would be great. And sustainable. [21:27:40] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732924 (10Pchelolo) >>! In T148714#2732918, @Halfak wrote: > Still, I don't mean to sug... [21:37:35] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2732931 (10awight) > listen to the stream ourselves Yeah, @Pchelolo if you don't mind,... [21:50:37] halfak: About the precache daemon you used to rely on, did it subscribe to multiple wikis' RCStreams, or is there a global stream? [21:51:10] awight, there's a global stream. I've got a comment I want to show you... [21:51:39] sorry if this isn't a problem you needed to solve ;) [21:51:47] *wasn't [21:52:01] https://github.com/wiki-ai/ores/blob/master/ores/utilities/precached.py#L150 [21:52:24] Na. You should know how it all comes together so you can help us figure things out. [21:52:30] The labs install is still using precached [21:53:16] As you can see in the code, we filter the global stream down to what we want. [21:53:56] I'm only anxious about the config DRY thing, otherwise the ChangeProp solution is pretty great. Oh and specifying models is lame. [21:56:54] +1 [21:57:00] We want flexibility per model though. [21:57:16] So it's nice to do that in the ORES config because we're specifying the models there anyway [21:58:41] what about pchelolo's point that other services might want all the same configurability [22:00:39] I think it's a good point. I don't want to build a custom solution, but if we're going to put a bunch of logic in ORES, that's essentially what we're doing. [22:01:14] awight, how come you didn't like having an API endpoint that provides config info to Change Propagation? [22:01:25] I suppose it complicates things. [22:02:00] We could *generate* Change Propagation config as part of our deploy. [22:02:10] Then it would be static and code-reviewed [22:03:13] So... I need to run. I'm sorry to cut this off, but I had a prior engagement [22:03:32] Thanks for looking into this awight. I promise to have new thoughts tomorrow. [22:03:36] And to read the scrollback :) [22:05:44] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2730864 (10GWicke) Writing your own Kafka consumer with proper multi-DC handling, retry,... [22:28:25] 06Revision-Scoring-As-A-Service, 10ChangeProp, 10EventBus, 06Services (watching), 15User-mobrovac: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2733148 (10awight) From IRC crash-scroll: > awight, how come you didn't like h...