[20:01:41] btw, for anybody attending RFC meeting about T201643 in an hour, am happy to answer any pre questions before or anytime really :) [20:01:41] T201643: RFC: Modern Event Platform: Schema Registry / Metadata Service - https://phabricator.wikimedia.org/T201643 [21:00:23] #startmeeting RFC meeting [21:00:23] Meeting started Wed Aug 29 21:00:23 2018 UTC and is due to finish in 60 minutes. The chair is kchapman. Information about MeetBot at http://wiki.debian.org/MeetBot. [21:00:23] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [21:00:23] The meeting name has been set to 'rfc_meeting' [21:00:30] o/ [21:00:31] #topic RFC: Modern Event Platform: Schema Registry / Metadata Service https://phabricator.wikimedia.org/T201643 [21:00:50] ottomata glad to see you here, I messed up a little and forgot make sure you were available [21:01:09] ha [21:01:16] #link https://phabricator.wikimedia.org/T201643 [21:01:17] been waiting for this meeting for a couple of weeks! [21:01:20] i'd better be! [21:01:33] actually i was going to go to a yoga class then remember this... :D [21:01:39] remembered* [21:02:10] ottomata btw we were wondering if the other RFC you are waiting on is blocking overall progress: RFC: Modern Event Platform: Scalable Event Intake Service https://phabricator.wikimedia.org/T201963 [21:02:29] they both are a bit, it'd be nice to get them through the pipeline [21:02:38] but, good news is that for this quarter we only commiitted to planning [21:02:40] not implementing [21:02:45] okay we'll take that into account [21:02:46] so we aren't behind in any way [21:03:08] is anyone else here for the RFC discussion? [21:03:50] hoping for some _joe_ and Pchelolo for sure, and mobrovac [21:04:01] I'm lurking for a bit [21:04:08] hiya [21:04:15] me too, just lurking [21:04:25] _joe_ and mobrovac didn't attend the techcom meeting, so they may be out for some reason or other [21:04:30] oh... [21:04:32] :/ [21:04:36] I generally type to slow for these, but I'll be following [21:04:48] and Marko has started his vacation already [21:04:51] oh... [21:04:59] well marko is mostly synced up about this [21:05:01] i guess that's ok [21:05:04] Pchelolo: looks like today, no fast typing is requried :) [21:05:16] soooo, anybody got questions? [21:05:47] ottomata: are there things you need guidance on? or are you looking for approval of the proposal as-is? [21:05:53] I wonder if we go with the git scheme, how hard it would be to make GUI on labs (or wiki?) to have some kind of nice schema display? [21:06:40] DanielK_WMDE: i think i'm seeking approval to ditch the on wiki schemas in favor of git + a read only GUI, and also possibly a metadata/config service maybe also readonly with git [21:06:42] I am fine with editing JSON schemas in my editor (that's what I do anyway) but displaying them using nice UI is helpful [21:06:47] SMalyshev: show-stuff-from-git-on-wiki is a long standing request for documentation. kind of fits in. let me find the ticket [21:06:51] SMalyshev: yeah, we need a read only GUI for sure [21:06:56] product managers etc. need that [21:07:03] they want to browse and search existing schemas and configs [21:07:13] oh interesting [21:07:30] I think if we have that, we keep most of the Pro's for Schema: [21:07:49] ottomata: so, if you don't need guidance, just approval, nobody attending is the best that can happen to you ;) if nobody complains, the rfc can go on last call. [21:08:00] hahah :) [21:08:06] sorry, i just hopped on and wasn't logging the channel. anything in the backscroll? [21:08:12] not really dr0ptp4kt [21:08:15] ha [21:08:28] i just said: ": i think i'm seeking approval to ditch the on wiki schemas in favor of git + a read only GUI, and also possibly a metadata/config service maybe also readonly with git" [21:08:30] dr0ptp4kt: logs at https://wm-bot.wmflabs.org/logs/%23wikimedia-office/ [21:08:41] I just started using schemas, but the only advantage of Schema: vs. git for me was ease of reading and having a good listing of existing schemas [21:09:37] DanielK_WMDE: oh ha, for some reason i didn't think that was tail'd. good to know [21:09:50] if we keep that and have stuff like what is on Schema talk:, then having it in git sounds just fine [21:09:54] in addition to gui, we will likely have an http service api that serves schemas/configs from the files in the git repo [21:10:11] the gui could use the service...or not! it could just use the git files [21:10:12] I'm curious why this decision is considered to be in the realm of TechCom. It seems that it will have major impact not just on engineers/developers, but also on users of this data [21:11:01] I posted some comments here: https://phabricator.wikimedia.org/T201063#4543616 [21:11:01] HaeB I think it is in the realm of both? but, since we are unifying the systems these things are relevant to production usages [21:11:06] SMalyshev, ottomata: https://phabricator.wikimedia.org/T91626 [21:11:16] editing the schemas on wiki is painful when it comes to multiline descriptions, if that gets easier I'm happy :) [21:11:51] so far with eventbus schemas, we've been storing them in yaml...so you can actually write inline comments too! [21:12:10] HaeB: one of the requirements from product engineers is to be able to more easily develop schemas without affecting production [21:12:11] ...TLDR: it seems that the task and discussion has been conducted almost exclusively from an engineering perspective. As a data analyst, I don't see my perspective and needs represented [21:12:23] HaeB: managinng event schemas is a cross cutting engineering concern, which puts it into techcom's scope. whether techcom's approval is *sufficient* is a different question entirely. [21:12:32] I personally would not mind losing the ability to directly edit it from a UI. In fact, I'd actually prefer it is in Git and offers code-review/acceptance. [21:12:41] DanielK_WMDE: looks cool, though I'd be happy with just a tools.labs page as well :) [21:12:44] we can't do that easily with a single centralized schema registry [21:12:45] But a viewer addressable by public URL is indeed a must in my opinion. [21:13:04] Krinkle: aye [21:13:04] I'm not the only one, product managers still would like to weigh in too https://phabricator.wikimedia.org/T201063#4543519 [21:13:25] HaeB: what are your needs that aren't addressed? [21:13:41] #info But a viewer addressable by public URL is indeed a must in my opinion. [21:13:45] we tried to do a pretty good job collecting requirements from everybody, including many folks on the product side, and all analysts [21:13:45] the thing I don't like in our current event-schemas repo is outrageous abuse of copy-paste. We have to be able to reference sub-schemas [21:14:18] Pchelolo: ya we might be able to get that. not sure if it is a good idea or not though. it would mean that the schema isn't readable just from the file [21:14:21] you'd have to get it from a service [21:14:35] which is a complaint I have about eventlogging right now (the on wiki schemas are not the complete schema) [21:14:40] HaeB: is your concern that git is hard to use for non-engineers? [21:14:51] Krinkle: it's not surprising if developers (who work in git / gerrit all the time and have accumulated their tools and expertise around it) find a GUI unecessary, see also https://phabricator.wikimedia.org/T201063#4543519 [21:15:16] HaeB: the current proposal still has a GUI, just not an editable one [21:15:16] the only problem with yaml is that the PHP libraries for it sucked, last time I checked [21:15:16] ottomata: Is there a desire to keep schema revision IDs? As in, does this RFC require in its scope a way for the MediaWiki extension to know the current schema version and does it conitnue to have to embed it in its beacons? [21:15:26] ottomata: you can add some kind of a build step to it, but with copy-paste approach now even though we're very cautious and have tests, schemas still diverge sometimes [21:15:34] I asked both JMinor and Adam baso about the read-only gui [21:15:34] for example they couldn't round-trip the string "yes", it would turn into boolean [21:15:37] also if you inherit schema parts, the parent parts would be very hard to change, as it implies all user schemas should agree on the change or be cloned [21:15:38] ottomata: (data analyst needs) that would need some more time to write up, like Josh plans to do from the PM perspective [21:15:40] and they both said it sounded fine to them [21:15:53] regarding the gui: [21:15:54] (reminder to use #info if you want things in the public minutes.) [21:15:55] the meeting mentioned in https://phabricator.wikimedia.org/T201063#4515886 [21:16:00] my concern is about process here - deciding this kind of thing in a techcom rfc [21:16:03] included two product folks [21:16:23] the yaml spec is extremely complicated, it's worse than XML [21:16:41] i only happened to see this earlier today and am taking some time out of my current work to participate here and post those comments on phab [21:16:41] it's more like a serialization format than a document format [21:16:52] TimStarling: aye, we mostly use yaml as a shorthand for json [21:17:02] the service api would respond with json [21:17:03] HaeB: I understand, but I did not mean in it in that way. My understanding is that the development and testing will need to happen in git regardless in order to be able to control deployment, and to allow development of a feature to happen without having to have the version in production. Right now, I have to "deploy" messy/draft versions on meta.wikimedia.org in order to develop a feature locally. This also means it does not work [21:17:03] offline/intermittent connection, which is counter-productive. [21:17:27] and our schemas will be restricted jsonschema anyway, so it is unlikely we'd be using fancy yaml features [21:17:49] HaeB: However, I do genuinely want to know - is it common for you or other analysts to edit schemas on meta-wiki? [21:17:57] HaeB: I'd be concerned if it was decided *without* a techcom rfc. But I see your point - there is (to my knowledge) no formalized process that ensures input from non-engineer stakeholders. [21:18:01] HaeB: you were included in the interview and comments process that happened last quarter. Many other produce folks have been involved in this so far [21:18:05] and, the process isn't closed! [21:18:13] so, we are hear listening :) [21:18:16] here* [21:18:18] Krinkle: sure, e.g. https://meta.wikimedia.org/w/index.php?title=Schema:PageIssues&action=history [21:18:21] HaeB: ottomata said he contacted analysts. was that not sufficient? what should he have done instead? [21:18:45] HaeB: For aspects that are not documentation I mean. [21:18:52] see also https://phabricator.wikimedia.org/T185233#4326260 [21:18:58] Because such change would require a change to the code to avoid breaking production. [21:19:24] and also [21:19:24] https://docs.google.com/document/d/1PllNUpMFs7n21fPaLydlrrzkEYsdOBbpWAOSY6OfVdc/edit [21:19:34] Right now, if we were to edit that schema page and add a mandatory property or change a properties type, the feature will break its events fatally untl the code is updated. Which means it's open with a conflict of authority. [21:19:36] yaml is convenient for human writing, unlike json that doesn't allow comments or litera newlines in strings. the phpyaml php extension is quite solid because it uses libyaml [21:19:54] ottomata: yes, but 1.) i don't recall us talking about this aspect specifically 2.) I made suggestions and raised concerns that don't seem to have made it into the RfCs/proposal (e.g. about the possibility of implementing more sophisiticated data validity checks server-side, if you recall) [21:19:57] Krinkle: that's not totally true, since the events are validated by schema and revision [21:19:59] not just latest schem [21:20:00] a [21:20:13] but, it would break for many systems, including ones for which we want to have all events of the same schema in the same queryable endpoint/table [21:20:15] like hive [21:20:24] ottomata: (this aspect = schema registry, the current meta-wiki system) [21:20:34] the change needs to be aligned with a code change. But a way to submit drafts or proposals, seems important indeed. But perhaps that could work jut as well via a comment on Phabricator with the proposed schema? Then, in the code commit that normally changes the feature and updates the revision id, they would now instead of changing the ID insert the schema itself. [21:21:21] HaeB: that is true, but as I understand it, most EL consumers do not actually validate by ID. And that is part of the problem. You can create false schema revisions and push beacons to EvenetLogging right now that will infect a "valid" topic in Kafka. [21:21:27] so I wonder how git version would address different versions of the schema. Right now Wiki page traks revisions for each page separately [21:21:28] Krinkle: currently simle gerrit code-review for event-bus schemas work really well [21:21:33] E.g. vandalism control. [21:21:35] but git tracks all revisions together [21:21:50] SMalyshev: we'd likely not actually use git versioning to version the schema revisoins [21:22:06] Krinkle: this was more about implementing alarms for implausible value combinations (if field A has value X, then field B has to be > 200, or such) [21:22:08] we might, but i think that hides the history, especially if we want to build a browseable gui and api service [21:22:19] instead, we'd just use new files like we do for eventbus now [21:22:20] ottomata: ok, but how would it work instead? Would we manually version them? [21:22:23] keep-it-simple :) [21:22:24] yes [21:22:27] I think we do need a way to differentiate drafts from published/accepted versions. That is currently missing, and would require EventLogging consumers to manually re-validate the incoming event stream by known revision IDs. [21:22:36] manually or with some helper scripting? but yes [21:22:36] ottomata: so, new version -> new file? [21:22:38] yes [21:23:01] Krinkle: ...offtopic here, it was just an example that not all input from the may / june discussions may have made it into the event platform proposals on phab [21:23:11] Okay [21:23:20] SMalyshev: that's how it works now, but that's another source of copy-paste. And diffing is problematic [21:23:25] HaeB: that's an interesting use case, we might be able to do that, but i think that's not part of schema registry [21:23:29] s/problematic/manual [21:23:31] #info SMalyshev: we'd likely not actually use git versioning to version the schema revisoins [21:23:42] #info instead, we'd just use new files like we do for eventbus now [21:24:03] Right. that allows the versions to be controlled. So you can make a change that doesn't increase the version number. [21:24:04] ottomata: yes, like i said, it was just a comment about the process ("ottomata: HaeB: you were included in the interview and comments process that happened last quarter. " [21:24:39] HaeB: i'm sorry if i don't remember that idea, let's add it into the scalable event service user stories [21:24:48] Krinkle: that may not be hard to do with files, by just making it something like 2.yaml.draft and having the process treat .draft ones differently? [21:24:56] it might not be part of event validation though. that sounds a little more like alerting on data (perhaps via stream processing) [21:25:08] SMalyshev: If we use git, drafts are redundant, they are just unmerged commits. [21:25:25] Krinkle: sure, that too :) [21:25:37] or a version+1 that isn't used yet [21:26:06] yeah, i'd expect drafts to be just like any other unapproved code stuff we do [21:26:09] either unmerged in gerrit [21:26:15] or in a branch/fork/pull request whatever [21:26:27] ottomata: thanks - and again, i'm just arguing we need to solicit input from data users about the specific topic here (schema registry/ discontinuing the existing Meta-wiki system), not just about event platform in general as done last quarter (which was appreciated!) [21:26:31] sounds good [21:26:38] ottomata: the example of controlling sampling rate (e.g. configuration for producers as opposed to data/schema format), is not entirely clear to me. Could you elaborate on what that would look like? [21:27:24] HaeB: I admittedly don't have many notes from our interview, but I think I tried to mention that we were considering no longer using wiki for schemas [21:27:32] apologies if i didn' tmention that to you then [21:28:04] Krinkle: it isn't 100% clear to me either how that would be implemented [21:28:05] but [21:28:10] let's assume it is a config somewhere [21:28:16] for now let's assume also in git [21:28:38] to clarify my point of view from an engineering perspective - a git repo (or repos) are great for the dev experience. regarding gui, though, what are the functionalities needed? i can see how a JS-based Special: page app or a web-based git editor should be able to do most things (although not all things - you need APIs or code that knows git) [21:28:39] thered' be a configuration that maps a schema -> topic-usage, and any other configs associated with that topic-usage, like sampling rate [21:28:53] that sampling rate configuration would be queryable from a public API endpoint [21:29:02] ottomata: it seems to me that the (non-git) versioning mechanism should be part of the rfc [21:29:05] and client side code would use it to configure their own sampling settings [21:29:32] dr0ptp4kt: I don't even think we need *editor* - just displaying the JSON/YAML like Schema: (+talk) namespace does would go a long way. [21:29:43] DanielK_WMDE: ok cool, I will add it [21:30:20] #action ottomata: it seems to me that the (non-git) versioning mechanism should be part of the rfc [21:30:23] ottomata: Right, but I see a potential paradox or inconsistency there. We currently don't store sampling rates centrally. This has the benefit of allowing clients to know their sampling rate without external dependencies or services, and to know it without loading the schema and related code yet. It means that to change the rate, the process is a configuration change and deployment. [21:30:24] SMalyshev: dr0ptp4kt, yeah something like that, hopefully with some better features. e.g. structured/queryable settings for ownership and configs [21:30:34] ottomata: yup! [21:30:59] Krinkle: yeah, this is a big complaint from product folks [21:31:02] ottomata: If it is in the schema, the point of authority becomes unclear to me. Presumably we cannot guruantee that the producer (e.g. mediawiki or an app) will be using the same version of the registry as the processor that validates it (EventLogging). [21:31:18] Also presumably we do not need to deploy a change ot the registry to EventLogging if it only change the sampling rate. [21:31:41] Krinkle: it isn't in the 'schema'. it is in the schema usage configuration [21:31:55] #info a way to submit drafts or proposals, seems important indeed. But perhaps that could work jut as well via a comment on Phabricator with the proposed schema [21:31:58] Sure, but I mean, it will be in data that the app needs a copy of in some way. [21:32:03] but yes, it would mean that the client would need to phone home to get the sampling configs [21:32:33] this part wasn't in my original modern event platform plan, but surfaced over and over again during interviews with product folks [21:32:51] they want to be able to modify things like sampling rate on the fly without having to wait for a swat/train deploy [21:33:11] We have recently been moving in the direction of apps and js clients not needing to phone home anymore. Perf is working milimetric to remove part of EventLogging JS that ships the schema, to serialise the data directly and validate as we do now. [21:33:27] and also they want to know what samping rate a client used to emit the event. I imagine to solve that we'd have the client include the sampling rate-config in the event itself [21:33:32] I fully support btw, centralising that configuration. [21:33:47] Krinkle: i think we might need both [21:34:05] a client-lite that doesn't phone home or have configurable sampling [21:34:20] and a full feature client that is dynamically configurable [21:34:25] but I'd want to make sure it's the kind of thing that is baked in, not fetched on-demand. I want to be able to download an app, and then turn off connectivity and open it, and have it perform sampling before knowing what to do, and serialise data into a queue for later. [21:34:32] I'm pretty sure mobile app folks have to write phone-home apps anyway for this stuff [21:34:36] since they can't just deploy new code to clients [21:34:51] Krinkle: +1 that makes sense [21:35:16] well, even if they fetch the schema from meta wiki in apps (which they dont' afaik), the code wouldn't know how to produce a different schema. It requires deploying new code either way. [21:35:23] but yeah, config change without update would be nice. [21:35:25] cached in some way. [21:35:30] that would mean that the current sampling configs might not be in sync with offline clients, but i think that's mostly ok. especially if the sampling config the client used is shipped back to us in the event [21:35:51] for schema for sure. [21:36:00] the client side validation done for schemas now is a little redundant [21:36:05] * DanielK_WMDE notes that this discussion isn't about schema management any more [21:36:06] i'd imagine is mostly useful when develeoping [21:36:07] and for mediawiki, it might need a script to sync data. E.g. in WikimediaEvents like we do now to bump revision ids. [21:36:09] how does the config change for sampling depend on implementing the new registry/abandoning the existing system? [21:36:10] We could automate that. [21:36:24] DanielK_WMDE: yeah, this schema registry is more than just schemas...it became that over the course of the interviews last quarter [21:36:31] there are 2 parts, which may or may not be in the same service/system [21:36:42] 1. schema repository [21:36:42] 2. schema usage configuration [21:36:42] ottomata: so it's schemas + config? [21:36:43] yeah it looks like we're getting into dynamic sampling configuration... is it planned to be in scope? [21:37:02] SMalyshev: when I proposed MEP as an annual plan, no. but for this RFC, yes. [21:37:24] i think maybe that isn't clear enough in the RFC ticket, i'll try to fix that [21:37:25] ok [21:37:32] its a little more clear in the parent task i think [21:37:48] ottomata: that should be explicit in the rfc, please [21:37:49] yeah it kinda mentions that but very in passing, easy to miss unless you look for it [21:38:34] #info DanielK_WMDE: yeah, this schema registry is more than just schemas...it became that over the course of the interviews last quarter [21:38:35] k will add it [21:38:42] #info there are 2 parts, which may or may not be in the same service/system [21:38:49] #info ottomata> 1. schema repository [21:38:56] #info 2. schema usage configuration [21:39:23] I think it would be fine to consider configuration outside the scope. It's not something we can do today and shouldn't imho be required to adopt the new registry. But, it is something to think about, so that we make sure we move in a direction where we can easily do that /next/. [21:39:31] #action Clarify RFC to make it clear there are two parts: schema repo and schema usage configuration [21:39:43] HaeB: to answer your question from above: it doesn't necessarily do so on its own. but MEP is about taking a holistic look at what we have, comparing it to what we want, and then making it so. [21:39:53] but we don't need to know exactly how we'll do it to move forward, I think. [21:40:27] Krinkle: so just to be clear: we do already change sampling rates via config changes, right? e.g. https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/373920/4/wmf-config/InitialiseSettings.php [21:40:51] Krinkle: yeah, it might be a totally different service from schema reg. the production configs might even in a centralized-not git service. dunno [21:41:09] HaeB: yes, but that requires a mw-deployment, which product managers wanted to avoid [21:41:21] well, ok, not mw-deployment, but i guess, train/swat? [21:41:34] whatever the process is, product managers want it to be more dynamic [21:41:43] HaeB: yes, that is correct. But every producer (every mw extension, mobile apps, other services) have their own way to configure it, so it is not centralised for you, and even for mw ones (which are all in wmf-config) they don't follow the same naming or format (e.g. 0-1 factors or 1-in-N pop sizes). [21:41:44] ottomata: yes, i thought this just needs SWAT now [21:42:03] that is correct. [21:42:15] to me that sounds like deployments should become easier to do [21:42:47] Nikerabbit: there's also the mobile apps use case [21:42:56] its not just mw javascript clients [21:43:31] if we move that config to the schema registry, it'll require a commit to to-be-named/schemas.git instead of wmf-config.git, and it'll require deploying that change to have the UI viewer show it, and then (depending on the consumer) it may also require something like a submodule update+deploy for the WikimediaEvents extension to see the same version of the registry. [21:43:57] Krinkle: so it would be partly about collecting these sampling rates all in wmf-config (like the above example already is) and partly about enforcing usage of the same format there? [21:44:14] given that MW would no longer fetch from a production hostname, given that we want it to be configurable locally, right? [21:44:19] HaeB: btw, 'sampling rate' here is a simplified version of what has been asked for [21:44:43] they also want consistent sampling settings, e.g. hash on some key (user_id?) and sample on that [21:44:48] not just random [21:45:04] HaeB: Yes, based on what I hear analysts/PMs/researchers would prefer, that would be the case. [21:45:07] ottomata: yes, like i said at https://phabricator.wikimedia.org/T201063#4543487 it should record the sampling method too [21:45:13] yup [21:45:15] coo [21:45:34] HaeB: i'm not sure i understand the last bit about submodule udpate and deploy [21:45:50] I think the method will be more documentative than configurative, given not every consumer is going to deploy code for every possible method supported somewhere in the system elsewhere. [21:45:55] 15 minutes remaining. ottomata do you have other feedback you desire in the remaining time? [21:46:00] i'm imagining a standalone service [21:46:18] ottomata: bit about submodule update where? [21:46:20] that would need new configs changed in (either via git or via a gui...this one might be editable in a datastore, dunno). [21:46:38] ottomata: a standalone service for config ditrsibution? or also schemas? or two services?... [21:46:42] then any client side code that use the system and want to opt into dynamically configured configs would know to reach out to the service [21:46:57] DanielK_WMDE: unclear as of yet if they are different services. [21:47:04] but, we could think of them as such for now [21:48:06] kchapman: i think for the RFC i'm looking for approval to proceed with a non-mw approach for the schema registry/service. Perhaps the schema-usage-config service is more complicated and might need an RFC of its own? [21:48:31] ottomata: the rfc says "no active schema API service". is that what you were just referring to? creating such a service? [21:48:41] yes [21:48:53] for jsut schemas [21:48:58] this coudl be very simple [21:49:04] ...then that should be part of the rfc [21:49:06] even just a http file server [21:49:11] oh [21:49:12] ottomata: "proceed with a non-mw approach" - again, I think this should have more input from data users (analysts and PMs) [21:49:12] sorry [21:49:18] active schema API service refers to schemas [21:49:39] schema-usage-configuration service is (possibly) different [21:50:26] ottomata: so for beta and local development, ideally installing EL + WikimediaEvents puts it in a state where it can start producing events. but sounds like one might need to also clone registry-viewer-and-read-api.git and install/start that service. [21:50:58] HaeB: ok, how do you propose I do more than I already have? centralization on wiki makes many of the other use cases difficult if not impossible [21:51:02] ottomata: if the schema-usage-config is going to be complicated and isn't clearly outlined yet it might be best in its own RFC [21:51:34] Krinkle: ah. yes you would have to clone the schema git repo [21:51:44] but, that sounds better than it is now [21:51:59] HaeB: I think the ability to easily view and/or propose changes in the web is a use case that can be discussed, but I'm not sure an implementation detail (mw or non-mw) should itself be a requirement. [21:52:00] with EL + WikimediaEvents now, if you want to develop a new event/schema [21:52:06] you need to create a new schema in the production repo [21:52:07] BTW, we would also lose the schema talk pages as central place for informal discussion and documentation about a schema (e.g. https://meta.wikimedia.org/wiki/Schema_talk:Edit https://meta.wikimedia.org/wiki/Schema_talk:Popups https://meta.wikimedia.org/wiki/Schema_talk:MobileWikiAppDailyStats ) ... [21:52:10] even if you end up not using it [21:52:30] kchapman: DanielK_WMDE I'd be fine with making a new RFC for schema-usage-configuration service. [21:52:39] and keeping this one schema-only service [21:52:41] .... even after we port over all the structured information that's currently in https://meta.wikimedia.org/wiki/Template:SchemaDoc [21:52:46] they may end up being the same thing? but it isn't not yet clear [21:53:06] #info [14:52:30] kchapman: DanielK_WMDE I'd be fine with making a new RFC for schema-usage-configuration service. [21:53:44] Krinkle: again, I think using a TechCom RfC to decide this kind of product question is a deeply flawed way of doing things [21:53:45] I don't think any of the teams involved with these schemas watch those talk pages. Discussion might be better had elsewhere, like on the talk page of the feature or team it relates to – which any viewer can point to. [21:54:10] HaeB: i dont' think that would be lost? that is just on wiki documentation [21:54:30] migiht not be a 'talk page' but i'd hope folks would still use wikis for documentation [21:54:57] Krinkle: "I don't think any of the teams involved with these schemas watch those talk pages." - i just showed you several example of teams actively using those talk pages [21:55:24] HaeB: TechCom helps bring stakeholders together to agree on acceptance criteria in the early phases of an RFC. In later phases others are involved to agree on how to implement those requirements. [21:55:42] btw perhaps T91626 would actually be a good GUI solution here? [21:55:42] T91626: Technology to transclude git content into wiki pages - https://phabricator.wikimedia.org/T91626 [21:55:48] Krinkle: that too is an example of not how to make product decisions ("I don't think users do that, so let's remove it") [21:55:54] i'm pretty agnostic about the GUI [21:56:00] HaeB: OK. But it's not like those can't happen anywhere else, and I do still think it is contributing to fragmentation and splitting of information over too many places, even if it happens to work for some cases. [21:56:02] i just want an offline decentralized data store [21:56:08] 5 minutes left, please wrap up and add anything to the minutes with #info that we might have missed [21:56:59] alright, let me rephrase the approval i'm seeking then: [21:57:10] i propose to use git to manage event schemas for both analytics and production [21:57:17] HaeB: i get why you don't think a techcom rfc is sufficient, but I don't get why you think it's problematic. "approved by techcom" doesn't meed "does not need approval from anyone else". [21:57:28] there will be a service API to query, and a GUI to view those schemas [21:57:31] HaeB: Also, the talk pages could easily be kept if they are vital/irreplacable. Constructing a url to "meta.wikimedia.org/wiki/Talk:Schemas:{name}? from the registry viewer is triiial. [21:57:36] trivial* [21:57:44] #info btw perhaps T91626 would actually be a good GUI solution here? [21:57:47] what the GUI is, i don't care that much. If it can work with mediawiki, and it solves the use cases, that is fine with me! [21:58:36] I'll add that approval seeking phrasing into the RFC [21:58:48] i'll reword it so it is about using git for schema storage [21:59:05] getting approval on that that would unblock the work [21:59:07] * DanielK_WMDE wants a git-backed mediawiki [21:59:09] HaeB: Thanks for indicating that discussion should be a use case we need to consider. [21:59:26] and allow us to look into possibly using mw for gui [21:59:37] one minute left :) [21:59:41] :) [21:59:42] #info The viewer should provide an easy way to contact/discuss owners and peers interested in the schemas. [22:00:00] so yeah, what should I do next? aside from these RFC modifications in order to get approval to ^^^^ [22:00:03] ? [22:00:25] once you make the updates TechCom can discuss again at our next meeting [22:00:33] ok [22:00:34] and then decide if it can be put on Last Call or not [22:01:03] alright then. [22:01:04] :) [22:01:07] thanks everyone for attending! [22:01:12] #endmeeting [22:01:12] Meeting ended Wed Aug 29 22:01:12 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [22:01:12] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-08-29-21.00.html [22:01:12] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-08-29-21.00.txt [22:01:12] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-08-29-21.00.wiki [22:01:12] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-08-29-21.00.log.html [22:02:01] thanks all! [22:02:13] i'll work on those RFC mods tomorrow