[15:32:55] Technical Advice IRC meeting starting in 30 minutes in channel #wikimedia-tech, hosts: @addshore & @CFisch_WMDE - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [22:01:50] #startmeeting TechCom [22:01:50] Meeting started Wed Jan 17 22:01:50 2018 UTC and is due to finish in 60 minutes. The chair is DanielK_WMDE. Information about MeetBot at http://wiki.debian.org/MeetBot. [22:01:50] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [22:01:50] The meeting name has been set to 'techcom' [22:01:50] Meeting started Wed Jan 17 22:01:50 2018 UTC and is due to finish in 60 minutes. The chair is DanielK_WMDE. Information about MeetBot at http://wiki.debian.org/MeetBot. [22:01:50] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [22:01:50] The meeting name has been set to 'techcom' [22:02:00] oh, great, two meetbots [22:02:04] can someone kill one of them? [22:02:42] #topic Evolving the MediaWiki Architecture (Developer Summit Session) [22:02:53] #link https://phabricator.wikimedia.org/T183313 [22:04:02] So, anyone here to talk about the summit session on evolving the MediaWiki Architecture? [22:04:42] It's a very broad topic, so I'd like to talk about what we should focus on, and what we shouldn't mmiss [22:04:42] Well I have a vaguely related question/idea [22:04:47] Here for that [22:05:12] hey coreyfloyd! [22:05:22] i'm here (also reading over the prior discussion on-ticket in another tab...) [22:05:43] I suppose a lot of people are in some planning meeting or other right now... bad timing, i guess [22:05:47] Dysklyver: what's your idea? [22:05:57] Hey, all… I added some things earlier to the ticket, and we had a session today on similar subject… [22:06:07] And yeah many people have attention divided [22:07:31] One question that has been bothering me is how the "architecture" session relates to the "third party" session. [22:07:33] #link https://phabricator.wikimedia.org/T183312 [22:07:38] Over the last few years object storage systems such as Amazons S3 service have become very popular, also many people like static storage systems like git (github), currently there appears to be no way to export mediwiki databases to these services, despite the fact that doing so could allow easy backup and/or static html sites to use data stored in this way. [22:08:14] o/ [22:08:19] tech comm? [22:08:19] hey Niharika! [22:08:25] apergos: yes [22:08:31] great, thanks [22:08:55] Would it be technically feasible to do this, that is export every page as plaintext into a static storage system via some interface? [22:09:23] I have a question. The task says: "The session will NOT be used to discuss any of the topics in-depth. Concrete engineering questions are not in scope.". So what are the expected outcomes of the session and who's possibly going to own them, if at all? [22:09:32] Dysklyver: for scmall wikis, sure. for large ones... maybe? But the technical details are off topic for this meeting. [22:09:56] Dysklyver: do you think this capability is something that is important to the overal technical strategy of mediawiki for the next few years? [22:10:14] On architecture as it relates to 3rd party: I think that those 2 are very intertwined. The implication is that 3rd parties can only be supported by shipping a installable PHP app in a LAMP stack. This limits our architecture decisions [22:10:27] Niharika: I think a hope is we'll hash out the beginnings of a few Big Things and actually get buy-in for resourcing to work on them after the summit [22:10:49] Niharika: the expected outcome is a lits of focus areas. "with respect to our platform architecture, these are the most important things we will have to look into" [22:10:56] DanielK_WMDE: the proposed session structure seems very bottom-up (identify current / short-term problems, try to compose them into a larger strategy) [22:11:11] its an idea that I would dearly like to see as a supported feature, many small wikis use mediawiki software and when they close due to lack of funds/interest, that data is unusable, obviously this is not a concern for wikipedia as such though. [22:11:34] I wonder if having top-down discussions can fit into the structure, or best not to expect those to happen [22:11:42] +1 to coreyfloyd; the future of a lot of the node.js work that we're doing over in Reading Infrastructure now seems very uncertain at the moment [22:11:43] Niharika: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Purpose_and_Results [22:11:49] #link https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Purpose_and_Results [22:11:59] however these static storage systems are very cheap, and can be afforded even with no/few pageviews, as a static site [22:12:06] tgr: there is a bit of an emergent accessing of our stack through bubbling up current problems. [22:12:11] <_joe_> DanielK_WMDE: it's not clear to me how we link the discussion about the architecture relates to the strategy process - that would seem to make sense as a top-down approach [22:12:14] (top-down as in start from a vision of where we want to be in 10+ years, work backwards from there) [22:12:20] o/ [22:12:52] tgr: by top-down, you mean "driven by features we want to build"? [22:12:54] <_joe_> start from the strategy direction, which is not very deeply defined though, can be challenging [22:13:00] brion: DanielK_WMDE: Okay. So is there a possibility that the session outcomes might affect the annual plan? [22:13:11] DanielK_WMDE: I mean driven by long-term strategy, not specific features [22:13:34] and that has to do with the idea that we may not want to impose installing node.js on third parties, among other things [22:13:39] marvin is not even aiming to support wiktionary let alone 3rd parties [22:13:44] Niharika: i don't know how the schedules line up ;) but theoretically yes. worst case it affends next year's plan :P [22:13:45] (sorry, package got delivered mid-comment...) [22:13:47] *affects [22:13:59] coreyfloyd, mdholloway: yes, i agree that "how long do we need to support LAMP" is an essential question. and one that cannot be asnered by engineers. it's a product decision. [22:14:08] e.g. dbarratt's proposal of switching to a model where most code is maintained by third parties is top down [22:14:33] the problem I have is that the 15 year strategy seems to be to be so high level that I don't know how we can even start talking about mw architecture with that as the basis [22:14:42] Niharika: the anual plan to some degree, but more the long term (3 to 5 year) strategy [22:14:45] <_joe_> apergos: my doubt exactly [22:14:48] brion: Nah - worst case is the etherpads get lost in an abyss and we start from scratch next year. Hopefully that won't happen. [22:14:54] lol [22:14:54] I wrote a proposal https://www.mediawiki.org/wiki/User:Tgr_(WMF)/Turn_MediaWiki_core_into_an_embeddable_library which is also top-down and wondering if the devsummit is the right place to try to discuss it [22:14:59] where I do have a lot of thought about how the mw arch might evolve over the next 5 (say) years, so does everyone else I bet [22:15:02] DanielK_WMDE: Noted, thanks. [22:15:23] Yes, we have a high level strategy that has not been turned into a product or technical strategy yet [22:15:30] apergos: ^ [22:15:55] Dysklyver: the merrits of specific storage systems for mw would have to be discussed in details elewhere. git, for instance, doesn't allow information to be hidden - which is a problem when private infor is leaked via a wiki. [22:15:55] <_joe_> so with no product strategy, nt even a high level one, how do we link to it at the tech level? [22:16:04] * brion would eventually love to raise things like "storage integrity" and "security in depth". we still have a "monolithic kernel" design in terms of database access, and it worries me. but is this too big-scope? [22:16:10] Dysklyver: you may be interested in levitation (old, but still cool) https://github.com/scy/levitation [22:16:38] I wonder if we should just forge ahead with our own set of questions [22:16:39] DanielK_WMDE, this is partly inspired by the work recently done on the wiki.js project [22:16:44] _joe_: is this our chance to help drive that product strategy? /me hmmms [22:16:49] "timeless vs marvin" is the main question of frontend architecture for me at this point [22:16:52] and then retcon them into the strategy! [22:16:53] _joe_ good question - I have given this feedback to Toby/Victoria to let them know it is a blocker for long term technical planning [22:17:04] _joe_: I think the strategy process should be informed from the technical side. In particular, needs for scaling and sustaining should be communicated, as well as constraints on feature development [22:17:17] *nod* [22:17:21] that being a wiki where the text of the pages is backed up on Git, and the server code is node.js, hence the name, wiki.js [22:17:31] What's marvin? [22:17:54] Dysklyver: that sounds very interesting -- I'll put it on my backlog of things to check out :) [22:17:55] I lieu of a “product strategy” I have asked that the PMs provide us with use cases (for current and near term features) [22:17:59] <_joe_> to be more specific - I think some of the strategy directions can affect our architecture - if we want to store efficiently things other than text, we are in for some necessary evolutions on multiple levels [22:18:18] marvin is a frontend written in typescript, a RESTBase consumer [22:18:32] tgr: the dev summit would be a good place to pitch this as a possible strategy, and see if other find it worth exploring. there will unfortunately not be enough time to explore the idea then and there. [22:18:34] imagined as convergence between mobile apps and web [22:18:56] brion, https://wiki.js.org/ is the link to their blurb, I installed it and it works well enough [22:18:58] DanielK_WMDE: Could we float an etherpad in advance of the session where people can put down their thoughts more freely (we got a few comments on the ticket for now) and we can dive into the popular/recurring topics at the actual session? There's gonna be WAY too many things to discuss and the session isn't terribly long. [22:19:07] thanks! [22:19:24] DanielK_WMDE: do we want to pass out the ATWG doc with all the issues? [22:19:36] Might help as a starting point [22:19:55] apergos: yes, we should forge ahead with our own set of questions. that is exactly the idea. we won't be able to provide solutions. but we can and should identify the important questions! [22:19:57] <_joe_> So I think some big questions about our architecture are: do we want a REST API? Should it be part of MediaWiki itself or stay external? How do we move away from the cache-but-also-storage-but-also-router-and-almost-ESB that restbase currently is [22:20:07] https://www.mediawiki.org/wiki/Marvin [22:20:08] coreyfloyd: sure, go ahead! [22:20:19] > convergence between mobile apps and web - I take it we're not talking about mw here, nevermind. [22:20:45] Sheet with collection of tech stack issues, questions, and goals: https://docs.google.com/spreadsheets/d/1DDtO5v5GAtNSRiKHyii8Kv-kNsar25Ac5Z0WIrU_Mt4/edit?ts=5a3bd366#gid=0 [22:20:47] DanielK_WMDE: to put it more generally, I think we should have a tech vision driving longterm planning, not just a product vision (not that we have a product vision right now...) and it would be cool if the devsummit (the MW architecture session more specifically) could be used to come up with it [22:20:54] I hope we talk about the future of microservices; what types of services are a good fit? what layers should they talk to? how do we enforce that? [22:21:03] not sure if that's actually feasible, just putting it out there [22:21:17] <_joe_> tgr: what do you think of the questions I just proposed? [22:21:40] <_joe_> I've seen several people put forward design documents about how to structure our architecture [22:21:51] _joe_: re efficiently storing things other than text: one outcome could be to say just that: "*if* the product strategy is to go for multimedia aggressively, we'll need to look into X, Y, and Z". [22:21:53] <_joe_> most are interesting/good [22:22:00] this year's dev summit is intended to inform strategic planning and so far I don't see that happening, it's more like the usual "let's make a tech roadmap for the next 3 years" [22:22:28] _joe_: FWIW on the router question, one of the things that came out of last weeks ATWG meeting is that we need routing in MediaWiki, but also need to route non-MW traffic (like analytics) [22:22:31] <_joe_> tgr: what is "strategic planning"? [22:22:32] TimStarling: We don't have the backend for that. The problems with mobile pretty much all go back to lacking things in mw core. How is this supposed to address any of that? [22:22:46] _joe_: i think those are a good start on the interface between "inside" and "outside" of mediawiki. i think we should seriously consider long-term things like "should password hashes be in a big mysql database" and "should someone who gets a privilege escalation on an app server have full access to private data?" [22:22:55] *also seriously [22:23:11] <_joe_> brion: of course not (passwords in mysql) [22:23:12] apergos: that's an important conversation - and we should identify it as such, and identify a venue for resolving it. the summit itself will not have enough time nor all the stakeholders present. [22:23:16] :) [22:23:16] <_joe_> and also not [22:23:20] _joe_: I agree those are among the most important current questions to discuss. I still think that's a bottom-up approach (it looks at what problems we have now, which are not necessarily related to strategy) [22:23:36] apergos: but the summit shoudl *raise* these issues, and agree on processes/venues for resolving them [22:23:50] * brion "Roadmap!" [22:23:52] <_joe_> tgr: still, they can set the directions we want to take on some topics [22:23:54] _joe_: strategic planning is, IMO, deciding where we want to arrive first and on the route to get there only after that [22:24:37] <_joe_> tgr: where depends a lot on what we want to do; I can name you 3 or 4 issues I see as critical for being able to grow in some directions [22:24:45] _joe_: if we could come to consensus on those issues, that would already be more productive than all past dev summits, sure [22:24:52] <_joe_> like - how to be able to scale inter-wiki relationships [22:25:00] DanielK_WMDE: I agree; we can't hash out any of these things in detail. But can we flag things as issues and give a high-level "services might fall into these sorts of categories, here are thoughts about those"? [22:25:01] as I said I'm not sure if strategic planning is a feasible goal [22:25:02] _joe_: thats a huge one [22:25:08] <_joe_> but that's about the *present* [22:25:20] tgr: that (a tech vision) is indeed the idea. the summit is intended to be the start of a process that will deliver that. right now, we have a ton of ideqas and issues. we need to structure them somehow so we can find a vision, and a strategy. [22:25:53] <_joe_> for the future, I'd see things like "how can we ingest more than 50 multimedia files at once"? for instance. Or what brion said, a security approach for the 2020s [22:26:15] _joe_: what is a reasonable time horizon for a tech stack? [22:26:28] <_joe_> coreyfloyd: 3 months? [22:26:28] <_joe_> :P [22:26:29] DanielK_WMDE: what I am trying to say is that the current structure proposed in the task seems to be focused on issues but not ideas (I might be misunderstanding it of course) [22:26:31] lol [22:26:38] <_joe_> (I'm half-joking) [22:26:40] I wonder because how far ahead should we plan? What is long term [22:27:16] <_joe_> coreyfloyd: long-term is anything that will develop over more than a year, or will not be needed before a year, I'd say [22:27:23] _joe_: i had a long conversation about inter-wiki change propagation with Marko the other day. [22:28:06] <_joe_> DanielK_WMDE: I think that's one of the challenges we have to tackle in order to sustain our *current* feature set for the next few years [22:28:19] _joe_ I feel like especially from the product side of the house has a 1 year horizon on features planning… and technology changes fast. So if we could create a tech strategy that lasted 5 years would that be a win? [22:28:39] 5 years is a long dang time, 15 years is unimaginable [22:29:05] tgr: i suppose you are right - and you are welcome to change that. My mind is pretty split between "issues we need to address to scale and sustain" and "what'S our technical vision" - but the later depends a lot of where we want to go with products. And that's still quite unclear. [22:29:16] _joe_: DanielK_WMDE I also think we need to split our concerns into short term (scaling current ops) and long term (strategy) [22:29:29] <_joe_> coreyfloyd: I think 5-year plans tend to fall on their back pretty fast, but sure, we could identify some directions that could influence our decisions on such a timeline [22:29:29] Haha… same idea [22:29:29] coreyfloyd: yes, i agree [22:29:42] most of the stuff i want in 5 years i want right now ;) [22:29:47] we could talk about what would be needed to support a massive increase in editors (example: 10-fold growth in wikidata) [22:29:49] question is whether it'll take 5 years to build the edifice for it [22:30:10] brion: haha, yeah basically I am wondering what time horizon are we comfortable setting a strategy for? [22:30:14] _joe_: we are not trying to make a 5 year plan. the idea is to identify the questions we need to think about to get a good idea of what we need to do to get to the place we want to be in in 5 years.. [22:30:24] <_joe_> coreyfloyd: well in some cases, current not-really-an-issue will be an issue in the future if we go in some specific directions, and require strategic planning [22:30:29] you can have a 5 year plan if it is incremental in nature [22:30:34] fwiw, i think our original "1 year plan" in parsing has ended up taking 5 yrs or so [22:30:38] strategy -> keep at 5 years. actual work plans will change every year. :D [22:30:41] don't know what that proves [22:30:43] <_joe_> cscott: ahah classic [22:30:54] Well I think there should be a move away from tons of serverside processes and complex storage, towards cheaper static storage and client side processes, perhaps using the tech used by other major websites as an example, but certainly less reliance on so much hard-to-setup, costly to run server architecture. However my view is based mainly on the view of MW as a general software for website, less what wikipedia itself is needing, [22:30:54] although I would imagine that cutting costs by cutting complexity would be a favorable outcome, especially when scaling. [22:30:57] if it is revolutionary in nature, e.g. rewrite everything in typescript, then it helps to re-evaluate after you do prototypes [22:31:10] apergos: that's a nice exercise ;) [22:31:10] cscott: pretty sure that proves: https://en.wikipedia.org/wiki/Hofstadter%27s_law [22:31:16] if the idea is to keep doing what we're doing, then we already have the prototype [22:31:18] _joe_: but it is very rewarding to finally be able to work on some of the things we talked about doing 5 years ago [22:32:03] making big changes on the wikis can be hard, there's a lot of momentum going in one direction [22:32:12] Dysklyver: the question is how you can suppoort the gazillion complex use cases that Wikimedia needs with such an architecture. [22:32:35] DanielK_WMDE: I think that begs the question “do we want to reduce our use cases?" [22:32:41] TimStarling: i could get behind a 1-year plan of 5 or so prototypes [22:32:55] each focusd on a different "key need" (or approach) [22:32:56] coreyfloyd: strategically we seem to want to increase our use cases, so ... :) [22:33:15] coreyfloyd: yes, it does. but that's not a question we can answer. and - yea. what brion said. [22:33:38] TimStarling: the danger being, of course, that we product-ize the prototypes w/o deprecating the things they were supposed to replace [22:33:38] brion: haha… I think that is something we can push back on. We can tell product that we need to scale back the features of the software in order to move forward [22:33:42] but that does give the opportunity to think about commonalities in those cases (for instance, do things need additional types of data storage -> leading to MCR etc) [22:33:45] DanielK_WMDE: ^ [22:33:48] google has done a significant amount of work towards that aim, I believe it is feasible to cover all the main customization of MW within a lightweight framework and what is effectively a web-app shell [22:33:52] we've done prototypes of node.js backend services and I'm not very impressed by them [22:34:10] coreyfloyd: that is one possible session outcome, yes [22:34:15] so if the plan is "let's do more things like RESTBase" then I would have concerns about that [22:34:40] i'd love for mediawiki-as-a-big-php-app to reduce itself into a core-with-an-api and do the ui separately, but doing it well is a tricky thing [22:34:44] tgr: do you think it is useful/sensible to try and develop a "top dow" tech vision for the platform before we know what products we want to build? [22:34:46] TimStarling: i think you need an answer to the 'last mile' q -- how do you actually scale your prototype to handle All The Things in the end, so you can turn off the old way of doing things [22:34:55] and i love the idea of storage becoming pluggable, but that's ........... hard [22:34:55] tgr: if yes, on what parameters would that be based? [22:34:58] DanielK_WMDE: so here's an example (cscott's, I think): the whole Wikimedia universe should be a single system internally. That's well beyond 5 years, doesn't depend that heavily on product plans, it can be broken down into meaningful short-term steps which won't be a complete waste of time even if the goal is abandoned after 2 years [22:35:07] tgr: sure, +1 [22:35:20] it's cross-cutting because it affects how we do cross-wiki and cross-language collaboration [22:35:25] brion: I think thats the feeling of many in audiences - to have a core and an API, then let clients build UIs [22:35:40] tgr: that's the "monolith" vision, as opposed to the SOA vision? [22:35:54] DanielK_WMDE: not necessarily, it could still be a collection of services [22:35:59] DanielK_WMDE: no, single in terms of no separate languages / projects [22:36:06] just break down the unnecessary barriers between (say) enwiki and dewiki [22:36:10] cscott: in that case, anything is a "single system".# [22:36:17] well, any set of things that communicate [22:36:18] more single-database [22:36:27] single-configuration [22:36:28] we could talk about the eternal parser issue (mw is the one true parser, should we put more resources again into changing that, or give up for good?)... [22:36:28] cscott: +1 [22:37:05] cscott: ah, ok, this is about dissolving the boundaries between sites/projects/communities, not so much about system architecture. [22:37:14] mw isn't really the one true parser, the one true parser is a platonic ideal no one has yet implemented. mw has known bugs, which are different from parsoid's known bugs, and the One True Parser is in between them. [22:37:22] DanielK_WMDE: yes [22:37:54] cscott: but that only makes sense if we anticipate that on the product level, we want/need that integration. [22:38:01] A simple way of looking at it is that all MW has to do is let people read and write files, and browse what looks like a website, everything else is UI customization and access control. [22:38:19] DanielK_WMDE: maybe. or maybe it makes sense in terms of simplifying our db config & etc purely from the tech/ops side as well. [22:38:22] Tgr on your example: I think that is a pretty neat vision [22:38:22] DanielK_WMDE: I think the summit is not a good place for coming up with one true vision (not inclusive + not enough time) but maybe as a place to figure out which vision proposals are interesting, and start some structured conversation about them? [22:38:50] Dysklyver: you are missing THE key feature that makesy wikipedia work: crowd-sourced curation and quality assurance [22:39:01] coreyfloyd, tgr https://en.wikipedia.org/wiki/User:Cscott/2030_Vision [22:39:02] cscott: mw has some edge cases that are exploited; that's why it gets to keep its status as authoritative, as far as I understand it [22:39:05] Dysklyver: that's what makes the software compelx [22:39:16] to elaborate on the "one database" idea from a product site ^^ [22:39:25] *the product side [22:40:39] DanielK_WMDE: I agree it's not going to be completely independent from product strategy. I worry that strategies put together by product people will have blind spots (just like those put together by tech people, of course) and having both kinds could be complimentary [22:40:40] tgr: yes, and asking pointed questions to product people. "so, *do* we need to support LAMP?", "do we need to see edits on wikidata on the local watchlist?" "Is supporting multi-lingual content for anonymous users important?" [22:40:43] apergos: it has some parts that don't work like they are supposed to, that folks have worked around. but that's a bit offtopic here i guess. [22:41:32] if you ask people what they want they'll say a faster horse and all that [22:41:34] cscott: tgr FWIW I am listening in on an annual planning session now in audiences and a big topic is breaking down barriers between projects to aid in content translation to improve content on other wikis [22:41:48] tgr: yes, absolutely. we need iterations between product and tech. the more the better. and the summit should at least ask for, if not establish, a process for this. should surface needs for sustaining, and questions for exploration/development. shoudl offer possibilities and opportunitites [22:41:56] from a technical view its just read/write on files, and logging, viewing of such. it's grown very complex, thats true, but fundamentally there are not that many needed functions to the software, it would be an idea to create some kind of roadmap as to what function are really needed before deciding exactly how to make the framework more lightweight. [22:42:24] tgr: yeah… they need help to understand the gaps in product planning. They don’t know about the constraints of the stack and what affects are [22:42:27] an integrated db and integrated software would mean you could split-screen enwiki and dewiki, for instance, and translate/compare content between them [22:42:41] Dysklyver: yes, that'S a pretty good point: we not only need a product strategy for new stuff. [22:42:44] or enwiki and hiwiki in india, probably a more realistic example. [22:42:48] we need a product strategy for getting rid of old stuff [22:42:55] yes well said [22:43:04] hm... [22:43:09] DanielK_WMDE: tgr yes, the only way forward will be iteration with audiences [22:43:15] DanielK_WMDE: yes, but not just removing it, sometimes also refactoring it so we can do the same things but with a better process. [22:43:37] so, perhaps one of the outcomes could be to identify "expensive" features. stuff that holds us back. stuff that would make life easier for engineers if we could agree to ditch it [22:43:52] DanielK_WMDE: i don't agree with that formulation [22:43:58] that puts the cart before the horse [22:44:05] the users are the reason why were are here [22:44:07] Dysklyver: DanielK_WMDE a new product strategy is pretty important, but audiences is operating on an “implicit strategy” that isn’t documented. Any new strategy probably assumes the old one will keep working [22:44:17] we don't get to arbitrarily decide that we don't like some of the things they are using [22:44:42] it's more productive to say, "how can we refactor this so we can do the same thing, but in a better way that doesn't make engineers tear their hair out" [22:44:47] even if no processes are removed it may make it easier to intergrate the new processes by identify similiar old ones that may be upgradable [22:44:53] we do if maintaining them in the long run means not doing other things for the users that will be more valuable [22:45:04] <_joe_> cscott: let's assume everyone is aware we're here for the users? [22:45:08] of course, who has that crystal ball, I'd like to know [22:45:12] DanielK_WMDE: having the "cost" side for some kind of cost/benefit evaluation would be very useful, although for the most important issues it's not achievable in a week IMO [22:45:29] cscott: i think it is also useful to be able to say "hey, this is really expensive, maybe doing 90% of that for 10% the cost would be enough"? I think we should ask that question. Not make the decision. [22:45:39] (which is not to say it's not worth trying) [22:45:43] DanielK_WMDE: sure [22:45:54] tgr, I for one view cost as a significant concern, but i do agree it takes time [22:45:54] DanielK_WMDE: that is definitely useful [22:46:03] i just think we've gotten ourselves into trouble with the community in the past when it looks like we're too "lazy" to maintain something they find valuable [22:46:07] tgr: yea, all we can do is to ask the right questions, and try to establish procecsses. [22:46:09] DanielK_WMDE: in Audiences engineering that is a common thing to do with product [22:46:25] from the user's perspective they should see us trying to "make the thing they love sustainable" not "kill the thing they love" [22:46:28] Its just we don’t have the same thing for engineers in tech [22:46:29] it's a big difference in perception [22:47:11] cscott: yes. on the other hand, trying to be everythingg for everyone will not work. we have gotten ourselves into trouoble that way, too. [22:47:33] Yeah that goes back to a product strategy [22:47:43] DanielK_WMDE: there's the meta-question of how we efficiently harness community involvement in order to spin off things we can't afford to maintain ourselves [22:47:43] I think I speak for all small wikis when I say that having to host an entire server just for a wiki that serves 100 people is annoying. [22:48:09] coreyfloyd: sure, that's how it should be: requirements/wishes from the one side, cost/constraints from the other. iterate, agree, rinse, repeat. that'S what we do for wikidata, too :) [22:48:17] Dysklyver: so are you asking for shared hosting? [22:48:39] coreyfloyd: i think we don't do that enough on the large scale. I hope we can start doing that for the strategy process. [22:48:39] suppose in 10 years most of our editors come from phones/tablets, or most of our edits (number, not necessarily amount of content), what would that mean? is that likely? do we want to support that? what would we have to deprioritize? [22:48:45] just tossing out possibilities here [22:48:52] not as such, just a lighter framework with better intergration with static object storage [22:49:07] i think it's 100% for sure that most of our edits will come from phones in 10 years [22:49:09] probably much sooner [22:49:10] DanielK_WMDE: yeah I think this is a large gap we are identifying… engineers who operate close to product seem to have this feedback loop, but it is non-existent at a high level [22:49:26] brion: edits, or new articles? [22:49:30] cscott: for example, one possible tech vision is a distributed-ownership microservice architecture where people can take over the fringe things that they love, and the "official" Wikimedia site interfaces them and recovers gracefully if they do a poor job and the service is not keeping up [22:49:31] the current db has to be hosted all the time to work, and it does not need to be that way [22:49:48] brion: google docs doesn't work well on phones. i think folks still use big machines & big screens when they want to actually tackle a big piece of editing. [22:49:56] Dysklyver: maybe the software for such a use case just shjouldn't be the same as the oone we use for servingg hundreds of millions of pages to hundredss of millions of users? Sometimes you need an oil tanker. Sometimes you need a speed boat. If you try to build something that is both, it's not going to work. [22:50:00] <_joe_> tgr: we have zero of that and we have a big ball of crazy interdependencies [22:50:01] tgr: yes, i'd subscribe to that vision [22:50:11] toolforge is sort if edging towards that directions, but it could be taken much farther [22:50:19] apergos: thats a good question… I can say that audiences is very focused on getting more mobile… as many new editors/readers coming on line do so on phones [22:50:20] brion: ...but not by typing. [22:50:27] cscott: the difference between "most" and "all" is significant [22:50:28] <_joe_> cscott: I would not, it would actually be very expensive and harmful to implement while maintaining the service to users [22:50:36] tgr: in my ideal world we'd have gradual on-ramps and off-ramps toward and away from "WMF maintanence of ". [22:50:39] ok, ten minute warning. [22:50:42] cscott: it doesn't work well on phones now. what app will do that work on phones in ten years (or sooner)? [22:50:50] tgr: with intermediate steps between "we take care of everything" and "we turn the service off" [22:50:51] i have gotten a lot of input here. thank you all for that [22:50:53] DanielK_WMDE, I agree with you wholeheartedly on that subject, perhaps MW and MW core could be separated a bit [22:50:59] tgr: which are well-communicated to the community [22:51:07] :D [22:51:22] our editors are not going to all migrate to tablets, tablet sales peaked in 2014 [22:51:37] https://www.statista.com/statistics/272595/global-shipments-forecast-for-tablets-laptops-and-desktop-pcs/ [22:51:47] apergos: phones are likely to become increasingly powerful as well [22:51:52] tablets are just big phones. phones are where it's at [22:51:58] +1 to both [22:52:07] now we are guessing at product strategy. [22:52:13] it's just as likely we'll have this mobile vision where your phone is actually your primary processor, and it takes over whatever TV screen and/or keyboard device you happen to be near [22:52:14] how else can we plan? [22:52:16] that said i have 2x UHD monitors and i love big screens ;) [22:52:19] wrong venue. let's try not to get lost in that [22:52:20] we have to make some guesses here [22:52:25] so there's not a sharp division between 'phone editors' and 'desktop editors' [22:52:27] <_joe_> I think we should not try to forecast tech trends ourselves, that's pretty much not what tech should think about. Flexibility, on the other hand, is something we should be concerned about [22:52:35] <_joe_> both on the frontend and the backend [22:52:54] ok let's have the mobile editing discussion another time cause we're out of time :D [22:52:59] <_joe_> flexibility allows you to react and build new pieces rationally on top of your architecture [22:53:11] apergos: yes, but we shouldn't waste too much time on making them [22:53:13] having a product strategy that's a little more evidence-based (or maybe public about the evidence?) would be great, but yeah, this is the wrong forum for that [22:53:24] <_joe_> tgr: heh [22:53:25] Hi [22:53:27] this goes back to TimStarling's mention of prototypes [22:53:32] Someone asked if I was in the forum [22:53:39] i think one way to be flexible is to invest in a number of prototypes on an on-going basis [22:53:43] apergos: or agreeing on them. in some cases, we should jhust say "Assuming X, we will need Y". and not worry too much if X will happen. [22:53:44] What was under disscusion [22:53:46] ? [22:53:46] that was me [22:53:51] so that we've done the research and are ready to pivot to whatever comes our way [22:54:10] DanielK_WMDE _joe_ So covering mobile plus flexibility: is it good to focus on APIs? Maybe focus on abstracting the front of MW from the core? [22:54:13] DanielK_WMDE: that works for me, as long as we map out possible needs for some of these big cases [22:54:14] this is a high level meeting for concerns and the future of mediwiki software [22:54:26] <_joe_> coreyfloyd: that's one way to do it, sure [22:54:34] <_joe_> well, one general approach [22:54:45] making core smaller is the way i think about it [22:54:46] _joe_: also that forces us to figure out “what is core to MW" [22:54:49] cscott: "I can write an experimental feature and release it to beta testers without risking site security and reliability" is a nice long-term tech vision too [22:54:50] Dysklyver: Long term strategy isn't relevant to my issue [22:54:52] coreyfloyd: that certainly sounds sensible to me. and one of the questiosn is "abstract how"? [22:55:08] coreyfloyd: bring this team back to life and staff it -- https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_API_Team [22:55:08] Other than one possible feature request, which can be handled via Phabricator [22:55:16] tgr: sure. the main problem like i said before is it could exacerbate the "two systems" problem [22:55:24] "Accelerate user interface innovation/evolution and increase efficiency of editor automation by making all business logic for MediaWiki available via remotely callable APIs." [22:55:27] our current mobile support is basically a prototype that got out of control, eg. [22:55:50] DanielK_WMDE: yeah, and if we go in with a goal of just separating the front and back end with APIs and supporting the same functionality that would be a huge step forward… but yes the layers need designed [22:55:50] we need to figure out how to roll the prototype back into core when it has demonstrated its utility [22:55:58] Dysklyver: Whose chairing so I can direct a question? [22:56:08] What bd808 said [22:56:28] perhaps it would be nice if mobile things were only css and not programming [22:56:31] agree with bd808 [22:56:43] we have 3 minutes left of this meeting, right? [22:56:45] DanielK_WMDE, is chairing [22:57:03] bd808: that sounds like a nice strategic goal :) [22:57:05] Thank you, DanielK_WMDE, TimStarling and All ! [22:57:25] TimStarling: yes, thanks! [22:57:25] Sveta: Late here, but it would sometimes be nice if certain things currently done inline were ultimatly strong armed into being part of the back end [22:57:33] so... any last topics to mention? [22:57:41] ShakespeareFan00: yes [22:57:43] I will summarize this discussion on the ticket ove rthe next days [22:57:47] Like countless templates on Wikisource should Really by CSS classes in the core software [22:57:50] DanielK_WMDE: we formed the team in 2015. Too bad it didn't survive the Reorg of Doom™ [22:57:50] before the actual session, obviously [22:58:01] bd808: the multitude of Special:XXX interfaces are a challenge for that goal [22:58:06] bd808: :/ [22:58:07] bd808: although i agree it's an excellent aim [22:58:11] DanielK_WMDE: i joined late, i'm not sure of the agenda or session scope [22:58:13] bd808: yes, indeed [22:58:28] There are also some things that Mediawiki doesn't properly support right now [22:58:35] cscott: not really a challange, just a job to be done [22:58:37] Sveta: no fixed agenda. scope is https://phabricator.wikimedia.org/T183313 [22:58:42] one minute left :) [22:58:45] Like beign able to run markup over multiple pages in a clean manner [22:58:58] cscott: yeah. there are hard parts. It won't get done overnight [22:59:00] https://en.wikisource.org/wiki/Short_Titles_Act_1896/First_Schedule/Pre_Union [22:59:04] tgr: it's a job that needs to be done for Marvin, etc, as well. [22:59:30] for example is broken by the new parser, because it was written in a specfic way to work around limitations [22:59:37] tgr: anyone who wants to make a *fully-functional* new UX for mediawiki needs to grapple with Special: [22:59:53] yeah special: is the special hell of mediawiki :D [22:59:57] I'm sorry if this outside the more specifc scope [23:00:01] basically everything in there needs to be abstracted sanely [23:00:06] cscott: I don't think marvin was seriously intended to be fully functional [23:00:14] brion: Hi [23:00:19] yea, refactoring special pages to share logic with APIs isn't *that* hard. it's quite doable, but it needs time. [23:00:22] tgr: yeah, and i have a problem with that. ;) [23:00:35] When you are free of meetings can i borrow your expertise on something? [23:01:01] DanielK_WMDE: Could some Special pages be re-factored as query results pages? [23:01:02] tgr: i don't like restricting the mobile experience arbitrarily to a subset of MW functionality [23:01:05] ShakespeareFan00: i feel like we will never again be free of meetings. [23:01:16] cscott: but if you wanted to be fully functional, you could get far with a form descriptor systems like the one HTMLForm uses, IMO [23:01:16] :) [23:01:32] FWIW I shared bd808’s statement with PMs and they are pretty psyched about that [23:01:35] DanielK_WMDE: Some of the Special pages are essentialy displaying query results [23:01:38] :) ShakespeareFan00: https://en.wikisource.org/wiki/Short_Titles_Act_1896/First_Schedule/Pre_Union ... [23:01:39] ShakespeareFan00: that'S what many of them are. and the way Query pages work is sadly incompatible with how query modules in the api work. [23:01:43] but, details [23:01:49] time is up... [23:01:54] thanks everyone! [23:01:57] * brion has to fight some javascript right now -- poke me by email later if needed :) [23:01:58] #endmeeting [23:01:58] Meeting ended Wed Jan 17 23:01:57 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [23:01:58] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.html [23:01:58] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.txt [23:01:58] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.wiki [23:01:58] Meeting ended Wed Jan 17 23:01:57 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [23:01:58] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.html [23:01:58] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.txt [23:01:58] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.wiki [23:01:58] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.log.html [23:01:58] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.2018-01-17-22.01.log.html [23:01:59] tgr: i just don't want to keep the existing PHP SpecialFOO.php code frozen in amber forever because it's the only way to edit Bar Baz and Bat. [23:02:00] Scott_WUaS: Yes... It breaks... [23:02:01] well thanks everyone ! [23:02:05] thanks, good night [23:02:07] :) [23:02:11] please continue any discussions in #wikimedia-tech [23:02:56] DanielK_WMDE: coreyfloyd: thanks for being the drivers of the architecture track! [23:03:44] Thanks everyone - this was a great conversaton