[22:01:05] #startmeeting RFC meeting [22:01:12] still no meetbot of course [22:01:57] Boo [22:02:22] * bd808 looks for the instructions to restart meetbot [22:02:26] https://phabricator.wikimedia.org/T108655 [22:02:38] yeah you have to be able to log in to tools, which apparently nobody can [22:03:17] https://wikitech.wikimedia.org/wiki/Tool:Meetbot [22:03:44] well hello wm-labs-meetbot [22:04:24] TimStarling: want to try the #startmeeting again? [22:04:38] #startmeeting RFC meeting [22:04:38] Meeting started Wed Feb 24 22:04:38 2016 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot. [22:04:39] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [22:04:39] The meeting name has been set to 'rfc_meeting' [22:04:53] Oops. [22:05:14] How do we get it to not over-write the /topic again? [22:05:15] #topic Standardise on how to access/register JavaScript interfaces | RFC meeting | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ [22:05:24] James_F: known crappy bug [22:05:41] Thanks TimStarling, bd808. [22:05:42] probably two lines of code that nobody could be bothered to write [22:05:59] #link https://phabricator.wikimedia.org/T108655 [22:08:22] RoanKattouw: did you want to say something about this to start off? [22:08:49] Yeah [22:09:13] So that task started fairly small in scope, but I'd like to gather some input on broader issues surrounding it [22:09:25] Quick recap: [22:09:26] The patches associated with the task implement the require / module.exports pattern [22:10:27] In every RL module, we'd make available symbols called require and module, so that a module can export its public API with module.exports = { doSomething: function () { ... } }; , and another module can then access that using require( 'moduleName' ).doSomething() [22:11:23] That seems to be fairly uncontroversial apart from a more recent comment on the task ( https://phabricator.wikimedia.org/T108655#2050855 ) that suggests this isn't actually compatible with node.js [22:11:44] (I don't really understand that comment, if somebody else does or if its author is here, I would love to hear more about that) [22:11:57] presumably module and require would be local variables injected into the script by RL's header? [22:12:04] Yes [22:12:29] We currently wrap with function ( $, jQuery ) { .... } IIRC [22:12:29] That would change to function ( $, jQuery, module, require ) { ... } [22:13:09] (ah, presumably that was a typo) [22:13:19] But another thing I want to look at, which I didn't really succeed in getting comments on the task about, is how this meshes with other module-scoped things we want to do in the future [22:13:29] YairRand: ? [22:13:45] Such as per-module config variables, or perhaps per-module i18n message exposure [22:13:45] "( $, jQuery )" [22:13:48] generally, the move towards the CommonJS modules sounds sane to me; I am wondering a bit whether we should also consider ES6 modules and async module loading; is this too early? [22:13:56] maybe he means that the string passed to require() is not the same as node? [22:14:05] in node you usually have slash separators [22:14:13] YairRand: Not a typo, sadly. Don't you just love backwards compatibility :/ [22:14:15] depends on the include path [22:14:45] * aude waves [22:14:53] gwicke: I think those are good things to consider as well [22:15:33] Re async module loading, the initial proposal is to still require dependencies to be listed in the RL module definitions, and for require() to throw an exception if you try to grab a module you don't depend on [22:15:50] That would also provide stronger guards against dependencies that work by accident [22:15:51] Krinkle proposes async loading in https://phabricator.wikimedia.org/T108655#1815998 [22:16:07] (e.g. A uses stuff from B but wrongly doesn't depend on B, but C usually loads too and C does depend on B) [22:16:12] * aude thinks commonjs would be nice and help with improved reusability and interoperability with other libraries and stuff [22:16:13] per-module config variables would be really nice. [22:16:46] and it's annoying to have to define resources stuff (e.g. resources.php) for each of our wikibase libraries, to register it with mediawiki [22:16:56] gwicke: Yes, though he since retracted his suggestion to use "require" as the name for that interface since it's not the same as CommonJS's require [22:17:13] ( https://phabricator.wikimedia.org/T108655#1838005) [22:17:30] What aude said is another thing [22:17:49] However, I'm not sure that's realistic [22:18:05] RL depends on dependency maps being known ahead of time for a lot of its logic [22:18:18] If everything is a dynamic dependency, performance suffers [22:18:28] (nodejs doesn't have this problem because loading stuff from disk is much much faster than loading stuff over the network) [22:18:35] RoanKattouw: there has to be some mor estandardized solution for that [22:18:41] * aude imagines so [22:19:04] I agree that it would be nice to have a nicer way to plug in libraries etc [22:19:25] so you wouldn't have module loading on click etc. anymore? [22:19:38] TimStarling: No, I'm not saying "kill dynamic dependencies" [22:19:48] What I'm concerned about is this [22:20:23] If you say something like "let's make it easier to plug in libraries by registering them in JS using some standard", that will probably lead to most/all dependencies being dynamic [22:20:36] More specifically, a lot of dependencies being dynamic while they could be static [22:21:03] And using dynamic dependencies in situations where static dependencies are more appropriate hurts performance a lot [22:21:34] this is the general bundling problem [22:21:49] Also --- CommonJS's require() is synchronous, which means you *can't* use it with dynamic dependencies and still be compliant with CommonJS [22:22:08] could we have something like the maintenance/generateLocalAutoload.php script that found the dynamic deps and made static mappings? [22:22:22] I'm not familiar with the state of the art in that space, and wouldn't be surprised if the ES6 module bundling tools were still a bit rough around the edges [22:22:25] (Again, in nodejs the interpreter can just stop the thread and do a disk read, but in browser JS you really don't want to do a sync AJAX call or whatever to fetch the dependency, if that's even possible) [22:22:40] Yeah I haven't looked at ES6 modules much yet [22:23:17] Question: Is this proposal for all modules? require( "mediawiki.user" ).foo, etc? bc at some point, it stops being worth it to even use them in the first place. [22:23:34] you could have require for static dependencies and mw.using or some other existing thing for dynamic dependencies [22:23:46] Going back to the proposal at hand, my feeling about it is that it's probably sane but that we might benefit from knowing (parts of) the answers to some questions about the future [22:24:15] Ones I identified on the task were 1) what do we do with future features like module-specific config, does that live in the module object or in some other injected parameter or where? and 2) how do we do async loading [22:24:30] From this discussion I also hear 3) what about ES6 modules, 4) how do we make registration easier [22:24:32] maybe others? [22:24:47] (by 4 I mean for plugging in external libs to work with RL) [22:24:56] TimStarling: Yes I think that's what we'll have to have [22:25:03] RoanKattouw: to me it sounds like one question is whether the cost of updating all the modules once for an intermediate solution is worth it, or whether we should wait a little longer until the tooling around ES 2015 modules is mature enough [22:25:13] And that's what the current proposal essentially is, since it only touches static deps [22:25:21] gwicke: Right, that too [22:25:36] I think we should at least wait a little bit to discuss some of these questions [22:25:41] Waiting for ES6 tooling to get up to scratch is too long IMO [22:26:13] YairRand: It's not clear to me whether it'd be required for all modules (Krinkle isn't here :/ ) but I do think that that's where it's going. [22:26:43] I know too little about the current state in that space to estimate how long of a wait this would be [22:26:44] That's not as bad as it sounds though, you can do var mwuser = require( 'mediawiki.user' ); and use mwuser.foo , mwuser.bar, etc. That's already how programming in nodejs works [22:26:51] It's just one extra line per module that you use [22:27:28] HTML5 has https://html.spec.whatwg.org/multipage/webappapis.html#module-script [22:28:01] Anyway -- I'm not necessarily saying we have to have a complete plan before we merge this relatively small require/module.exports thing, but I think we should explore these issues a little bit to see if merging this paints us into a corner [22:28:13] RoanKattouw: It's barely worth it with the necessary mw.loader.using( "mediawiki.user", function () { ... already required. sometimes this is just for a function that's barely a line long. [22:28:48] YairRand: That's only if the dependency is dynamic though. For static deps you don't need that [22:29:14] given the popularity of CommonJS modules, it seems very likely that there will be no shortage of automated migration paths to ES6 modules eventually [22:29:44] For example, I suggested adding module.getConfig() for module-specific config, but then Krinkle said something about how CommonJS doesn't expect/want there to be other properties on the module object. I didn't quite understand that [22:30:10] * RoanKattouw pings Krinkle to ask him to come in here [22:30:10] RoanKattouw: To the best of my knowledge, that's the only way to have dependencies for user scripts and site js, dynamic or not. [22:30:26] YairRand: Yes, that's right :S [22:30:43] For gadgets we have static deps nowadays, but for site/user JS, only dynamic dependencies are available [22:30:58] RoanKattouw: have you considered using CommonJS tools like browserify? [22:32:22] * RoanKattouw looks through browserify docs [22:32:34] Looks like it does static analysis of JS and builds bundles [22:32:53] yeah; it also has some ability to bridge differences between browser and node [22:33:53] That seems secondary though? [22:34:00] What I'm seeing in the docs is all about it being a server-side build step for bundling and dependency management [22:34:08] yeah, the bundling / minification part is more important [22:34:29] Oh I see, there's a builtins section way down the page [22:35:23] that's done on an as-needed basis IIRC, so if your code doesn't do anything node-specific, no overhead is added [22:36:06] Right, through more static analysis [22:37:08] Something like that (static analysis to identify dependencies, allowing the same API to define dependencies in in-git code and in user scripts) could be the future [22:37:20] * Krinkle is here [22:37:35] I'm not totally sure if that's where I want RL to go, but it seems like we'd benefit from being compatible with widely used standards [22:37:36] (now) [22:37:43] Hey :) [22:38:53] it's often combined with google's closure compiler for further minimization [22:39:42] like dead code elimination [22:40:12] gwicke: I proposed async loading indeed, but I've changed my mind. For lazy-loading we can continue to use mw.loader.using() with callback, in which you can require(x) any modules that have been loaded. We can add a convience signature to require() that does this, but that's trivial. The main pipeline requires dependencies and that's mostly for performance [22:40:12] and consistency reasons. [22:40:55] gwicke: Also, good point from earlier, there are likely to be CommonJS->ES6 migration tools out there in the future so if we go with CommonJS-compatible things that's probably pretty safe for ES6 in the fuutre [22:41:13] The proposal is for the system. No modules will have it by default since JS has no mechanism to indicate what symbol is exported (that's what this is about), so at first require('mediawiki.util') wouldn't work, but we would indeed add a line to util.js that exports the object in question and keep the public identifier as back-compat [22:41:22] RoanKattouw: yeah, seems like a safe bet to me as well [22:41:26] It's opt-in. Not required. [22:41:46] Public identifiers will still be there for back-compat. Though some new modules may choose not to expose global variables [22:41:46] And the intention is to eventually migrate everything to this export/require pattern, right? [22:41:50] As far as static dependencies go that is [22:42:21] Like, in the (far?) future we could conceptually remove the back-compat globals / public identifiers [22:43:07] Maybe, but I'm not sure that's desirable or feasible at this point. Plugins and frameworks based on globals will be based on globals. [22:43:24] True [22:43:36] It's also uncertain how we'll handle things from our own plugin=based systems evne, like mediawiki.api.* [22:43:53] It's probably pretty difficult to wean ourselves (or our libraries) off the $/jQuery global at this point, for example [22:43:53] I think it'll work fine (the export will be the added method(s)) [22:44:27] Yeah I think that should be OK [22:44:35] But yeah, shims and plugin-based libraries will likely remain global. [22:44:50] We'd make require() the preferred way of doing things though, I suppose [22:44:52] The main benefit is not needing to have a global identifier for everything [22:45:07] Yeah, whenever sensibly possible, that should be used. [22:45:43] browserify seems irrelevant to us. That's for converting node code primarily. We don't have the problem it is solving. [22:45:45] So, going back to my list of future questions from earlier: 1) what do we do with future features like module-specific config, does that live in the module object or in some other injected parameter or where? 2) how do we do async loading, 3) what about ES6 modules, 4) how do we make registration easier [22:46:11] re 3, as Gabriel says, migration tools or shims from CommonJS to ES6 will probably become available once ES6 modules are a serious thing [22:46:33] Krinkle: the bundling-via-require part seemed relevant [22:46:37] re 2, AIUI you propose keeping mw.loader.using, perhaps with a require-like alias [22:47:10] gwicke: Well that's basically what ResourceLoader does, except that RL doesn't do static code analysis to get dependency information, and browserify doesn't support a bunch of weird things that we need that RL has [22:47:36] What is going to be the pattern for dynamic dependencies? [22:47:40] (1) mw.loader.using( 'module.foo' ).done( function ( foo ) { foo.doThing(); } ); [22:47:44] or (2) mw.loader.using( 'module.foo' ).done( function () { require( 'module.foo' ).doThing(); } ); [22:47:51] Like dynamic dependencies and modules with dynamic contents etc [22:48:14] Hmm that's a good question [22:48:17] #1 is what we currently have and we'd have to keep it for b/c [22:48:18] Ricordisamoa: It would be mw.loader.using( 'module.foo' ).done( function () { require( 'module.foo' ).doThing(); } ); or require(['a', 'b'], function (a, b) [22:48:31] never mw.loader.using( .., function ( foo ) {} because promises must only resolve a single value [22:48:36] there are a lot of switches, options and tools around it, so I wouldn't rule out that browserify could still be useful as part of the overall system [22:48:53] Krinkle: oh I missed that thx [22:49:05] Or, wait, no, sorry, #1 is not what we currently have, my mistake [22:49:06] I think what Krinkle says makes sense [22:49:24] more generally, it would possibly keep us closer to node / standard commonjs [22:49:26] Ricordisamoa: and this only applies for user scripts. FOr anything else you don't call mw.loader.using() at all, the dependenices are there as now. The only change is that instead of mw.Title() you do var Title = require('mediawiki.Title'); [22:49:38] You couldn't do mw.loader.using( ['foo', 'bar'] ).done( function( foo, bar ) { ... because you can't pass two params to a done callback [22:50:10] Krinkle: yes I understand. The problem is, I deeply care about user scripts :-S [22:50:12] RoanKattour: also because we currently don't know what a module exported anyway [22:50:13] gwicke: Apart from polyfilling things like nodejs's url handling, I don't see much value there IMO [22:50:28] Krinkle: Yeah, I realized that, ignore me :) [22:51:11] Krinkle: BTW we discussed per-module config at one point, and I suggested something like module.getConfig( 'wgFoo' ), but you said something about not wanting to add things to the module object? [22:51:20] Do I remember that right? [22:51:49] Yeah. [22:52:09] It makes inter-operability complicated because 'module' has certain assumptions with it. [22:52:12] With mw.Map shouldn't that be module.getConfig().get( 'wgFoo' ) ? [22:52:47] It's a read-only map, so we can just as easily expose config.get() getConfig(), no problem. [22:52:50] Ricordisamoa: Yeah, or module.config.get/set. Good point, you probably want to have set() available [22:52:53] in node there is some random stuff in the module object [22:53:22] True [22:53:25] Krinkle: What (roughly speaking) are the interoperability assumptions around the module object? [22:53:38] like module.children, module.filename, module.id [22:53:41] I don't necessarily need all the details, but I'm trying to understand how and why this is an issue [22:53:44] I don't have a strong opinion about browserify; I do hope though that we can leverage CommonJS tools as much as possible, and avoid implementing something that's almost-commonjs-but-different-enough-to-prevent-node-compat-and-tools-reuse. [22:54:13] RoanKattouw: haven't looked too far yet. My main concern is that all but .exports is "standard" (standard enough that we share code, e.g. oojs is node-compatible and RL-compatible, same distribution, same for other libraries we have). [22:54:21] Which means if we add things to it, we can clash. [22:54:33] gwicke: I agree that we should use the CommonJS *API*, but not necessarily their *tools* unless it makes sense [22:54:39] And it'll be a mess for developers to figure out when or how to use what. [22:54:40] Oh, I see [22:54:52] Using module.* in Node is obscure and usually reserved for things that are truely specific to Node [22:55:01] Because everything else is non-standard, what if module.config appears in other CommonJS-compatible things [22:55:13] Which is reasonable I suppose. If something needs mw.* it is RL specific as well. [22:55:35] OK that convinces me why attaching important things to module.* is a bad idea [22:55:44] maybe mw.getConfig(module) is better [22:55:57] gwicke: To that point, we should probably also talk about how Krinkle and jdlrobson's require() proposal uses module names whereas nodejs can also use paths [22:56:03] mw can look at module.id or whatever to get an index into a table [22:56:07] We'd need a module.__name property because we don't have WeakMaps [22:56:08] Yeah [22:56:18] I think it's okay to stuff things in module for private consumption [22:56:31] +1 to mw.loader.getConf( Object module ) [22:56:48] it's time to wrap up now [22:57:15] should this go to a last call? [22:57:16] (interesting bit that may no longer be relevant: mw.loader.using( ['foo', 'bar'] ).done( function( [ foo, bar ] ) { /* ... */ } ); is now technically valid syntax, iiuc.) [22:57:39] RoanKattouw: yeah, but when you are creating your own tools it's easy to get carried away, and break compatibility in the process [22:57:54] Hmm, that feels very "C-binding for a C++ API" to me... but maybe [22:57:54] We'd only need one name property for that to work [22:57:54] It's just frustrating to have an object and not be able to call methods on it but instead have to hand it off to another function [22:57:56] and then you lose a lot of the benefits of trying to use CommonJS in the first place [22:58:28] can we please stop talking about specifics for a minute and make some summaries? [22:58:58] how about having one extra field in module, like __mw__ or whatever, and attaching everything else to that? [22:59:00] RoanKattouw: If we have more things we can generalise. e.g. var t = mw.loader.things(module); t.config, t.messages [22:59:04] OK, yes, let's wrap up [22:59:06] the chance of clashes would be minimal [22:59:28] I think it's OK for this to go to last call [22:59:31] module.exports/require is basically approved, details can be discussed in gerrit, correct? [22:59:35] DanielK_WMDE_: Yeah, that's fair. [22:59:46] (Although no underscores please) [22:59:50] I'll ask a question on the task about module name vs path stuff [23:00:15] generated modules have no path btw. [23:00:18] RoanKattouw: could you also summarize the discussion here? [23:00:18] I also think we agree that we should really aim for something that's actually CommonJS-compatible, not almost-but-not-quite as gwicke fears [23:00:27] Krinkle: module.dumpStuffHere [23:00:44] RoanKattouw: I found it useful to write up a couple of paragraphs, and then post it on the task & the list [23:00:57] gwicke: Will do in 1-2 hours [23:01:12] I have another meeting first [23:01:19] cool, thank you! [23:01:29] DanielK_WMDE_: module.mw could work I think [23:01:39] Alright, I think we're done [23:01:40] yea, i was just thinking that [23:01:47] TimStarling: Anything else procedural or meetbot-related we need to do? [23:01:51] #agreed LAST CALL: backwards-compatible introduction of module.exports and require() [23:01:54] We can namewar on Gerrit [23:02:07] https://people.wikimedia.org/~gwicke/bikeshed.svg [23:02:26] Is mw.loader going to be the last global™ or will you have to require( 'mw.loader' ) ? [23:02:31] thanks, everyone! [23:02:33] #info CommonJS compatibility if at all possible [23:03:05] #info module object members are not preferred as a public interface but the object may have some private properties [23:03:40] Ricordisamoa: It can be available. It'll have to be public either way since the server won't have a context yet. So there's no reality in which it can be removed, even with everything converted. So might as well save the lookup and use it. [23:03:48] Node has other globals as well besides module, such as process. [23:03:55] mw.loader is like our process, in a way [23:04:01] I guess that is it [23:04:06] Krinkle: thanks [23:04:19] #action RoanKattouw will write a summary on the task [23:04:25] * RoanKattouw hopes that's the right syntax [23:04:30] yup [23:04:35] #endmeeting [23:04:35] Meeting ended Wed Feb 24 23:04:35 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [23:04:35] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-02-24-22.04.html [23:04:35] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-02-24-22.04.txt [23:04:35] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-02-24-22.04.wiki [23:04:35] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-02-24-22.04.log.html [23:04:59] RoanKattouw: One thing to look into is the recent breakage of at least one module that defines its own local 'require' function and uses it in a module with 2 files. In debug we currently end up plugging that away with our own half-way. [23:05:05] thedj filed it [23:05:14] when you use an IRC nickname as the first parameter to #action, it sorts the action items by person assigned [23:05:20] sigh, of course [23:05:36] Nice, I figured the first param needed to be my nick [23:05:39] debug-only which means wontfix is an option, in favour of referring to 'fix debug' [23:05:42] and then when people screw it up and put some other thing at the start of the #action line, it puts those action items in an unassigned section [23:05:55] * RoanKattouw goes away for another meeting, back in an hour [23:05:56] Krinkle: Oh I saw that task. Yeah I agree [23:06:30] mw.loader.using( 'mediawiki.user', function () { require( 'mediawiki.user' ).isAnon() } ) is quite a lot longer than mw.config.get( 'wgUserName' ) !== null... ( or wgUserName !== null, even, deprecated or not) [23:07:07] YairRand: require( 'mediawiki.user', function ( mwuser ) { ... mwuser.isAnon() ; } ); is shorter though [23:07:16] TimStarling: I looked at the code for meetbot. Fixing the topic stuff will take hacks or adding new functionality. :/ [23:07:38] The "Brains" of the beast are https://github.com/openstack-infra/meetbot/blob/f56a92a74a41833edff5d1e9cbae4675f9800be8/ircmeeting/meeting.py [23:08:05] RoanKattouw_away srry i got completely side tracked by a meeting i forgot to go to. Am catching up on the back scroll [23:08:09] RoanKattouw_away: no it's not, it's almost twice as long... ? [23:08:13] hah it's from openstack-infra, why am I surprised [23:09:02] Oh than the wg thing. Sorry yes [23:09:21] mwconfig is more practical there [23:09:44] RoanKattouw_away: it's actually a fork of a debian bot and uses some interesting shared bot framework from them [23:18:42] is there another office hours here in three hours? [23:24:19] YairRand: Don't think so.