[18:09:22] Hey! Would anyone mind if I changed the top portion of the Etherpad to gray? It's very hard to read in dark blue. [18:10:34] melodykramer: +1 [18:11:47] Done! [18:27:22] That was the first CREDIT showcase that I managed to watch live :) [18:27:24] Cool stuff [18:27:36] s/live/realtime [18:34:32] I would love feedback on the new CREDIT template: what you liked, what you didn't like, suggestions. [18:34:55] We modified the template based on the one Mozilla uses for their monthly conference calls (because it's easier for people to parse afterwards.) [20:56:59] upcoming discussion: https://phabricator.wikimedia.org/E287 [20:57:14] https://phabricator.wikimedia.org/T145604 "Future of magic links" [20:57:43] https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links [21:01:02] #startmeeting RFC meeting [21:01:02] Meeting started Wed Oct 5 21:01:02 2016 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot. [21:01:02] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [21:01:02] The meeting name has been set to 'rfc_meeting' [21:01:05] good afternoon [21:01:18] #topic RFC: Future of magic links | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) |​ Logs: https://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ [21:01:54] #link https://phabricator.wikimedia.org/T145604 [21:02:08] #link https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links [21:02:49] hi everyone! hi legokt^H^H^H^H^H^Hcscott :-) [21:03:32] i will be serving as your legokt today [21:04:28] there are three steps we discussed that are summarized in the description of T145604 [21:04:29] T145604: [RfC] Future of magic links - https://phabricator.wikimedia.org/T145604 [21:05:04] Hi [21:05:07] step 1: Disable the magic link functionality by default for the MediaWiki 1.28 release, and mark it as deprecated. [21:05:30] 2. Deprecate magic links on Wikimedia wikis (e.g. Wikipedia), providing alternatives for this functionality and tools to aid the migration [21:05:30] 3. Disable magic links functionality a year or so after the MediaWiki 1.28 release (in time for the next MediaWiki LTS release) [21:05:50] is there anyone that objects to step 1? [21:06:34] is legoktm online? [21:06:43] maybe we should find out who's here? (that is, actively participating in this discussion.) [21:07:03] TimStarling: no, legoktm is in class, so I'm being his surrogate for this meeting. [21:07:16] legoktm is at a class right now, so if there are objections or things he needs to clarify, we'll need to plan a followup [21:08:02] (Hi CScott - I'm reading along, typing occasionally) [21:08:05] it seems, though, that this is a "yeah, we should just do this" kind of thing for step 1 [21:08:08] * subbu is around and is generally happy with the direction, modulo details that might need tweaking [21:08:31] is there going to be an extension to replace it? [21:08:39] step 1 is normal deprecation procedure I think [21:09:02] a workalike extension could be created if someone wants one but i don't know if anyone does :) [21:09:08] i think there's still a semi-open question w/ what the replacement should be: 1) ordinary template, 2) parser function, 3) extension, 4) interwiki link, 5) something else? [21:09:13] if there's no migration path for external users then I don't think it should be deprecated [21:09:13] Kunal wrote something about that in email/phab today... [21:09:21] just disabled by default [21:09:50] my inclination is ordinary template is the right thing for wikipedia [21:09:54] 1) and 2) look like {{ISBN|xxx}} and require content edits, 3) would be a workalike using a parser hook, 4) would look like [[isbn:xxx]] and require content edits. [21:09:58] legoktm's position is that disabling just disables the hyperlinks, but it's still readable [21:10:27] it is a relatively clean 'failure mode' yes [21:10:38] no explosions/php fatals :) [21:10:40] i think we already have a config option for disabling it in the parser? legoktm wrote that patch. so it's just a matter of turning that config option off on WMF wikis. [21:11:27] we can discuss removing the feature from the codebase and/or moving it to a parser extension as a separate thing. [21:11:31] I don't like option (3) [21:11:38] TimStarling: "We would move the Booksources code and ISBN parser function to an [21:11:39] a extension." -- https://lists.wikimedia.org/pipermail/wikitech-l/2016-October/086716.html [21:12:11] i.e. option (3) for parsing "ISBN ....." as a magic link with a parser hook [21:14:15] i don't like it for WMF wikis, but it would be a reasonable thing to allow 3rd parties to continue to support magic links on their external wikis if they felt strongly about it [21:14:21] using square brackets seems elegant to me [21:14:27] re TimStarling's "if there's no migration path for external users then I don't think it should be deprecated" [21:14:32] considering it is an actual link [21:15:03] I can go with either {{isbn|..}} or [[isbn:..]] personally [21:15:04] pmid and rfc are proposed to become default interwiki links [21:15:24] isbn is the only oddball right? [21:15:27] [[rfc:1234]] would work fine [21:15:37] isbn is the oddball because it goes to *the wiki itself* [21:15:42] *nod* [21:15:54] yeah, it would be like a namespace alias [21:15:55] although i don't think there's anything preventing interwiki links from being relative URLs? [21:15:58] like media [21:16:29] it could even be a negative namespace ID rather than an interwiki prefix [21:17:15] interesting idea [21:17:37] i think i prefer an interwiki, with special:booksources as a special service that happens to be the link target [21:18:27] is there a precedent for magic link functionality in other wiki markups? e.g. does anyone else automatically turn "ISBN \d+" into a hyperlink automatically? [21:18:36] but negative (magic) namespace has a certain elegance to it too [21:19:11] robla: external link hyperlinking is the precedent [21:19:16] ie, http://foo.bar/bat [21:19:34] most other markup languages have that [21:19:43] bugzilla autolinks textual things like 'bug 1234' [21:19:49] when given a bare url [21:19:57] and of course many mobile browsers autolink phone numbers (which i hate ;) [21:20:00] github markdown autolinks hashes and bug references [21:20:25] $RFCPattern = "RFC\\s?(\\d+)"; [21:20:25] $ISBNPattern = "ISBN:?([0-9- xX]{10,})"; [21:20:32] s/$RFCPattern/&StoreRFC($1)/geo; [21:20:38] s/$ISBNPattern/&StoreISBN($1)/geo; [21:20:40] https://help.github.com/articles/autolinked-references-and-urls/ [21:20:40] says usemod [21:20:49] so this feature was in fact carried over from usemodwiki [21:21:08] PMID was a more recent addition [21:21:47] that would imply a precedent that we're continuing rather than merely an oddball legacy feature of our own [21:22:17] weeelll. [21:22:30] that said, I think I agree with brion's hatred of autolinked phone numbers [21:22:38] precent becomes oddball legacy given enough time (and lack of external adoption) [21:22:45] usemod is also responsible for the "free link" terminology [21:23:25] it's quite likely just an oddball legacy feature that we inherited from grandpa usemod :-) [21:24:45] the argument for deprecation would be simpler parser code? [21:25:27] simple parser spec & removal of an english-specific bit of markup [21:25:42] but as long as we magically link free URLs, we will still have Parser::doMagicLinks() [21:25:48] RFCs in particular are english-only and not really relevant for most of our projects [21:26:15] that's the argument for step 2 [21:26:16] TimStarling: And the magic of image links -> remote transclusion when that evil flag is enabled. [21:26:20] I mean the argument for step 1 [21:26:23] sure, but doMagicLinks() could be greatly simplified and only match https?: prefixes [21:26:30] for example [21:26:39] instead of the pretty complicated regexp for ISBNs [21:26:40] I am fine with step 2 and 3 but not so much with step 1 [21:27:02] and our free URLs autolinking is pretty crazy complicated because of all the different protocol schemes we support in theory. [21:27:30] I don't like "magic links" because they suddenly add special meaning to some text substrings ... [21:27:59] it complicates WTS as well, you have to watch for all sorts of boundary conditions when serializing [21:28:01] if it's disabled on WMF wikis then parsoid doesn't need to support it [21:28:19] Parsoid is for more than just WMF users, eventually. [21:28:22] instead of using uniform syntax .. and as someone argued, RFC xyz is ocfusing since mediawiki has RFCs. [21:28:25] o_0 parsoid is Wikimedia only TimStarling? [21:28:29] *confusing [21:28:44] and if we're going to continue to do round-trips for content migration, whether to wikitext 2.0 or markdown or , then simplifying WTS (wikitext serialization) will continue to be desirable [21:29:02] parsoid has a reduced feature set compared to MW [21:29:04] it seems like protocol links (like mailto:.*, http?s:.*) have ample precedent in both email and wikis. Magic words as arbitrary barewords are relatively rare [21:29:05] and all the nowiki crap. [21:29:09] it doesn't even support wikipedia properly [21:29:24] well that I can't argue against [21:29:38] robla: yes, but have you looked at the full list of protocols we support? [21:29:54] there are no plans to make parsoid support every single MW parser extension [21:30:01] cscott: not in a while [21:30:05] * robla goes to look [21:30:22] $wgUrlProtocols = [ 'bitcoin:', 'ftp://', 'ftps://', 'geo:', 'git://', 'gopher://', 'http:// ', 'https://', 'irc://', 'ircs://', 'magnet:', 'mailto:', 'mms://', 'news:' , 'nntp://', 'redis://', 'sftp://', 'sip:', 'sips:', 'sms:', 'ssh://', 'svn://', 'tel:', 'telnet://', 'urn:', 'worldwind://', 'xmpp:', '//' ]; [21:30:27] TimStarling, yes .. i've proposed that parser hooks that rely on parser internals should instead be deprecated. [21:30:46] parsoid and php parser have different internals. [21:30:47] i'm pretty sure we could drop gopher:// and worldwind:// [21:31:05] I hear gopher is going to make a comeback ;) [21:31:25] free external links could support a reduced set of protocols compared to bracketed external links [21:31:27] but in general, I like {{bitcoin|}} or [[bitcoin:hash]] better than autolinking bitcoin:cafebabe in text [21:31:42] but, anyway ... we do want to get to a unified model .. and that parsoid doesn't support wikipedias is neither here nor there .. whatever parser it is that supports wikipedias will need to grapple with the roundtripping and html2wt issue. [21:31:45] that's why we have so many protocols, because non-WMF users want to link to those things [21:32:25] * robla doesn't think the list cscott provided seems so crazy [21:32:29] TimStarling: perhaps, but I think they could manage to use slightly different markup to accomplish the same task. [21:32:55] like bracketed external links? [21:32:59] exactly [21:33:23] or double-bracketed links for some things perhaps [21:33:26] [[Gopher (protocol)]] does of course have many gopher links on it [21:33:29] robla: agreed - cscott's list could make sense [21:33:40] but all bracketed as far as I can see [21:33:50] cscott: Pretty sure gopher:// is officially deprecated now, from what I vaguely recall [21:35:55] so i'm going to be a poor advocate for legoktm and say I'm not particularly chuffed by magic links. i'd get rid of them for wikitext 2.0 as part of a broader simplification, but now that they are implemented and working it's not really helping us much to get rid of them. [21:36:24] so...perhaps we can agree to make the linking aspect off by default, without necessarily declaring them deprecated in 1.28 [21:36:35] i guess the question is whether you think we can slim down wikitext incrementally by turning off these weird crufty bits one by one, or whether we should treat it as a completely new markup language [21:36:48] and use something like "parsoid2" to do round-trips between "wikitext 1" and "wikitext 2.0" [21:37:03] what i think of wikitext 2.0 is probably different from what cscott thinks of wikitext 2.0 probably :) [21:37:15] * marktraceur may have been wrong, the minnpost article he must have seen was just a random summary of Gopher [21:37:27] i imagine a language with a formal grammar, small enough to fit on a single sheet of paper. [21:37:36] * robla doesn't want to rabbithole on that topic, when we might actually be able to make a decision about Mediawiki 1.28 [21:37:36] but otherwise looking as much as possible like wikitext 1.0 [21:37:36] marktraceur, ah ... now it makes sense gopher came form UMn ... and that is why gopher [21:37:53] cscott: How big is the piece of paper? Implementation details... [21:38:13] can we turn magic words off by default in 1.28 without deprecating? [21:38:27] and so, listing 28 different external link prefixes and 12 or so separate productions for ISBN/RFC/PMID is not going to help wikitext 2.0 fit on a single sheet of paper [21:39:16] er...I suppose I should have said "magic links" [21:39:28] can we turn magic links off by default in 1.28 without deprecating? [21:40:01] sure .. i guess the discussion is more about whether deprecation is required / desirable. [21:40:12] https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pegTokenizer.pegjs.txt#L449 is the grammar for RFC/PMID/ISBN in parsoid, for reference. [21:40:46] as far as i'm concerned the real question is: can we get the *wiki communities to start rewriting content to use whatever our preferred replacement markup is? [21:40:46] subbu: yes, but that seems to be sending us down the rabbithole of talking about Wikitext 2.0, which we aren't going to finish in this hour [21:41:06] personally, i don't think we need to talk about wikitext 2.0 there. [21:41:20] https://en.wikipedia.org/wiki/Template:ISBN was created by MZMcBride [21:42:01] and it seems to already be used by quite a number of pages [21:42:15] as a compromise, how about not deprecating it immediately, but revisit after a year or two and see how many people are turning on the option? [21:42:31] i think it is a worthwhile discussion in its own merit. but, like cscott i don't have strong feelings ... but yes, it will simplify some code in parsoid .. but i won't be heartbroken if it is kept as is. [21:42:37] one intermediate step might be to hack the parser to add a parser warning if you use magic link syntax, suggesting an appropriate rewrite [21:42:46] * robla likes TimStarling's suggestion [21:43:08] we can do that w/o committing to deprecation, just encouraging people not to use that syntax. [21:43:14] THen we'd just need a way of collecting that data [21:43:19] and then, as TimStarling suggests, give it a year or so and see what our trends are. [21:43:30] ya what bd808 says .. do we have a mechanism for collecting that data? [21:43:45] we could probably hack together a dumpGrepper script that would collect repeatable numbers for long-term comparison purposes [21:43:47] there's that pingback feature ori introduced... [21:44:06] we could also add [[Category:Uses Magic Link Syntax]] [21:44:38] * robla looks for the link to the aforementioned pingback feature [21:44:44] i know (or am i imagining it?) kaldari had strong feelings about magic links. curious if he is around. [21:44:49] currently it doesn't send any configuration [21:45:31] #link https://www.mediawiki.org/wiki/Manual:$wgPingback [21:46:03] My suggestion was to just kill the RFC magic linking entirely, as it's usefulness is very marginal [21:46:22] otherwise, I support making it configurable [21:46:42] and depricating [21:46:46] I think that part has been done now. there's a feature flag for it [21:47:14] and this discussion is about deprecation and removal from core [21:47:26] the dynamic dates was removed, there is that precedent, but dynamic dates was never on by default and was rarely used by non-WMF wikis as far as we know [21:47:57] we could conceivably deprecate in 1.29 if the 1.28 release goes well [21:48:03] whereas we've been told that magic links are regularly used [21:48:22] ISBN in citations only from what I can tell. [21:48:31] I mean outside WMF [21:48:35] ah .. [21:48:40] for WMF we can run bots, outside WMF not so much [21:48:57] outside WMF we don't know if people even want to run bots, or if they like their magic links the way they are [21:49:04] the pingback feature would give us more certainty, but I think even just "did anyone complain?" might be a good enough test in this case [21:49:35] or would anyone who would complain actually upgrade anyway... [21:50:35] it would be nice if WMF published a bot framework and scripts for it w/ every major upgrade [21:50:53] like how we upgrade database schemas, we could provide tools for external wikis to update their content [21:51:02] I guess I'm not sure why it needs to die if there is a feature flag other than hypothetical future parser world that would like to not support the feature [21:51:18] cscott: I think we call that pywikibot [21:51:22] I suppose deprecating in 1.28 would be the "be bold" way of doing it. we could be prepared to "undeprecate" if people hate that we turned it off [21:51:44] bd808: sure, it's the "official" part and "with scripts for every major upgrade" which is missing. [21:51:47] Do "off by default" changes need to go through the one-version-notice process? [21:52:07] heh [21:52:14] disable by default then silently remove in 1.29? [21:52:20] Tut. :-) [21:53:05] James_F: I don't *think* we have a policy like that, but I suppose that may be what legoktm's other rfc covers in more depth [21:53:27] cscott: I'm not sure I agree on "official" and WMF being intrinsically intertwined, but some suggested wikitext cleanup scripts would be an interesting thing to add when changing the syntax [21:53:38] Sorry, yes, I more meant "would it be covered by the semi-in-practice policy?". [21:53:46] the deprecation rfc: T146965 [21:53:47] T146965: [RfC] Deprecation policy for PHP code in MediaWiki - https://phabricator.wikimedia.org/T146965 [21:53:52] i'm always interested in process improvements to make it easier for us to evolve wikitext syntax [21:54:57] so should we call the non-controversial parts of this approved? [21:55:13] * subbu cares less about syntax and more about underlying processing model / semantics except where the syntax gets in the way [21:56:09] wfm .. but, explicitly listing out the non-controversial parts would be useful. [21:56:26] #agreed 1. Disable the magic link functionality by default for the MediaWiki 1.28 release [21:56:58] #agreed 2. Deprecate magic links on Wikimedia wikis (e.g. Wikipedia), providing alternatives for this functionality and tools to aid the migration [21:57:26] hmmm....I'd feel more comfortable with broader input on #2 [21:57:42] sure. and the controversial step 3 would be removing the magic link code from core (perhaps moving it to an extension)? [21:57:52] I suppose "start step #2" isn't yet controversial [21:58:01] Yes. [21:58:10] i'd propose 2(a) -- add a category and parser warning for magic links on wikimedia wikis, without officially deprecating the feature. [21:58:22] cscott: To move to an extension it'd have to keep the horrible hooks into the PHP parser, though? [21:58:28] then sit back and see if folks are getting the magic links cleaned up or not. [21:58:35] #info no consensus on removing the functionality from MW or flagging its pending removal via deprecation [21:59:21] no, step 3 is disabling magic links on WMF wikis [21:59:23] James_F: yes, although in my big picture worldview we're adding a pluggable parser API, and gradually moving from the "old" PHP parser to a newer one. [21:59:32] * James_F nods. [21:59:34] removing from MW core was 1b [21:59:45] so the hooks would only stay in the legacy PHP parser. which, of course, you could keep running if you like and are willing to keep it maintained. [22:00:05] that's one possible universe, yes cscott [22:00:06] (Thanks, All) [22:00:27] bd808: i keep pushing the universe towards my platonic ideal of it. ;) [22:00:35] * cscott has to turn into a pumpkin [22:00:54] I like the "everything in the core php app" universe better [22:01:03] but that's why we have these chats [22:01:10] * robla plans to turn into a different type of squash [22:01:29] its a dangerous time of year to be a pumpkin [22:01:37] srsly [22:01:46] could be gutted and filled with candles at any point [22:01:47] what was next week's RFC again robla? [22:01:57] next week we'll plan on talking about CREDITs file [22:01:59] * robla finds link [22:02:18] https://phabricator.wikimedia.org/T139300 [22:02:20] bd808: pluggable doesn't mean not monolithic or not php, btw [22:02:26] just that things are decoupled [22:02:39] so you can have a pure-PHP single-process markdown wiki if you like. [22:02:42] ok, all done [22:02:45] #endmeeting [22:02:45] Meeting ended Wed Oct 5 22:02:45 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [22:02:45] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-10-05-21.01.html [22:02:45] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-10-05-21.01.txt [22:02:45] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-10-05-21.01.wiki [22:02:45] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-10-05-21.01.log.html [22:02:53] cscott: *nod* service interfaces are abstractly good [22:03:14] or a mixed-language collection-of-services wikitext 1.0 wiki using the legacy PHP parser-as-a-service. [22:03:14] etc [22:40:02] o/ [22:40:22] not a single mention of composer, maybe I should skip these meetings more often ;-) [22:40:28] but thanks all for discussing without me :) [22:44:10] * robla tries to figure out if legoktm was making a remark about frequent composer tangents in RFC meetings, or if that was about something else [22:45:02] the former :) [22:48:26] ah....yeah, thanks. I sometimes need jokes explained to me :-) [22:49:09] explaining jokes makes them funnier! :-) [23:24:21] * Debra hugs legoktm.