[12:55:34] Language Engineering office hour starts here in 5 mins [12:56:52] yeehaaaw [13:00:15] And its time [13:00:27] #startmeeting Language Engineering monthly office hour - February 2015 [13:00:27] Meeting started Wed Feb 18 13:00:26 2015 UTC and is due to finish in 60 minutes. The chair is arrbee. Information about MeetBot at http://wiki.debian.org/MeetBot. [13:00:27] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [13:00:27] The meeting name has been set to 'language_engineering_monthly_office_hour___february_2015' [13:00:53] Hello, Welcome to the monthly office hour of the Wikimedia Language Engineering team. [13:01:15] We are late by a week this time, but early on the hour [13:01:36] * arrbee is wondering who all are around [13:02:13] I am Runa, the Outreach coordinator for our team [13:02:19] arrbee: hello! [13:03:06] and along with me are my team mates: kart_ Nikerabbit [13:04:00] First, the really important message: The chat today will be logged and publicly posted. [13:04:44] Our team's last office hour was on January 14 [13:04:55] The logs are at: [13:04:57] #link https://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2015-01-14 [13:05:17] Hi aharoni [13:05:57] And our general team page is: [13:06:04] #link https://www.mediawiki.org/wiki/Wikimedia_Language_engineering [13:06:26] Salom! [13:06:29] We opted for an earlier time this month [13:07:12] If it works well then we would consider sticking with it. So we would love to hear opinions [13:07:53] * arrbee suspects this is prime commute time for India and some parts in the east. [13:08:45] During our last month's office hour we were in the middle of the first deployments of Content Translation [13:09:13] And by the end of that week, we had Content Translation running in 8 Wikipedias [13:09:31] In case you missed the announcement, its right here: [13:09:40] #link http://blog.wikimedia.org/2015/01/20/try-content-translation/ [13:10:18] Its been just over a month and we are still tracking the stats to get a better understanding on how the tool is being used [13:10:32] A quick view of the usage numbers can be seen here: [13:10:44] #link http://language-reportcard.wmflabs.org/ [13:12:22] For numbers related to individual wikis, you can use the Special:CXStats pages [13:13:32] Earlier this month, Content Translaltion was enabled on the Norwegian Nynorsk Wikipedia after we received a request from the community [13:14:09] We also enabled publishing directly into the main namespace for Catalan, Indonesian and Portuguese Wikipedia [13:14:40] Currently we are working on several things that make it easier to get Content Translation in more languages [13:15:24] kart_ and Santhosh (not here right now) are making these changes that will simplify how we choose the source and target languages [13:15:44] We will be making this change available in the beta server sometime later this week [13:16:06] specifically, simplifying it on configuration side, on UI we already have adapted ULS for language selection [13:16:31] Thats right. Its a configuration change [13:16:34] Largely users won't notice it :) [13:16:50] But developers will love it! [13:17:04] Yes, it's an internal change, but the main idea behind it is that it's supposed to make adding new languages simpler for us developers, so the time from the request until the deployment should become shorter. [13:18:24] Presently the cycle extends anything between 15-20 days or more [13:18:34] including testing cycles [13:19:09] kart_: one of the things that we are often asked is about the testing environments we maintain [13:19:35] We also mention it in the graduation plan [13:19:38] #link https://www.mediawiki.org/wiki/Content_translation/Languages [13:19:59] labs, beta-labs and production [13:20:29] kart_: can you perhaps give us an overview about this [13:20:34] ? [13:20:48] Whats the flexibility, restrictions etc. [13:21:07] * arrbee waves to santhosh_ [13:23:37] hello [13:24:02] kart_: hi again, i thought you had dropped off :) [13:24:10] Beta Labs are closer to Production. [13:24:35] So, it makes testing easier for us [13:25:21] arrbee: slow machine, excuse me for typos and lagging (may be network too) [13:25:37] no worries, we have ~35 more mins [13:26:23] Good thing about Beta Labs (not to confuse with with Labs :)) always runs on master code for CX/cxserver [13:26:49] so, we usually detects nice bugs from there. [13:27:29] That's all for Beta. [13:28:17] Labs are experimental setup by different team members from team. We really don't suggest to use for testing unless really needed. [13:29:13] We maintain cx.wmflabs.org instance. [13:29:21] No warranty :) [13:29:25] :) [13:30:05] So cx.wmflabs.org is essentially the most unreliable yet quite possibly be running more interesting stuff at the same time? ;) [13:30:41] yep [13:31:03] arrbee: yes. Specially, since we control it fully, we can do whatever with it (config specially) [13:31:29] Thats right [13:31:57] more unreliable than beta? [13:33:00] Krenair: really. It has mw instance, that we forget to update sometime. [13:33:15] beta at least strives to be stable, cx.wmflabs.org has no kind of guarantees [13:33:46] there are cron setup, which now updates mw, cx, cxserver - but we don't give any assurance that all will work all the time. [13:34:02] though, beta (aka deployment-prep) is much more complicated ecosystem so it is more likely to break for various reasons possibly unrelated to us [13:35:24] So far, complicated setup like cxserver/apertium services has been useful by deploying in Beta first. [13:35:45] oh. and Labs runs on nighty version of Apertium. [13:36:07] May break your favourite language pair anytime! [13:38:23] That doens't mean we don't love Labs ;) [13:38:41] Both has been very useful for us. [13:38:49] arrbee: over to you. [13:38:50] kart_: +1 [13:39:06] kart_: one more for you :) [13:39:35] The other problem that often gets in the way is the 'publishing error' [13:39:45] We know that sometimes its related to Parsoid [13:40:05] kart_: Could you please tell us a bit more about why that happens? [13:42:40] ok.. looks like the network lag is really bad for kart_ [13:42:48] umm.. Nikerabbit santhosh_ ? [13:43:01] well there are various reasons for that [13:43:49] it could be parsoid is having problems with that article, or the parsoid service could be temporarily down, or that the target wiki does not like the page, for example abuse filter may prevent edits [13:45:01] Thanks Nikerabbit [13:45:06] with improved logging we should be better able to monitor this in the future [13:46:23] I think aharoni had noticed something about abuse filter in the Spanish Wikipedia [13:47:46] Yes - the Spanish Wikipedia has a (sensible) abuse filter setting, [13:48:12] which warns people when they try to publish a page in the user space with categories that should be in the article space. [13:48:46] ContentTransation adds categories automatically, so articles with auto-adapted categories couldn't be published. [13:48:52] To fix this we did two things: [13:49:24] 1. Fixed ContentTranslation so that aut-adapted categories would be put inside unless published to the main space. [13:50:19] 2. Contacted the Spanish Wikipedia community and asked to tweak the abuse filter after this fix. [13:50:27] (Thanks to abian for doing this!) [13:51:03] Now there's another issue that affects publishing: [13:51:54] If a CAPTCHA is supposed to appear during publishing, it may be hidden if the page is scrolled all the way to the top, so that publishing appears to be stuck. [13:52:01] This is already fixed in the code, and will be deployed soon. [13:52:17] (Till then, if publishing appears to be stuck, try scrolling down.) [13:53:10] aharoni: This was noticed in the Portuguese Wikipedia, isn't it? [13:53:23] I noticed it in Spanish, but it can happen anywhere. [13:53:29] oh ok [13:54:06] aharoni: should be deploy tomorrow if everything goes smooth :) [13:54:34] kart_: excellent. [13:55:11] We are nearing the end of the hour [13:55:27] And another little comment: [13:55:31] * arrbee looks around for any questions [13:56:01] After enabling direct publishing to the article space in Catalan, Indonesian and Portuguese Wikipedias, we are looking to more languages. [13:56:32] The update in Norwegian Nynorsk seems to be quick and successful - 20 out of 23 articles were published, [13:56:38] and Esperanto is going well, too, [13:56:58] so these two seem like likely candidates for direct main namespace publishing. [13:57:04] There will definitely be more :) [13:58:05] Finally, a happy announcement: [13:58:14] We generally announce this well ahead of time in each Wikipedia [13:58:17] After a break of a few months for technical reasons, [13:58:46] we finally restored the "most commonly used MediaWiki messages" group in translatewiki.net. [13:59:38] Alright! We are down to the last minute now [13:59:42] thanks for that [14:00:13] Thanks everyone for joining in. Especially kart_ aharoni Nikerabbit for the explanations :) [14:00:15] you're welcome, purodha :) [14:00:31] Our next office hour is scheduled to be on March 11. But do please check our announcements [14:00:39] * purodha is happily smiling [14:00:45] Most of our office hours were at 1700 UTC, but if this hour works well we would like to continue with it [14:00:53] #endmeeting [14:00:54] Meeting ended Wed Feb 18 14:00:53 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [14:00:54] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-02-18-13.00.html [14:00:54] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-02-18-13.00.txt [14:00:54] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-02-18-13.00.wiki [14:00:54] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-02-18-13.00.log.html [14:01:19] * arrbee will post the logs soon. Thanks again! [14:01:30] bbye! [15:56:40] The second weekly bug triage meeting for VisualEditor will start in about five minutes. Instructions for joining the WebEx portion of the meeting are at https://www.mediawiki.org/wiki/Talk:VisualEditor/Portal I plan to post links to the bugs being discussed here if you want to follow along in IRC. [16:02:11] The list of tasks to be discussed today was posted at https://phabricator.wikimedia.org/project/board/1015/ [16:02:14] * andre__ waits for WebEx to do something in his browser [16:02:31] I'm handling audio for this meeting, any comments (even positive!) direct to me, please. [16:02:52] andre__: you're the Fedora user, right? [16:02:57] yeah [16:03:51] https://phabricator.wikimedia.org/project/profile/1015/ has the list of objectives. [16:03:51] can someone please share a link to the release criteria? [16:03:54] thanks :) [16:04:03] seems to work... [16:04:15] awjr, https://phabricator.wikimedia.org/project/profile/1015/ [16:04:20] ty [16:06:42] Hi, is there a meeting number or URL to join from Android app? [16:07:01] Webex, I mean. VisualEditor meeting. [16:07:11] Let me see... [16:07:19] qgil_: the meeting number is 803 811 433 [16:07:30] Thanks, marcoil [16:07:42] but maybe you'll need to get to the webpage to get a attendee ID [16:07:57] np :) [16:07:57] No attendee ID needed [16:08:06] ah, great [16:09:12] We're currently discussing previously discussed (now resolved) items [16:09:21] hm, I see participants and "arthur speaking" but no audio. Lemme see... [16:10:05] https://phabricator.wikimedia.org/maniphest/query/K9TVM696Wulv/#R [16:10:06] cndiv: audio gets here really well, fyi [16:10:19] cndiv marcoil same here - ty :) [16:11:01] ah, now it works [16:11:47] Audio over a T1 via webex web/native client is nominal. Crystal clear and I can hear all speakers. [16:12:12] i am on the phone and audio is fine here. [16:13:14] https://phabricator.wikimedia.org/project/sprint/ [16:13:23] The project "§ Fundraising Dash" is not set up for Sprint because it has not been assigned a start date [16:13:23] To do this, go to the Project Edit Details Page [16:13:33] This is due to the Phabricator upgrade an hour ago [16:13:33] whatami, i joined late .. and what is the list of bugs that james is going over right now? [16:13:42] https://phabricator.wikimedia.org/maniphest/query/K9TVM696Wulv/#R [16:13:49] https://phabricator.wikimedia.org/maniphest/query/K9TVM696Wulv/#R - they are the resolved issues from last week [16:13:55] It's bugs that were resolved a while ago. [16:14:12] Or yesterday, in other cases. [16:14:23] We're taking comments on those bugs right now [16:14:41] I mean, if you go to https://phabricator.wikimedia.org/tag/%C2%A7_visualeditor_q3_blockers/ and you click on the burndown link, this is the error message you get [16:15:02] thanks. [16:15:34] The task list is in https://phabricator.wikimedia.org/project/board/1015/ and James F is likely to take them in order from the first column. [16:15:45] ah, https://phabricator.wikimedia.org/project/sprint/view/1015/ works. Sorry for the noise -- the burndown looks a lot better now! [16:15:46] We're going to move onto Nominated Blockers Review [16:16:00] Currently 16 nominated blockers here: https://phabricator.wikimedia.org/project/view/1015/ [16:16:33] https://phabricator.wikimedia.org/T89072 [16:16:34] First item: https://phabricator.wikimedia.org/T89788 [16:16:45] No, it's the 9072 item. [16:16:54] sorry wrong link by me: follow whatami -s link [16:17:27] For the records, both https://phabricator.wikimedia.org/project/sprint/board/1015/ and https://phabricator.wikimedia.org/project/board/1015/ work, but the first link also shows points [16:17:41] (Sorry, we upgraded Phab an hour ago and some links changed and quite some UI) [16:17:46] I think katie has a delay. [16:18:40] https://phabricator.wikimedia.org/T88152 [16:19:10] THis has already been fixed in development, and so it's moving to accepted [16:19:19] https://phabricator.wikimedia.org/T89075 [16:20:05] This has been accepted as it bocks the objective of having properly tested code [16:20:09] https://phabricator.wikimedia.org/T89192 [16:20:42] https://phabricator.wikimedia.org/T89309 [16:21:10] https://phabricator.wikimedia.org/T89543 [16:21:12] Both of these have just been accepted [16:21:23] https://phabricator.wikimedia.org/T89423 [16:21:38] 543 and 423 are being taken as a unit [16:23:19] https://phabricator.wikimedia.org/T89536 and https://phabricator.wikimedia.org/T89735 [16:23:53] Accepting both of these [16:25:04] https://phabricator.wikimedia.org/T89352 [16:27:00] Will be accepting [16:27:04] https://phabricator.wikimedia.org/T52036 [16:28:48] WIll be accepting but will be working on it a bit later [16:28:51] https://phabricator.wikimedia.org/T73085 [16:32:05] Provisionally accepted as a task to review but it may be complicated to implement [16:34:28] The feedback we are receiving on this call is that we should block. I think that it’s a good call to block. :) [16:34:31] https://phabricator.wikimedia.org/T89788 [16:36:03] qgil_, general process so far has been to accept things even if they're at the investigatory stage. we flag in the tasks that there are still open questions. [16:36:38] +1 gnubeard [16:39:02] Maybe it was unclear what I was proposing: keep the task in "Nominated", but only move to "Accepted" tasks that we really believe are blockers and we will do our best to fix -- but OK, no stop energy here. :) [16:40:11] qgil_, I understand your concern. It would be nice if nothing ever needed to leave the "Accepted" column. But I don't think that the devs look at the nomination column, and there isn't a special "maybe" column. [16:40:14] https://phabricator.wikimedia.org/T89072 [16:40:22] https://phabricator.wikimedia.org/T89074 [16:40:30] This is what Kaity's talking about. [16:40:56] Creating a "maybe" or "needs investigation" column is trivial, if that is the problem... [16:41:49] qgil_: I understand. You were proposing to keep it as a nomination. In my experience, it’s better to empty the nomination queue every time you triage. Engineers will not be looking at the nomination queue. They will be looking at the blocker queue. If we want to find an answer that requires an engineer, it should live in the queue where engineers live. [16:43:47] :D [16:43:49] heh [16:44:14] https://phabricator.wikimedia.org/T76398 will be next [16:44:35] Please keep your church bells on mute, folks [16:46:05] I think this is really a question of whether or not we are at a point to accept relatively complex new features at this point. [16:47:38] -1 on a complex opening experience, +1 for simple tweaks to current dialog [16:47:48] I think that’s the right call, James. [16:47:55] +1 [16:47:59] 1 [16:48:02] (Those were clock chimes, not church bells: https://en.wikipedia.org/wiki/Westminster_Quarters ) [16:48:11] https://phabricator.wikimedia.org/T76398 [16:48:17] https://phabricator.wikimedia.org/T76397 [16:48:22] These are the last two. [16:48:32] kaity have you guys done user testing with prototypes or something on https://phabricator.wikimedia.org/T89074 [16:48:33] ? [16:49:09] awjr: yes initial testing with a prototype [16:49:23] looking to do more next week [16:49:31] I might be wrong, but I thought that one of the introductory tours was using (or planning to use?) VisualEditor at a Wikipedia where VisualEditor is already available to everyone by default. [16:49:32] kaity: nice - were you by chance able to get good data on the difference in the editing experiences? [16:49:57] kaity I would love to talk with you about it further [16:50:17] rdaiccherlb: great! [16:51:14] kaity: im asking in part because one of the release criteria currently defined is 'Better task performance by users without editing experience compared with wikitext' - while 'better' is a bit ambiguous at the moment, i think that might be a good rubric by which to frame the feature with the data [16:51:33] That's the end of the list. [16:51:46] We're taking any follow up feedback, questions etc [16:51:53] in the last few minutes of the meeting [16:51:58] Dropping off the call, and I’m really happy with how these are going. I like that we’re listening to direct feedback. [16:52:12] its hard to get really scientific data on it but working with abbey to develop something [16:52:15] I don't manage to enable my audio, sigh. [16:53:09] My question is basically if folks went through the most popular complaints in past RfC's on en.wp and such for disabling VE and made sure that those complaints are on the blockers list. [16:53:17] (I think I basically stole this qustion from Quim) [16:53:38] awjr: initial testing was showing users who had never edited VE and asking them how confident they could do a few tasks [16:53:45] thank you James_F|Away :) [16:53:47] Thank you everyone [16:53:49] thanks James_F|Away :) [16:53:52] and everyone! [16:53:57] thanks [16:53:59] then showed them the tour and asked them to rate their confidence on those same tasks [16:54:02] andre__: Will ask about your question. [16:54:16] Thank you! [16:54:32] kaity, I think the tour is a good idea but I'm worried about the dev complexity at this time. and, we need to account for different user groups getting this dialog. [16:54:32] Hmm. Last time I could at least enable my microphone. This time even that didn't work... [16:54:58] the most obvious issue that I see with the first open experience right now is that the welcome dialog has way too much text and doesn't make it easy to switch back to wikitext if that's what you want to do [16:55:00] Eloquence: thats good feedback, thanks! [16:55:25] I think it should address the 2 edit buttons also [16:55:43] This is our new, easier way to edit. It's still in beta, which means you might find parts of the page you can't edit, or encounter issues that need to be fixed. We encourage you to review your changes, and we welcome reports about any issues you might encounter in using VisualEditor (click the "Help" button to submit feedback). You can switch to the wikitext editor at any time by clicking on the "Edit" tab, keeping any changes you have [16:55:43] made. [16:55:55] that's a lot of text :) [16:56:34] i bet especially when it's translated to something like german :p [16:56:40] yes we can improve that for sure [16:56:41] and the "switch to the wikitext editor" is textual instruction and not an actual part of the dialog [21:00:37] #startmeeting RFC meeting [21:00:38] Meeting started Wed Feb 18 21:00:37 2015 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot. [21:00:38] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [21:00:38] The meeting name has been set to 'rfc_meeting' [21:01:02] #topic Improving extension management | RFC meeting | Wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ [21:01:15] #link https://www.mediawiki.org/wiki/RFC_metadata [21:01:23] o/ [21:01:28] Oh, whoops, sorry Tim. [21:01:29] * James_F fixes. [21:02:07] o/ [21:03:30] https://www.mediawiki.org/wiki/Requests_for_comment/Improving_extension_management [21:03:50] * hexmode waves [21:03:59] legoktm: do you want to start with a quick overview? [21:04:13] sure [21:05:05] So right now extension management is a mess and usually requires a bunch of manual work to do things which could be automated [21:05:14] plus it requires editing PHP files :( [21:05:35] I've split the RfC into 3 steps: [21:05:42] 1. Allowing extensions to specify their compatibility and dependencies in a standard and machine-readable method. [21:05:54] 2. Providing an API to expose said metadata (along with other stuff in extension.json) [21:06:00] 3. Building a web interface that allows system administrators to search for extensions, install them, and enable/disable them. [21:06:02] hey [21:06:34] legoktm: about 2.: where does this API live? is that something central, on mediawiki.org [21:06:36] ? [21:06:49] So far, I think #1 is the most controversial so I'd like to mainly discuss that [21:07:18] (i think #3 is also rather controversial) [21:07:19] DanielK_WMDE_: yup. I'd like to combine it with the extension distributor setup [21:07:22] legoktm: if I understand right, enable/disable works per-wiki on a wiki farm, right? [21:07:49] DanielK_WMDE_: I don't think 3 is controversial. Who contests it? [21:07:50] legoktm: that makes sense, thanks. would be good to clarify this in the rfc, wasn't obvious to me on first glance [21:08:03] would there be a way to express "load x first if it's available, otherwise keep going"? we have a few cases where that could be useful [21:08:06] DanielK_WMDE_: As I understand it, assumine #1 and #2 are approved, #3 could even be implemented as its own extension, and not necessary to core. [21:08:22] what is controversial about 1? [21:08:24] Krenair: optional dependencies for ordering purposes? [21:08:28] yes [21:08:32] hexmode: yes, and you'd be able to do a global enable or disable. This would be combined with the configuration database RfC [21:08:36] yeah that would be nice [21:08:40] TimStarling: whether to do it in extension.json or use composer [21:08:45] hexmode: installing files and registering runnable code via a web interface? a lot of people contest that idea. I'm ok with it if it's optional, but there should be a well supported non-wen way [21:09:22] DanielK_WMDE_: I was assuming it was optional, so I guess I understand [21:09:22] DanielK_WMDE_: In the RfC I said it would be optional and there would be a maint script also for CLI. We already let people run the updater from the web so this would be comparable IMO [21:09:36] legoktm: to me it seems pretty clear that extensions are not libraries, and vanilla composer is pretty useless for managing extensions. it gets the dependencies backwards [21:10:01] DanielK_WMDE_: i'm not clear on how the dependencies are calculated backwards? [21:10:01] DanielK_WMDE_: I agree with you :) but I think other people don't :P [21:10:08] yeah, DanielK_WMDE_ has told me a couple of times that composer is not the way to go [21:10:09] vanilla composer isn't going to work, true. [21:10:10] legoktm, hexmode: i agree re 3 - if it's optional, and cli is well-maintained, i see no problem [21:10:22] also this does appear to be pretty closely integrated with composer [21:10:26] DanielK_WMDE_: I have come around to that feeling myself as well after starting in the "why not just composer?" camp [21:10:27] I do think we should be using Composer for what it is meant for: dependency management. It just needs to be supplemented by our own system that works with it. [21:10:33] so we can almost claim buzzword-compliance, right? [21:11:13] bd808: :D but your "local dependencies" thing would still be useful, i think. we *should* use composer. as one component in the extension management process. [21:11:17] So is anyone actually arguing that "we should just add a composer.json file to every extension" at this point? [21:11:26] Because if not, what's left to discuss? [21:11:27] I poked around with composer internals, and I think we can re-use their dependency resolver system by interacting with the code directly and using it as a library instead of a CLI tool [21:11:48] But I think composer could be extended so that we could still use it. MW devs have already extended composer a bit. [21:11:50] James_F: If an extension has library dependencies, i.e., non-other-extension dependencies, it should have a composer.json. [21:12:04] parent5446: agreed [21:12:13] bd808: I think we should extend composer a bit, but i don't think that's sufficient. we should treat composer as a tool and build on top and around it. not use it as the entry point. [21:12:15] parent5446: Sure, but not as a "this is how you install the extension" concept? [21:12:45] James_F: perhaps not to install, but it should certainly encapsulate the inter-extension dependencies i think? [21:12:46] James_F: composer is already a requirement for some extensions, that is part of what is left. [21:13:05] Indeed. The idea is that our built-in extension system will see the extension's composer.json and automatically use it (using the composer-merge plugin or something similar) [21:13:16] we *could* discuss *how* we want to use composer. we can't just have every dependency use it independently. [21:13:18] hexmode: s/some/two/ as far as I know [21:13:20] that would lead to conflicts [21:13:21] That way if two extensions, while unrelated, use the same library, there is no duplication [21:13:41] bd808: skins use it, so more than 2 [21:14:02] anyone want to address ebernhardson's question directly? [21:14:19] almost all the SMW extensions use it. SMW is not a tiny use case. [21:14:50] DanielK_WMDE_: I'm imagining something similar to the composer-merge-plugin. You say you want to enable Extension:Foo and Extension:Bar, MW using composer process thoses extensions dependencies, resolves them and figures out which versions of extensions and libraries to download and install [21:14:56] SMW and Wikidata were my two broadly construed [21:15:19] I think Translate is using it as well [21:15:28] legoktm: yes, that's what i was thinking of. is this in the proposal yet? [21:15:29] bd808: so we agree :) but I wouldn't say "only" two ;) [21:15:44] DanielK_WMDE_: yes, but I might have not made that totally obvious [21:15:54] ebernhardson: using composer to say that extension A needs extension B sounds reasonable to me [21:15:57] MLEB is installable via composer, but it is not *required8 (we don't run composer on the cluster) [21:16:05] legoktm: and i didn't read it thouroghly again, my appologies for that :) [21:16:09] but even better A and B should be refactored to use a common library [21:16:24] bd808: well i mean like how the Thanks extension doesn't do anything unless you install Echo [21:16:24] legoktm: So in the magical future, every extension will have extension.json and also some will have composer.json? Or we'll put the composer.json stuff inside extension.json? [21:16:32] legoktm: Or do you not know at this point? [21:16:48] legoktm: could you outline what the process for installing would be, step by step? [21:17:02] legoktm: as long as the work SMW and MLEB put into using composer was not in vain, I'm fine with every solution [21:17:07] James_F: I've proposed moving info from extension.json into composer.json [21:17:13] composer.json and extension.json have to be separate (or everything has to get crammed into compsoer.json) [21:17:14] hexmode: Eww. [21:17:16] +1 to DanielK_WMDE_ [21:17:26] James_F: I was thinking that every extension has extension.json, some might have composer.json. We could put everything in extension.json if we wanted I suppose. [21:17:32] cramming things into composer.json just because it's there seems icky [21:17:39] bd808: +1. [21:17:56] I still maintain that having autoloader information in composer.json is preferred, but yeah extension.json would still be separate, containing information specific for our extension installation system. [21:18:01] As long as there is no duplicaton, I guess I'm fine with that. [21:18:09] that is just shifting more and more back to ExtensionName.php where all the things happen [21:18:51] hexmode: I think duplication would be minimal [21:19:12] name, license, authors, ... some bits like that [21:19:29] So why not grab that from comoser.json [21:19:30] ? [21:19:33] DanielK_WMDE_: Go to web interface and say you want to install E:Foo, MW through composer hits API and determines E:Foo depends on library foo/bar, does composer magic to figure out what versions are needed, downloads foo/bar and E:Foo, rebuilds vendor/autoload.php for foo/bar, updates database to enable E:Foo. [21:19:33] Technically, name and license aren't really required in composer unless you are putting on packagist [21:19:39] iirc [21:19:43] hexmode: Because almost all extensions will never have a composer.json file. [21:19:57] hexmode: Adding complexity for the ultra-marginal case seems illogical. [21:20:04] * hexmode revisits "cram them together" [21:20:20] "ultra-marginal" for WMF maybe, but not other users. [21:20:36] Honestly, how often does an extension name change? Pretty much never. [21:20:39] parent5446: true, but we have sort of standardized on a jenkins test for all composer.json files that are in gerritt that wants a "valid" composer.json that would be ok for publication [21:20:51] I'm OK with adding optional composer hinting to extension.json / skin.json, but calling something that has nothing to do with Composer "composer.json" doesn't seem helpful. [21:21:09] I'm interested in security and privilege separation [21:21:23] TimStarling: A better topic than our bikeshedding. :-) [21:21:28] James_F: composer hinting == look in composer if needed, but not required? [21:21:57] so we have two separate dependency issues: other extensions, and PHP libs installed via composer. yes? [21:22:06] I could get behind that, James_F [21:22:11] TimStarling: for the web interface? I think we'd do something similar to the installer with $wgUpgradeKey [21:22:13] brion: yes [21:22:14] hexmode: can you give concrete examples of data that you fear duplication would burdensome? [21:22:19] for updating web apps there are maybe 3 different security models [21:22:47] 1. have scripts owned by the web user. Have the webserver updated its own code [21:23:22] 2. provide FTP/SSH password and let the webserver connect to itself and write script files as a different user. Discard login information after update. [21:24:02] 3. provide instructions to the user as to what commands to run as the privileged user, what tarballs to download, where to unpack them [21:24:35] bd808: "burdensome" is subjective. If they get out of sync, who cares? TBH, I've been busy supporting slightly older systems so I haven't had a chance to dig into extension.json too much. But the people who have been bitten by it sound like a real issue. [21:24:37] I've mainly been thinking about supporting #1 and #3, but #2 is also interesting as a compromise between the two [21:24:54] * hexmode goes and looks at extension.json schema [21:25:07] I'm not really in favour of #1 [21:25:14] 2 sounds scary for sending ssh passwords to the web server hand handling them in php [21:25:19] hexmode: but composer.json would bit the same people. The "who moved my cheese" thread was about moving things from php to a config file [21:25:34] it causes escalation from local file write vulnerabilities to persistent complete compromise [21:25:40] hexmode: right now we're only duplicating everything from ExtName.php because we want to migrate all of WMF production at once, it's not intended to be a long term thing unless people want to support older MW versions [21:25:44] an more specifically duping the php config in json as a migration step [21:25:44] 2 is much less scary to me [21:25:46] TimStarling: i'd let people do #1 if they really want to, but default to / fall back to #3 otherwise. [21:25:52] #2 sounds interestingf but hard to get right [21:26:26] #1 is madness and we shouldn't do anything to encourage it [21:26:29] an example escalation in #1 would be path traversal vulnerability, ../../ in a cache filename etc. [21:26:37] legoktm: "not long term" -- duplication between php and json, right? [21:26:37] OTOH, #1 would let us trigger remote auto-updates for installs that opt-in whenever a new security release is issued [21:26:41] hexmode: yes [21:26:58] or to put it differently: i'd start with #3 which is mostly secure by default. and once that works, look into implementing #2 or #1. [21:27:14] DanielK_WMDE_: +1 [21:27:35] i'd basically put off that decision, because it's separate from the actual installation/registration mechanism [21:27:39] I'd even add custom tooling for #3 before moving on to the idea of #2 [21:27:40] Agreed with TimStarling and bd808 on not allowing #1 [21:28:15] legoktm: btw, how about extension configuration? [21:28:18] Wordpress does #2, and while it is not the best solution, it is safer than 1. [21:28:35]