[21:59:25] #startmeeting RFC meeting [21:59:25] Meeting started Wed Nov 4 21:59:24 2015 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot. [21:59:25] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [21:59:25] The meeting name has been set to 'rfc_meeting' [21:59:32] o/ [21:59:44] \o [21:59:48] #topic Streamlining Composer usage | Wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ [22:00:34] o/ [22:01:11] #link https://www.mediawiki.org/wiki/Requests_for_comment/Streamlining_Composer_usage [22:03:28] jzerebecki: what do you want to get out of this meeting? [22:04:19] https://www.mediawiki.org/wiki/Requests_for_comment/Streamlining_Composer_usage#Open_questions [22:04:34] Is there agreement to only use git signed tags (or branches where the HEAD is a signed commit)? [22:04:55] For WMF controled libraries only right? [22:05:39] there is also the question of how to use this then in our deployment (and build) process [22:06:08] this explains shortly how this would look like https://www.mediawiki.org/wiki/Requests_for_comment/Streamlining_Composer_usage#Proposed_composer_implementation [22:06:15] bd808: no [22:06:57] o/ [22:07:03] no, meaning that we would not use any library via composer that did not implement signed tags? [22:07:14] jzerebecki: Can you expand what you mean by only using signed commits? As in, the WMF would only deploy repos where a tag/head is signed by someone we trust and can verify the signature? [22:07:33] twentyafterfour: wann attend? and pitch in regarding how this fits into the train? [22:08:07] Can someone elaborate a bit on what this meeting is about? xD [22:08:47] I'm not sure how the signing would work - would we have the DB of the trusted keys for signing? we'd also have to keep up with revocation, etc. [22:09:12] jzerebecki: ok [22:09:17] bd808: yes or we need to use a git fork that adds the signature to each tag we use, which composer AFAIK supports fine [22:09:43] Also, does anyone know what is actually signed when a commit is signed? Chad and I were trying to verify that, and couldn't find an authoritative answer-- is it just the sha1 up to that point? or a more secure hash up until that point? [22:09:47] I'm not excited about forking every library we might use honestly [22:10:25] csteipp: just the sha up to that point afaik [22:11:17] Zavi: this meeting is about https://www.mediawiki.org/wiki/Requests_for_comment/Streamlining_Composer_usage which is basically make the pain in using composer dependencies in mediawiki less so we can make more use of components [22:11:18] bd808: the massive street cred from WMF using your library will inspire devs to sign their tags! :) [22:12:23] is this just an irc meeting or is there a hangout that I'm supposed to be aware of? [22:12:29] csteipp: it's the sha1 but since the theory of git is that sha1 of current commit contains the previous history this means the history is signed too [22:12:31] just IRC [22:12:34] k [22:12:36] csteipp: what would you want elaborated regarding only signed tags/commits? getting to what is exactly signed. [22:12:38] legoktm: thanks [22:13:10] jzerebecki: ah, thanks for explaining. [22:13:36] jzerebecki: What do you mean by "only use". Only deploy repos where the last commit is signed? Signed by whom? That's what I'm trying to understand. [22:14:07] csteipp: what twentyafterfour said. can be seen in the output of git verify-tag -v [22:15:22] https://github.com/wikimedia/composer-merge-plugin uses signed tags if anyone wants to look at some real examples [22:15:28] bd808: it is only to add a signature, think of it more like a mirror on gerrit so it is available. also it would be better to get the authors of libs we use to sign and have a trust path to them. [22:16:26] csteipp: signed by a GPG key in a list of fingerprints we specify [22:16:30] sample git verify-tag -v output: http://paste.tstarling.com/p/pnLuls.html [22:17:08] and yes only deploy repos where the last commit or the tag we use is signed [22:17:18] It makes sense to me that we should mirror the repositories of libs we use but then there will be maintenance overhead keeping them up to date with upstream. [22:17:38] and the primary driver for all of this is to replace mediawiki/vendor.git with a machine managed system under the assumption that this will be less painful than the current gerrit based process [22:17:53] git doesn't seem to have a way to verify *who* is signing the tag [22:18:22] a 32-bit key ID is not highly reassuring to me [22:18:36] but I'm sure we can hack something up [22:18:41] I still am kind of iffy on whether there is really a lot of benefit in the whole solution [22:19:00] maybe you can only verify signatures that are already imported into your keychain? [22:19:07] that would work [22:19:33] the double code review argument seems to be the one that is strongest [22:19:55] I don't see how this fixes problem #4 (usage without dependency) [22:20:35] and #3 remains regardless as well [22:20:44] we could presumably deal with double reviews in other ways [22:20:45] TimStarling: for me the output contains the full fingerprint. anyway that information is contained within the signature itself. [22:21:02] #2 is mostly solved today wiht the exception of breaking API cahnges in updated libraries [22:21:28] it seems to me that we are going from 1 or 2 reviews to 0 or 1 [22:21:39] !info need to check if fingerprint in git verify-tag/commit output depends on gpg settings [22:21:56] use #info jzerebecki [22:22:00] I'm kind of feeling the implication is that we'd have to either fork every library we use (and keep them updated) or do key management for like 20 different projects is not ideal... [22:22:03] #info need to check if fingerprint in git verify-tag/commit output depends on gpg settings [22:22:07] i'm just wondering... could this be implemented on top of composer, instead of inside composer? ideally in a way that would allow us to use the same system with other package management systems to, say, for JavaScript. [22:22:36] ugh, i dropped out for a minute. did i miss a reply? [22:22:39] TimStarling: yes the key needs to be in the keyring and how i'd implement this the key needs to be trusted according to gpg settings. [22:22:50] instead of trusting a WMF staff developer we are trusting all of the upstream developers, right? [22:23:08] DanielK_WMDE__: no reply missed [22:23:14] tnx [22:23:31] which is presumably quite a lot of people [22:23:38] TimStarling: if we want to. or only trust their code after we reviewd and signed it. [22:23:54] if we nither trust them now have reviewed their code, we shouldn't be running that code anyway... [22:24:02] DanielK_WMDE__: the part that is in composer is very small, basically if git-gpg trust this then use else abort. most of it is git and gpg. [22:24:04] DanielK_WMDE__: in node land, we use deploy repositories that are updated using 'npm install', but are code reviewed; those repositories have our actual code as submodules [22:24:41] it's not ideal, but avoids the issue of having no code review [22:24:54] gwicke: that's functionally what we do with the mediawiki/vendor.git repo as well [22:25:16] yeah [22:25:18] just the submodule setup is reversed (vendor a submodule of core) [22:26:01] I'd personally prefer CI to handle this, but then we'd have to at least review hashes of dependencies [22:26:25] this proposal also avoids 0 code review cases. either you trust upstream (as a specific set of people) with review or you sign yourself in the fork [22:26:33] gwicke: CI is also an important use case for us [22:27:08] yeah, I mean review by 0 people with core +2 rights [22:27:29] I wouldn't trust upstream in general, but I might trust the specific hash I reviewed [22:27:38] if we code review some tag in a third party's repo and they have done `git tag --sign` it, then we're good, right? I'm serious, I think devs would be willing to sign tags for WMF. [22:28:28] spagewmf: that's the idea, yes [22:28:31] spagewmf: Only for developers who we trust for some reason [22:28:41] I'm a bit confused about this signing business; if we ask for a tag anyway, why don't we just lock down the git hash & rely on that? [22:28:48] (if I understand the proposal right) [22:28:57] +1 [22:29:02] bd808: "4.3 Generated autoloader needs frequent updates" does not remain as in the sense that it is still frequent but as it is automated it is just always done [22:29:10] csteipp: it doesn't matter if we trust the people who tagged it, if we do a review of that tag, and then sign the tag. [22:29:29] jzerebecki: sure but either way Composer manages that [22:29:36] DanielK_WMDE__: Oh, I thought spagewmf was saying they would sign, and would accept their signature. [22:29:51] But yeah, if we sign it, then yeah, we would trust it [22:29:55] why we need to sign the tag? why not just have a config with the tag? [22:30:08] SMalyshev: or hash [22:30:24] Hash, please. Not tag :) [22:30:24] anyway - i think if we want to be able to use composer safely, we need a way to validate what composer downloads. even if this proposal doesn't solve all problems, we'll need some verification mechanism like this in any case. [22:30:27] gwicke: yeah I mean the hash behind the tag [22:30:34] npm supports fetching an exact hash as well [22:31:02] csteipp: for some specific people, we might also trust their signature, i suppose. [22:31:04] although, we'd need some automation around review [22:31:33] composer schema says "require and require-dev additionally support explicit references (i.e. commit) for dev versions to make sure they are locked to a given state, even when you run update. These only work if you explicitly require a dev version and append the reference with #." [22:31:34] bd808: the reason for "4.4 Usage of libraries without depending on them" is that they can't just add the dependency to composer.json . probably same reason why wikidata has a build. if it is easy to declare the dependency then we can just fix that. [22:32:55] one thing that specifying hashes would fix is compiling binary dependencies for the exact distro used in prod / CI [22:33:18] TimStarling: if you don't want to trust people without mediawiki +2 then don't add those to the keyring nor composere trusted fingerpint list [22:33:21] we don't have that problem with Composer; no binary deps there [22:33:32] we currently use a docker script to make sure that everything is compiled using jessie, but it would be nicer if CI would take care of that [22:33:42] spagewmf: Yes, if every dependency had a hash specified (and +2'ed by a wmf developer), that would give us about the same security as trusting all of those +2er's signatures. [22:34:28] jzerebecki: then how would the tag be signed by a MW+2? would it be signed in a github fork? [22:35:04] gwicke: there are many extensions we maintain ourselves there we want a system that just works for newer releases. we still can use the feature of composer to lock down to a commit has. [22:35:17] csteipp: composer seems to say you can't do "oyejorge/less.php": "1.7.0.9#2eb0c0978d290a1c45346a1955188929cb4e5db7", it has to be a "dev-" tag. Odd limitation. [22:35:17] s/extension/composer components/ [22:36:17] spagewmf: not really that odd, Composer is doing a git clone and needs to know which branch the hash appears in [22:36:23] Really? Sad. [22:36:25] jzerebecki: it seems that we need a solution for third party dependencies in any case, so if we streamline updating the hashes then reasonably easy handling of our own dependencies might just fall out of it as a side-product [22:36:25] specifying explicit git hashes in composer.json is not recommended, and causes weird bugs according to upstream [22:37:03] it also forces you to use git clones for everything which are generally bloated [22:37:23] (c.f. https://github.com/composer/composer/issues/4485) [22:37:30] (can't exclude tests, docs, etc that aren't needed at runtime) [22:37:35] the alternative would be a checksum of a tarball, I guess [22:37:47] but, more work to implement [22:38:24] regarding hashes, there are dependencies of dependencies, keeping track of all those hashes for each mediawiki build gets tedious, this proposal decentralizes that to each dependency [22:38:57] jzerebecki: recursive dependencies are a good point -- are all those fetched via git? [22:39:21] in node land that would be a bit of a pain [22:39:49] csteipp: needing a branch as a starting point is necessary in git itself, otherwise how do you know which end of the commit graph you need to download? [22:40:05] they would be yes, but that is also not always deterministic. A library that we depend on an explicit version of x.y.z may say that it works with x.* of a dependency. [22:40:45] gwicke: re: solution for 3rd party deps: i guess [22:40:55] that means that each composer run that doesn't come with a composer.lock file may get a different version of the indirect requierment [22:41:03] bd808: which would be fine if we trusted the developers of x.* who have signed the version that actually gets installed [22:41:51] so we just need to establish a universal web of trust for all package providers everywhere? [22:42:15] bd808: this proposal would force us to use git clones, too. sadly as the zips are different each invocation and also how to store, obtain and transport the signature. all in principle solvable, but so much work. [22:42:29] bd808: is that before or after we abolish all wars? [22:42:44] bd808: no, but perhaps find out if we can trust the developers of some popular stable libraries... [22:42:47] say, phpunit [22:43:12] so if we can manage to be positive for a second... [22:43:22] I'm still pretty unclear on how any of this is any easier for anyone except possible WMDE [22:43:28] bd808: seems mediawiki/vendor involes 11 top-level packages, I don't know about their own dependencies. I know the devs of oojs-ui can't be trusted :-) [22:43:34] can we agree on any changes that we actually do what jzerebecki to implement? [22:43:36] *packagers [22:43:41] s/what/want/ [22:44:03] bd808: definite yes to your snark. also we already use one, see the unviversal operating system we use... [22:44:13] e.g. do we want signed tag support in composer and will we have use cases for that? [22:44:20] jzerebecki: you mean HURD? [22:44:35] [22:44:41] gwicke: no Debian HURD ;) [22:44:46] TimStarling: i think having a way to validate signatures in composer is a Good Thing, no matter how we end up using it. [22:45:03] jzerebecki: ;) [22:45:03] do we want server-side mediawiki/vendor updates? [22:45:10] i'm pretty sure this ability would come in handy, if not prove essential [22:45:34] as a non-voter: I agree, it's a useful feature. I'm skeptical we will be able to use it effectively in our deployments. [22:45:37] btw. phpunit has signed .phar files, didn't yet check the tags, but I bet I can convice Sebastian to do that if he doesn't yet [22:46:24] bd808: I think this proposal means a n00b developer (me) can have more trust in running `composer update` (or vagrant git-update), but I didn't know to be worried before this meeting ;-) [22:46:32] is there a way to run a private composer registry? [22:46:34] csteipp: I don't think you can claim non-voter status on this [22:47:07] "as a non-member of the arch committee" [22:47:09] gwicke: yes there are a couple FLOSS solutions for that [22:47:36] spagewmf: composer not only installs but also immediately executes it downloads from an arbitrary location on the web via an unsafe connection. real fun :) [22:47:44] I wonder if that could be an alternative that's less invasive for the build process itself [22:47:47] and we would need to do that (run our own registry) to deal with the forks [22:47:48] +code [22:48:28] gwicke: like packagist.wikimedia.org? [22:48:30] gwicke: yes there is, it is called Satis. or you can even run the software used for packagist.org itself [22:48:34] could be an option, yea [22:48:38] (assuming there is a way to gatekeep what's available in that registry & force our code to exclusively use it) [22:48:51] DanielK_WMDE__: but the "industry standard" for install tools is something like curl | sudo sh so ... [22:49:22] * DanielK_WMDE__ shudders [22:49:53] DanielK_WMDE__: shudder all you want, but you know he's not wrong :-) [22:50:11] robla: that's why i shudder ;) [22:51:08] `sudo composer self-update` is basically just that :) [22:51:09] is anyone proposing we stop verifying which code we deploy? then we could just automate vendor updates without doing signatures. [22:51:54] s/stop verifying/never start verifying/ [22:52:00] heh [22:52:13] I don't think anyone was proposing that [22:52:22] #agreed signed tag support in composer would be nice to have [22:52:25] it would definitly be nice not to be force to coordinate patches to two repositories when adding a new dependency to mediawikis composer.json... [22:52:40] I think separating the CI concerns from the production cluster concerns (which is being actively worked on) makes a lot of the complexity of this less useful honestly [22:53:38] bd808: oh what is the proposed solution for production cluster concerns? [22:53:40] bd808: what should we do to improve on that? [22:54:22] jzerebecki: just what we do today, manually reviewed, strongly versioned mediawiki/vendor [22:54:37] but the CI system is moving towards running composer [22:54:59] was there a feature request for better specified hash support in composer? [22:55:07] bd808: who will do that for all the wikidata dependencies? [22:55:13] I could actually see some utility in the CI system preparing a vendor commit for each run [22:55:18] jzerebecki: wikidata [22:55:20] which could then optionally be reviewed & merged [22:55:26] before promoting to prod [22:55:50] bd808: so you are saying not useful as you don't have to do the work it saves? [22:56:10] that kind of workflow would also be nice for node, and it would ensure that the code is tested with an exact version of dependencies [22:56:13] the same as editing does is version bump patches for oosjui [22:56:13] #info rough consensus that everything sucks and our lives will be horrible regardless of which solution is implemented [22:56:39] hehehe [22:56:40] sorry for being a lump TimStarling [22:56:51] so if we can manage to be positive for a second... [22:57:15] any action items, next steps for the RFC document? [22:57:34] i for one would be very happy if we could test against different sets/versions of dependencies via jenkins [22:58:04] DanielK_WMDE__: we probably already can, not sure what you mean [22:58:56] not for the stuff that is in vendor, can we? [22:59:16] bd808: which mailing list is the right one to burrow down and get some initial consensus? [22:59:28] bd808: can you propose a system that actually allows us to stem the change rate of wikidata dependencies? [22:59:46] robla: engineering-l? I'm not sure wikitech-l would care [23:00:02] * robla looks for a small-ish existing mailing list that can proxy as a "working group" [23:00:06] bd808: is engineering-l closed? [23:00:25] yeah, engineering@ is WMF [23:00:32] I thought it was everyone with deploy rights [23:00:52] TimStarling: seems like the big question is whether it's worth doing, how useful it would be. Were there any suggestions for improving the way the verification itself is done? [23:01:19] resourcing should not be our focus here [23:01:42] well then, i'm all for it :) [23:01:46] Is there a full workflow proposed yet? [23:02:01] I have no objections to people making Composer better [23:02:02] DanielK_WMDE__: I think something needs to be done for wmde. Aiui, wikidata is being deployed without review of the composed pieces, and that's an issue. [23:02:03] * DanielK_WMDE__ is running out of battery [23:02:10] yeah, time up [23:02:19] next week we will have two of Daniel's RFCs [23:02:39] T114394 and T114640 [23:03:00] that is "PageLookup service and PageRecord object" and "make Parser::getTargetLanguage aware of multilingual wikis" [23:03:14] is there some next step for this composer RFC? [23:03:25] csteipp: we don't have any external dependencies though. [23:03:52] well, we do in require-dev [23:03:55] DanielK_WMDE__: Right, but your ISP can inject any code it wants into the composed repo :) [23:04:11] there's that [23:04:12] jzerebecki: I think an email asking "no consensus yet...what are the next steps?" to wikitech-l would be good [23:04:41] jzerebecki: maybe write some more about workflows since that seems to be the bit that has everyone depressed [23:04:47] I would personally prefer to focus on automating vendor / deploy repo updates in CI & reviews before promoting to prod, mostly because it seems to provide a more complete and universal solution at a limited cost [23:05:04] if we have a really awesome UI then all the solutions start to look better [23:05:18] #info add more detailed full workflow into rfc [23:05:36] there is no consensus on actually deploying your proposal [23:05:53] jzerebecki: Only speaking for myself, if you lay out a workflow that will work for wikidata, I think that would be a start towards having a concrete plan that we could adapt for production. [23:06:07] but there is consensus that it would be nice to have some more features in composer should you wish to implement them just out of the kindness of your heart [23:06:21] :) [23:06:28] I guess I just feel that this proposal is designed to remove work from preparing wikidata at the expense of everyone else [23:06:55] bd808: how would it make anything worse for everybody else? [23:07:07] maintaining signed forks of upstreams specifially [23:07:08] no, no questions DanielK_WMDE__ [23:07:15] talk somewhere else [23:07:19] bd808: that's about the same as maintaing vendor. [23:07:27] #endmeeting [23:07:28] Meeting ended Wed Nov 4 23:07:27 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [23:07:28] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-11-04-21.59.html [23:07:28] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-11-04-21.59.txt [23:07:28] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-11-04-21.59.wiki [23:07:28] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-11-04-21.59.log.html [23:07:42] DanielK_WMDE__: that can at least be automated [23:07:48] DanielK_WMDE__: move to -tech? [23:07:56] ok [23:16:47] bd808, jzerebecki: regarding this "composer" thing, does mw.org mention or recommend mediawiki/vendor.git at all? To keep my wiki up to date I either run `vagrant git-update` or `git pull --ff-only` and then I somehow learned to run `composer install`. [23:17:51] spagewmf: -> lets discuss in #wikimedia-tech