[21:22:25] bd808: hiya! [21:22:36] hey sumanah [21:22:49] bd808: no comments yet on https://www.mediawiki.org/wiki/Talk:Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster :/ [21:23:03] bd808: this will change sooooooooon [21:23:22] I told you that I had a magic power for this [21:23:31] :/ [21:23:36] +10 cloak of RFC invisibility [21:23:39] I want to overcome your magic with mine [21:23:47] like Harry Potter duelling with Voldemort [21:24:09] That pheonix gave one other feather... [21:24:16] I will cause your browser to emit, in reverse chronological order, all the RfCs you've ever written [21:24:32] My wife booked us for the HP tour in London. She's super excited [21:24:36] oh wow! [21:24:44] my niece & nephew went on that and loved it [21:25:01] * sumanah makes super obsolete joke about The HP Way [21:25:09] * bd808 gets it [21:25:26] bd808: most important question: what music will you be playing during the RfC meeting [21:25:43] I'm on shuffle. I never know what's coming at me next [21:25:50] in music and in life [21:26:07] true enough [21:26:29] bd808: second-most important question: you have a little "what I want out of today" speech you can make? you can probably copy and paste what you said onlist [21:27:14] yeah I'll come up with something. bio break and fresh glass of water first [21:27:47] no prob [21:30:00] oh speaking of obsolete things: https://en.wikipedia.org/wiki/Category:Internet_properties_disestablished_in_2013 [21:31:16] * bd808 still misses AltaVista a bit [21:44:44] sumanah: I've got a starter script -- http://paste.debian.net/plain/111320 [21:44:55] :) [21:45:06] sounds good! [21:55:32] talking about https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster here in 5 min [22:00:03] ok! hi folks! [22:00:08] #startmeeting Composer-managed libraries on WMF cluster | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours [22:00:08] Meeting started Wed Jul 23 22:00:08 2014 UTC and is due to finish in 60 minutes. The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot. [22:00:09] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [22:00:09] The meeting name has been set to 'composer_managed_libraries_on_wmf_cluster___channel_is_logged_and_publicly_posted__do_not_remove_this_note___https___meta_wikimedia_org_wiki_irc_office_hours' [22:00:12] #chair sumanah brion TimStarling [22:00:13] Current chairs: TimStarling brion sumanah [22:00:20] * brion waves [22:00:22] #chair sumanah brion TimStarling mark [22:00:22] Current chairs: TimStarling brion mark sumanah [22:00:27] #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-23 [22:00:31] * bd808 waves to the crowd [22:00:33] First off - administrative note. [22:00:38] #topic Future of RFC/architecture stuff [22:00:40] * aude wave [22:00:41] * awjr waves [22:00:43] * legoktm waves [22:00:49] * mwalker waves [22:00:50] #info I'm going on vacation in mid-August, and am going to be stepping away from running these meetings and from coordinating the process overall - including after I come back from vacation. I'm gonna work with Rachel Farrand, Quim Gil, and the architecture committee to make sure stuff still happens. [22:01:07] #info Next week we'll be talking about Recent Changes & rcstream, https://www.mediawiki.org/wiki/Requests_for_comment/Publishing_the_RecentChanges_feed , PubSubHubbub vs WebSockets vs rcstream, and so on. [22:01:10] sumanah: enjoy! :) [22:01:13] Thanks :) [22:01:22] #info thanks to sumanah for lots of great work on the process so far! [22:01:33] +1 [22:01:36] so legoktm Krinkle|detached Lydia_WMDE you want to maybe reply on wikitech-l to help make that discussion happen :) [22:01:44] * legoktm nods [22:01:59] Thanks brion & bd808 :) [22:02:06] ok, now on with today's programme [22:02:06] #topic Composer managed libraries for use on WMF cluster [22:02:15] #info Today we're talking about revamping how we manage some libraries on the WMF cluster, and using Composer. [22:02:20] #link https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster [22:02:31] Bryan bd808, perhaps you would like to start by giving an overview of how the RFC has evolved in the last 2 weeks? [22:02:37] or of what you'd like today [22:02:39] #link http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077619.html - Bryan Davis's note to wikitech-l [22:02:47] Thanks sumanah [22:02:50] I'm looking for a general consensus that managing 3rd party libraries with Composer is sane. The RFP is really about a procedure for managing those dependencies in a way that will work for the WMF beta and prod clusters, but if Composer is a crap idea then it's moot. [22:03:02] On the proposed procedures, I'd like input on possible use cases that aren't covered. [22:03:08] I'd also like help filling in sketchy bits like how we will track upstream security issues and ensure that they are resolved. [22:03:26] i feel like it’s a more general solution than pulling things in with git submodules / svn externals like the olden days [22:03:37] generally looks sane to me [22:04:00] +1 [22:04:08] it mostly seems pretty similar to what I was supporting on wikitech-l, except for one minor detail [22:04:18] I hoped aude would like it. I think it's a lot like what wikidata is doing [22:04:19] bd808: my main concern is whether this is going to be wmf-deployment-specific or whether we’d expect similar usage outside of wmf? [22:04:20] aude or I can give an overview of how the Wikidata team uses composer for WMF production [22:04:22] which is mediawiki/core/vendor versus mediawiki/vendor for the gerrit project name [22:04:48] brion: the repository is set up so if people don't want to use composer, they can clone our vendor repo and use it in place [22:05:07] TimStarling: I think changing the repo name is trivial. I can ask to have that done. [22:05:17] nice [22:05:19] so it's wmf-deployment specific, but can easily be reused [22:05:35] I think that for developers, it is nice to be able to check out the whole gerrit hierarchy even if you want to use composer directly [22:05:45] bd808: Sorry if this is already expressed in the RFC. Are you talking about having one repository per composer-managed extension? Or one big repository that contains the current state of all extensions and dependencies? [22:06:01] even if we recommend that all developers use the submodules, there'll still be a need for testing composer [22:06:08] andrewbogott: One big pile of the things that are blessed for prod deploy [22:06:28] what’s the alternative for getting those deps? manual installation? or git submodules? [22:06:31] So if I need special extensions for wikitech, this doesn't help me, huh? [22:06:32] andrewbogott: But the upstream development will be from other repos [22:06:49] Or would the wikitech extensions be rolled into the repo but just not pulled in for normal prod? [22:07:07] andrewbogott: You are asking because of SMW? [22:07:11] for third parties, should update.php also update composer dependencies? [22:07:47] * aude also notes that vagrant now supports composer [22:07:51] andrewbogott: it's not for extensions, it's for core dependencies [22:08:04] and think there is progress for jenkins, but don't know the details [22:08:13] yes, SMW [22:08:19] (the vagrant support goes a long way for us providing a wikibase role) [22:08:23] brion: individual submodules could be done or something like PEAR where the code lands in the search path somehow as an alternative. [22:08:23] composer* [22:08:32] extensions can implement an analogous mechanism [22:08:48] brion: But I think both of those options have issues. Versioning across branches is one. [22:08:51] *nod* i’ve just had bad experiences with hidden dependencies in extensions requiring, say, a PEAR installation in include_path [22:09:41] Uhoh, I maybe don't understand the difference between 'extension' and 'core dependency'. [22:09:52] TimStarling: andrewbogott is asking because installation via Composer has become the (only?) way to install SMW [22:10:06] oh! [22:10:10] I didn't realize that [22:10:12] same goes for Wikibase, btw [22:10:55] wikibase has mostly moved to github, which seems like something that we maybe should have discussed more broadly at the time [22:10:58] Of course I could follow your pattern but manage my own repo that pulls in wikitech dependencies. [22:11:20] imo extensions should have their own dependency trees probably? hmm [22:11:32] TimStarling: That's not the point... even when more stuff was in gerrit, we started to not really support non-Composer installs [22:11:32] ugh that gets ugly fast though with shared deps [22:11:47] andrewbogott: We should really talk more about the needs for wikitech. Maybe start an email thread somewhere on it. [22:11:51] if the number of components is to high, manually managing them isn't an option anymore [22:12:08] [15:10] TimStarling wikibase has mostly moved to github, which seems like something that we maybe should have discussed more broadly at the time [22:12:15] bd808: It's the same problem as the one you're talking about, right? Just with a different set of dependencies? [22:12:19] Ahm, don't we have like a /policy/ of how all deployed code is *required* to be mastered in Gerrit? [22:12:33] andrewbogott: yes. I think it is. [22:12:37] RoanKattouw: No [22:12:40] RoanKattouw: well, we deploy wikibase, so I'm not sure how that can be [22:12:45] (Not exactly on-topic for Composer, I realize) [22:12:52] RoanKattouw: we make a build that is on gerrit [22:12:53] let's get back to composer, yeah [22:13:03] RoanKattouw: feel free to bring that up onlist later? [22:13:08] point is: We use to composer to pull everything together [22:13:12] good question though imo [22:13:16] and then we push that to gerrit and deploy it [22:13:19] in a nutshell [22:13:25] We keep the code that goes into prod in gerrit. But how it gets there is an open question I think. [22:13:28] the build is deployed, similar to what bd808 proposes for the core libraries [22:13:34] MatmaRex: AaronSchulz http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140723.txt in case you want to catch up. [22:13:38] * aude doesn't want to derail... [22:13:40] And I'm asking to bring in libraries that are "not invented here" [22:13:41] *nod* it all makes some kind of sense :) [22:14:22] does anyone other than me have an opinion on mediawiki/vendor versus mediawiki/core/vendor? [22:14:42] i have no opinion on the repository naming :) [22:15:04] * aude no opinion [22:15:11] What is the vendor repo? [22:15:15] because IIRC there were no opinions given on wikitech-l, and no comments here other than bd808 "easy enough to implement" [22:15:16] The RFC page doesn't seem to explain this [22:15:19] if it only has core dependencies, I think "mediawiki/core/vendor" makes more sense, but I don't really care tbh [22:15:21] I am mildly in favor of concision. Like, super mildly [22:15:30] as in, less typing [22:15:37] RoanKattouw: that’s the git repo that contains the stuff that composer checked out, basically [22:15:41] RoanKattouw: at teh moment, just stuff for logging [22:15:43] I see it like legoktm [22:15:44] monolog, psr [22:16:01] Oh it's where all the 3rd party stuff we import lives? [22:16:07] https://git.wikimedia.org/summary/mediawiki%2Fcore%2Fvendor [22:16:10] right [22:16:14] https://git.wikimedia.org/tree/mediawiki%2Fcore%2Fvendor [22:16:15] Or rather, the composer infra for pulling it in [22:16:21] then in production, we only have to check out that repo instead of running composer on every node or something [22:16:32] brion: yes [22:16:33] brion: exactly [22:16:36] RoanKattouw: Yes. "vendor" is the name Composer uses by default for libraries it manages. [22:16:47] composer really isn't a thing for deployment [22:16:58] just to package / assemble the dependencies [22:17:18] not a dpeloyment tool [22:17:19] Yeah, we use it to pull stuff together, then we "freeze" that into a git repo which can be deployed just like any other [22:17:21] bah [22:17:32] brion: We are actually already doing it (deploying the vendor directory to prod) [22:17:34] aude, you can always dsh composer update :P [22:17:41] excellent [22:18:26] It is branched in the make-branch script and added as a submodule to the wmf branch just like an extension or skin [22:19:30] Yeah [22:19:34] so any open issues remaining with this rfc? [22:19:49] I'd mildly favor mw/core/vendor over mw/vendor because of the way it's submodule-d in, but I have no strong opinion either [22:20:00] we should talk about making sure we're up to date [22:20:02] TimStarling: brion: can you summarize with #info or #agreed on what we've decided here? [22:20:30] #agreed general agreement that this approach makes much more sense than running composer on deployment nodes :) [22:20:36] RoanKattouw: fair point [22:20:37] mwalker: Yes. Especially for security fixes [22:20:51] #info some uncertainty about mediawiki/vendor vs mediawiki/core/vendor [22:21:06] is there definitely a submodule in master? the RFC was unclear about this [22:21:31] it said that there is a submodule in the deployment branch [22:21:32] there's no submodule in master, just in wmf deployment branches [22:21:36] The submodule is not in master. We only add it to the 1.24wmfX branches [22:21:41] * YuviPanda wonders about git subtrees instead of submodules [22:22:12] right [22:22:20] Adding the submodule to master would really be stepping on the toes of anyone using Composer independently. [22:22:20] I take it back then, unfair point ;) [22:22:41] TimStarling: I'll do the leg work to rename. It's easy enough. [22:22:49] thanks bd808 [22:23:20] #action bd808 to get repo renamed from mw/core/vendor to mw/vendor [22:23:39] sumanah: was that how I log that sort of thing? [22:23:51] yep! [22:24:20] So ... tracking security updates? [22:24:32] How do we do that in general? [22:24:51] apt-get update? :) [22:24:55] Basically each 3rd party lib we add could have vulnerabilities [22:25:15] is that core’s problem? or ops’ problem? to watch for those… [22:25:18] we often have bugs in our bugzilla that are proxies for upstream bugs [22:25:50] brion: csteipp wanted just to ensure that it wasn't his problem :) [22:25:56] :D [22:26:26] One idea he and I talked about was that each library should have a "sponsor" [22:26:38] *nod* [22:26:44] But keeping up with this over time is obviously problematic [22:26:55] We already have extensions that want for maintainers [22:27:02] We actually have a keyword for bugs in our bugzilla which are just tracking upstream [22:27:09] Except people keep using it for things that haven't been reported upstream yet [22:27:13] we are talking about the work of watching vendor release notes for security updates? [22:27:14] And the platform core team can only take on so much [22:27:49] TimStarling: I think so. And at least poking folks to update our copies [22:28:21] there are deeper issues too -- because dependencies of dependencies might have updates that are not reflected in the release notes; and what do we do about dependencies that go unmaintained [22:28:28] I don't think it should be spread over too many people [22:29:15] mwalker: True enough. We would need to track all the way down for things we are using. [22:29:16] we'll lose track and updates will fall through the cracks [22:29:28] unless there is quite a heavyweight process on top of it [22:29:47] I mean, I'm sure things fall through the cracks at the moment [22:29:53] Oh hi lilatretikov! [22:30:08] there are automated services for nodejs that will look at your repo and see if recursively everything is up to date -- I just searched and didn't find anything for composer but my googlefoo has been failing me today [22:30:11] lilatretikov: we're discussing https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster [22:30:33] https://security.sensiolabs.org/check or such might help? [22:30:42] there must be a way to help automate this [22:30:50] sumanah: We're just getting her set up with IRC now :) [22:30:54] aude: Nice! [22:30:57] nice :) [22:31:20] So… surely there's a top-level composer call that will just update every dependency and every dependency's dependency, etc. right? [22:31:26] aude: do you know if they compile that vulnerability list themselves? or is there a standard way to report security vulnerabilities? [22:31:29] Which would make us one big patch for our repo with every change? [22:31:31] Maybe we could have a jenkins job to check that daily/weekly and email "someone" [22:31:41] TimStarling: i would have to look into this more to say for sure [22:31:46] andrewbogott, yep -- that exists [22:31:57] andrewbogott: Yes. -- https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster#Adding.2Fupdating_a_library [22:32:10] oh, it comes from https://github.com/sensiolabs/security-advisories [22:32:13] So, is the current proposal to /not/ do that, but instead only selectively update libraries as needed? [22:32:22] which they apparently compile [22:32:26] nice that it is public though [22:32:33] I think the problem is that we dont trust it -- but it's gotten to the point in the pdf renderer that I basically just have to trust it (I have 1M lines of code from dependencies, I cant look at them all) [22:32:35] so we can build our own tool on top of that git repository [22:32:57] should be possible [22:33:36] mwalker: if you can't review then you should sandbox [22:34:14] *nods* -- as much as possible, I have an apparmor profile that's a stand in for that [22:35:01] andrewbogott: The proposal is just about how to prepare the "big commit" mostly [22:35:33] We don't want to do it daily/weekly, we just want to pull in new and/or updated things when we need them. [22:35:43] * andrewbogott nods [22:36:06] And we want to version "this lib went with this deploy branch" [22:36:34] If when we do need an update we're pulling in everything, then... [22:36:55] well, that sounds to me like an argument for frequent updates. Because otherwise a serious emergency bug will force us to review an epic patch in a rush [22:37:11] Composer supports "pinning" verisons so we can know what we are getting. [22:37:23] commit message: "Fix bug and also pull in 4 month's worth of associated updates in a million packages" [22:37:23] ah that should help [22:38:02] regular updates seems good to me [22:38:07] I'm currently being very specific -- https://github.com/wikimedia/mediawiki-core-vendor/blob/master/composer.json [22:38:22] And I would suggest we continue that pattern [22:38:26] there is also the potential for some incompatibility between some version of a library and core [22:38:53] easier to deal with / minimize that when we make smaller, more frequent updates [22:38:53] Yes. It's the class "DLL hell" problem [22:39:06] *classic [22:39:49] But nobody wants to update a library like monolog every 2-4 weeks. Maybe quarterly? [22:39:56] maybe [22:40:48] the more stable the library is, less often is ok [22:40:50] When we use Ubuntu LTS releases we basically commit to only getting major updates once every 2 years [22:40:59] and hope we use only nice stable things, preferably [22:41:19] The less stable the library is the less I want to see it used for core, but that's another debate. [22:41:32] yep [22:42:00] exception might be our own stuff (e.g. the oojs stuff) [22:42:02] * bd808 excludes his own code from the stability debate :) [22:42:12] which is updated all the time [22:42:19] If we're committed to blind-merging these big composer patches then this isn't a big deal. It's only the prospect of having to actually read them that would argue for frequent tiny updates. [22:42:21] outside of composer [22:42:41] Well, and also general CI values :) [22:42:42] we could queue up updates in gerrit automatically as if it were pushed. it might be healthy to have gerrit implicitly nagging us to upgrade as often as the upstream changes. [22:43:06] ^ +1 to robla [22:43:51] * aude looks at the diff for wikidata builds but also the tests are quite important [22:43:53] I'd be ok with that within a major version. API bumps are a different story [22:44:03] particular attention to the composer lock file [22:44:22] We don't deploy the latest jquery regularly do we? [22:44:23] bd808, does composor support semantic versioning? (e.g. I want to only update within the 4.x series?) [22:44:32] mwalker: Yes it demands it [22:45:11] bd808: nope, we don't, though we used to, and I was the one of the main detractors of frequent updates there...so, I'll fess up to being wildly inconsistent there :-) [22:45:37] I wouldn't suggest that we automatically +2 upstream updates [22:45:53] mwalker: Some discussion of how composer versioning works at -- https://igor.io/2013/01/07/composer-versioning.html [22:46:09] #link https://igor.io/2013/01/07/composer-versioning.html Some discussion of how composer versioning [22:46:22] robla: We just need to write a security review bot :) [22:46:58] oh dear [22:47:09] Chris is away for ONE DAY and you start replacing him [22:47:12] If we don't automatically +2 upstream versions, then I almost don't understand how this will work. What happens if we review upstream changes and find them wanting? Do we need a system to declare 'upstream freeze' and 'upstream unfreeze' [22:47:54] andrewbogott: it's as much also about ensuring core is updated to be compatible with the changes [22:47:56] andrewbogott: We could fork or freeze I think -- https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster#Hotfix_for_an_external_library [22:47:58] andrewbogott: basically....we discuss what to do on a case-by-case basis [22:48:11] per robla about changes we don't want [22:48:38] I just hope we have some plan to avoid accidentally forking forever [22:48:50] #info I just hope we have some plan to avoid accidentally forking forever [22:48:55] (I think that is important) [22:48:56] andrewbogott: i think ‘get tired of forking and fix it’ may handle that [22:49:09] but it’s all about coefficient of static friction :) [22:49:13] or contribute upstream :) [22:49:26] to fix wahtever issue [22:49:40] if the upstream goes silly, we have a problem no matter what mechanism we choose [22:49:45] andrewbogott: You can sort of look at this as us being our own Debian/Ubuntu to gate php libraries into our environment. [22:49:46] ok, so that's a reasonable answer. If the upstream is sketchy then we fork + scramble to to resolve the upstream problem so we can unfork. [22:50:12] you know we're finally unforking librsvg for trusty [22:50:14] sometimes it's ok to live with the fork/freeze. it's kind of what we're doing with Lua, for example. [22:50:40] oh librsvg. my old nemesis [22:50:51] after what, 6 or 8 years? [22:50:57] :) [22:51:09] None of these issues are new to this method of importing libraries. [22:51:21] bd808: agreed [22:51:25] What I'm trying to propose is just a method for the madness [22:51:33] so do we need to work this out in more detail or are we pretty happy ith the notions so far? [22:52:14] I think security update tracking needs more work, but maybe it's not a blocker to the larger RFC? [22:52:20] Yep, I didn't mean express an objection, just hoping for a roadmap when things go wrong. [22:52:38] andrewbogott: Sure. The questions are good. [22:52:40] *nod* [22:52:53] ok, we have 8 min left [22:53:04] any wrapup #info or #agreed or #action items? [22:53:41] should we push the security update plannning to a future action and agree on the rest? [22:53:44] or anythign else left? [22:53:56] #action Look into https://security.sensiolabs.org/check and https://github.com/sensiolabs/security-advisories for vulnerability tracking [22:54:02] i think we’re pretty agreed on everything else [22:54:27] all good [22:54:45] #action csteipp or similar person to do security update planning with Ops, Bryan, and other foresaken souls [22:54:52] #agreed general acceptance of the basic plan [22:54:56] ok, that may have been over the top [22:54:59] hehe [22:55:42] mwalker: (as long as I have you - who is taking over https://www.mediawiki.org/wiki/Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy ?) [22:56:08] unknown -- fundraising is getting a contractor to deal with centralnotice over the next couple of months [22:56:14] and it is still on the services team roadmap [22:56:37] ok, brion bd808 TimStarling mark anything else that needs deciding or marking here? [22:56:45] Heh, marking. [22:56:46] bd808: and are you at least vaguely satisfied? [22:57:11] nope [22:57:21] sumanah: I'm ecstatic! [22:57:34] \o/ [22:57:39] :DD [22:57:42] #info rename tracked in https://bugzilla.wikimedia.org/show_bug.cgi?id=68485 [22:57:55] bd808: I totally want an animated gif depicting your current state of joy from https://andtheniwaslike.co/ [22:57:58] can we mark it accepted then? [22:58:36] * andrewbogott is sad that this won't magically fix wikitech [22:58:52] #action RfC accepted, with future actions to improve the security response plan [22:58:53] andrewbogott: We can work on that. wikitech needs love too [22:59:01] #agreed RFC accepted! [22:59:07] \o/ [22:59:13] omg this is great [22:59:16] bd808: yep, this will be useful anyway [22:59:53] OK, this is the end of the meeting then! reminder: next week, rcstream, pubsubhubbub, websockets, et alia [22:59:56] achievement unlocked [23:00:00] #endmeeting [23:00:01] Meeting ended Wed Jul 23 23:00:00 2014 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [23:00:01] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-07-23-22.00.html [23:00:01] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-07-23-22.00.txt [23:00:01] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-07-23-22.00.wiki [23:00:01] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-07-23-22.00.log.html [23:00:52] https://www.mediawiki.org/w/index.php?title=Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster&diff=1075394&oldid=1075386 \o/ [23:01:06] well done everybody [23:01:11] brion: MARVELOUS. Thank you [23:01:12] thanks all [23:01:12] good points raised [23:01:29] :D [23:01:59] see y’all next time or yknow in all those other channels ;) [23:02:43] thanks bd808 [23:02:48] and all [23:02:58] aude: I hereby remind you! about assert and linker and so on! [23:03:11] sumanah: We got that :P [23:03:19] :) [23:03:23] bye all [23:03:26] * aude noted [23:03:26] Daniel wanted to comment on that today... not sure he came to it