[00:03:53] hmm...interesting: https://packagist.org/packages/roave/security-advisories [00:04:17] Kunal pointed that out yesterday [00:04:52] It's a kind of neat idea but I'm not sure that it would really scale for all of packagist [00:06:21] Chris has a POC jenkins job to scan using the sensiolabs service -- https://phabricator.wikimedia.org/T74193 [00:06:52] I need to poke him next week to see what else he wants to do with that [00:07:18] * bd808 also needs to bug hashar about jenkins/zuul and composer stuff again [00:08:35] 3Librarization, MediaWiki-Core-Team: Blog post about work on librarization project and future hopes - https://phabricator.wikimedia.org/T85482#947869 (10bd808) [00:10:19] 3MediaWiki-Core-Team, Librarization: Have a check for reported security issues in dependencies - https://phabricator.wikimedia.org/T74193#949994 (10bd808) [00:10:26] bd808: he was going to add the monolog security issue to the git repo and test that it reported that monolog had an issue [00:11:08] ah. yeah that would be good. Like an actual test [00:11:32] I suppose that means we want to hold off a bit on updating to the version that has the patch [00:12:37] yeah. I filed https://phabricator.wikimedia.org/T85535?workflow=create for upgrading monolog, there's a b/c break we'll have to account for [00:14:09] yeah I was going to look at that. I don't think it will make any difference at all in what we are using [00:14:46] but I should make sure and adjust as necessary [00:15:12] do you think we could enlist the author of the roave/security-advisories thing to adapt something that would work for us? [00:15:54] https://security.sensiolabs.org/check [00:16:12] That's what Chris is using in the test he wrote [00:16:21] I kinda figured the roave thing was using Sensio's checker [00:16:53] (didn't look closely enough to see) [00:17:27] if it's not, then yeah, the Sensio thing is definitely where its at [00:18:16] The "database" behind sensio's tool is -- https://github.com/FriendsOfPHP/security-advisories [00:50:08] bd808: I made it to "MediaWiki is great software" [00:50:26] robla wrote that :) [00:50:43] :-P [00:50:51] 3wikidata-query-service, Wikidata, MediaWiki-Core-Team: Figure out quantity representation - https://phabricator.wikimedia.org/T85298#950096 (10Smalyshev) Titan allows indexing the floats, but not in vertex-centric indexes. Elasticsearch indexes support floats, for example. I'm not sure what is the actual impact... [00:52:23] so legoktm: here's the thing: for performance, scalabiliity, security and stability, it kicks ass. For readability and modularity, not so much. [00:55:38] yeah, I kept reading :P [01:04:38] 3Librarization, MediaWiki-Vendor, MediaWiki-Core-Team: Upgrade monolog to 1.12.0 - https://phabricator.wikimedia.org/T85535#950117 (10bd808) [01:05:25] 3Librarization, MediaWiki-Vendor, MediaWiki-Core-Team: Upgrade monolog to 1.12.0 - https://phabricator.wikimedia.org/T85535#949087 (10bd808) We should hold off on doing this until @csteipp has had a chance to test his vulnerability scanner using this test case. [01:29:40] http://blog.wikimedia.org/2014/12/29/how-we-made-editing-wikipedia-twice-as-fast/ [01:30:32] hacker news: https://news.ycombinator.com/item?id=8816056 [01:33:04] {{COI}} [01:33:05] :) [01:42:26] and how [01:51:42] ori, \m/ [12:31:12] 3CirrusSearch, MediaWiki-Core-Team: PHP Warnings when "startingOver" an index - https://phabricator.wikimedia.org/T75457#950562 (10SmartK) I can confirm that this is **resolved** now. I tried the "master" version of CirrusSearch together with MediaWiki 1.24.1. Thank you again. [15:22:51] <^d> manybubbles: Do you think https://gerrit.wikimedia.org/r/#/c/180995/ is ok to go? [15:23:04] * manybubbles read read read [15:23:36] I think I said I'd test it locally and then forgot. checking now [15:31:05] <^d> hehe ok :) [15:39:45] ^d: I uploaded another patch version. +2 otherwise [15:41:41] <^d> How did I miss that? [15:42:04] <^d> +1 [16:05:44] <^d> bd808: My patches for less MWTestCases got merged :) [16:05:56] sweet [17:09:15] <^d> robla: I just saw the e-mail about heat being out. Still getting over my cold and I don't want to sit in the cold and make it worse. I'll still be available for our 1:1 on hangout in ~2h. [17:30:25] Anything for scrum of scrums folks? [17:30:58] Things I know about: legoktm starting review of whitelist/grey list for Yuvi and the new RFC draft from me [17:38:31] * bd808 huddles over his heat vent. 22F/-5C outside here today [17:39:09] <^d> it's 45 in SF. [17:39:34] <^d> And apparently the office heat is out until monday [17:39:38] yuck [17:39:46] WFH [17:40:09] or just say fuck it and start the long weekend early [17:40:10] <^d> I had packed my bag and was about to walk out the door when I got the e-mail :p [18:26:07] 3Librarization, MediaWiki-Core-Team: Update WMF $wgDebugLogGroups config to set a level threshold for memcached errors - https://phabricator.wikimedia.org/T85628#951103 (10bd808) [18:36:57] <^d> manybubbles: i'm wondering if there's any other cleanup/removing junk we can do in Cirrus. [18:39:07] surely [18:40:42] <^d> Random was a fun experiment but just made things more complicated :) [18:58:23] hi Chad, Ori's running late for our 1:1 just before yours, which probably means I'll be running late shortly thereafter [18:58:38] ^d ^ [18:59:04] * robla tries to figure out why his nick isn't changing [18:59:22] <^d> no worries [19:56:05] bd808: so, I noticed that when you add the merge plugin to core, the \Wikimedia\Composer\MergePlugin class is added to the autoloader. [19:56:33] yeah that will happen [19:57:15] Not sure what it would/could hurt [19:57:39] if the composer autoloader is so bad that having a single class added to it cause problems... [20:01:58] * bd808 switches to a laptop with a better view of the sportsmans playing sportsball [20:02:15] well I was thinking of it like a maint script, which are all guarded by PHP_SAPI !== 'cli': die. Except in this case if you try running class_exists on it, it'll fatal because the Composer interfaces aren't being loaded. [20:02:58] Don't try to make real software fit our weird patterns. :) [20:03:46] The same thing would happen with the monolog spi for logging [20:03:51] <^d> I hate the initial slashes. [20:03:58] <^d> \Foo vs Foo [20:04:36] Foo\Bar is relative to the current namespace, \Foo\Bar is absolute [20:05:46] And absolute lets you leave off a uses clause for something that is just a one off which I like personally [20:05:56] <^d> I prefer use :) [20:06:08] <^d> Be explicit about your dependencies :) [20:07:26] I don't like the things you can do with use like "use Foo\Bar as Baz;" [20:07:53] but that's a whole other thing I guess [20:11:00] speaking of composer, ewwwtf: https://performance.wikimedia.org/xenon/svgs/hourly/2014-12-31_07.svgz [20:11:48] 5% of overall CPU time just on composer init, really? [20:14:33] MaxSem: I think that is wikidata not setting the autoloaded to append instead of prepend yet [20:14:55] hoo has a patch to fix it as I recall [20:15:05] I should probably file that upstream bug now... [20:17:41] we don't have any graphs of vendor using prepend autoload do we? [20:18:04] I don't think so, no [20:18:22] it needs a reproduction case [20:25:44] well, I can reproduce it locally [20:30:33] https://www.drupal.org/node/1920666 looks interesting btw [20:41:08] bd808: "[Composer's] install command checks if a lock file is present, and if it is, it downloads the versions specified there (regardless of what composer.json says)." Does wikimedia/composer-merge-plugin take that into account? [20:41:19] (quote is from https://getcomposer.org/doc/01-basic-usage.md) [20:42:25] ori: It should because it doesn't mess with anything itself, it just presents more dependencies to the composer core [20:43:48] so composer.local.json cannot override version constraints in composer.json? [20:44:29] it can just add to, if you give it something incompatible it'll throw an exception [20:46:19] and packages unique to the local composer file will be installed even when composer.lock is present? [20:47:15] yes [20:47:22] cool. [20:47:51] and they will be added to composer.lock [20:48:02] happy to merge then, unless you want to wait for additional feedback [20:48:16] basically it's just saying "depend on this random package" [20:48:59] I'd be happy for a +1 for now. I don't want to end up with a "you rammed this through" pile of complaints [20:49:16] but thanks greatly for offering! [20:49:21] no problem [20:50:34] the protected vs private tirade is making me tired [20:51:56] so just ignore him [20:56:49] he sent a pull request to change everything from protected to private :/ [21:01:57] bd808, ori: does https://dpaste.de/4TYj/raw look like a good bug report for the composer autoloader issue? [21:02:32] 3Librarization, MediaWiki-Core-Team: Publish MediaWiki codesniffer config on Packagist - https://phabricator.wikimedia.org/T85631#951161 (10bd808) [21:03:16] hmm [21:03:32] I think wikidata does have prepend-autoloader: false set. [21:04:13] I don't think it is live on the cluster (at least on the `pedias) [21:04:40] I looked at php-1.25wmf12 and I see prepend-autoloader: false in extensions/Wikidata/composer.json [21:05:45] legoktm: lgtm [21:09:03] eh, did graphite perf metrics just die on 12/23? [21:20:25] filed as https://github.com/composer/composer/issues/3603 [21:21:01] 3Librarization, MediaWiki-Core-Team: File upstream bug with composer about the autoloader being slow - https://phabricator.wikimedia.org/T85182#951251 (10Legoktm) a:3Legoktm https://github.com/composer/composer/issues/3603 [21:41:11] 3Librarization, MediaWiki-Core-Team: Create RFC for library extraction (PHP & javascript) practices - https://phabricator.wikimedia.org/T1017#951293 (10bd808) 5Open>3Resolved Announced on list. [21:41:55] 3Librarization, MediaWiki-Core-Team: Update WMF $wgDebugLogGroups config to set a level threshold for memcached errors - https://phabricator.wikimedia.org/T85628#951300 (10bd808) [21:41:56] 3Librarization, MediaWiki-Core-Team: Support filtering log events by level in $wgDebugLogGroups - https://phabricator.wikimedia.org/T85073#951299 (10bd808) 5Open>3Resolved [22:50:21] :o wtf [22:50:40] bd808: I've been wondering whether it would make sense for us to just require monolog in core and do the $wgDebugLogGroups mapping when creating the handlers rather than mapping it during config [22:51:25] There are pros and cons I suppose [22:52:10] ultimately I'd like to kill off $wgDebugLogGroups but I think we need to prove that it can be done first :) [22:53:19] why? [22:56:25] ... [22:56:29] ok, I'm back. [22:57:06] Why kill it off? Or why prove that it can be killed? [22:57:29] yeah, why kill it? [22:57:52] The why kill it off is that monolog would be much more expressive and it's just duplicate to a good monolog config [22:58:50] There is a lot more you can do with monolog than $wgDebugLogGroups can and should express [22:59:12] configuring monolog appears significantly more complicated than $wgDebugLogGroups['broken-extension'] = '/var/log/brokenextension.log'; [22:59:20] es [22:59:22] yes [23:00:37] having a zillion log files is kind of dumb once you have structured logging. Really one set of handlers at a reasonable verbosity should work for almost everyone [23:01:15] if you want to split things up you can do that on the message receiving side or filter the logs with tools [23:03:09] so you'd just set $wgDebugLogFile and everything would be logged to it? [23:03:48] most of the time I don't need logging, except when I need to debug one specific thing, so I'll turn on logging just for that thing. [23:03:55] sort of, you'd configure $wmgMonologConfig and have most everything got to the same set of handlers [23:04:25] everybody needs logging all the time :) they just may not know it [23:09:56] hmm [23:10:18] why do you think you don't need logging? [23:11:17] Because 99% of the time the site isn't broken :P [23:12:08] (I'm mainly thinking of how we do stuff on Uncyclopedia right now) [23:12:51] right after we do an upgrade, I'll turn on logging for everything, and I have a simple udp --> IRC script to spam our IRC channel. After about a week I shut it off and then only turn it back on when someone reports something is broken [23:13:08] so when something breaks you rush to add logging and then hope it breaks again so you can figure out what went wrong? [23:13:24] that's not awesome for edge case bugs [23:13:26] pretty much! [23:13:33] but do what you like :) [23:14:03] * bd808 has food to eat now [23:14:30] enjoy :D