[09:40:01] bvibber: disk space before or after directory as targz? Afaik the disk space issue was mainly in relation to docker upload/store/download for scap, not runtime or build time allocation. [11:30:54] I could use a review/approval for UploadWizard which is no more maintained. My aim is to have a structure test (ContentHandlerFunctionalTest) to pass when the optional `EventLogging` is not loaded. [11:30:54] EventLogging is required by a contenthandler, I have made the handler to only be registered when EventLogging is available https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UploadWizard/+/1214625 [11:32:04] on WMF prod we always have EventLogging enabled, so that is not affected :] [11:40:55] looks like UploadWizard uses EventLogging just to validate a JSON schema? don't we have a schema validator library included with core that it could use instead these days? [11:52:28] maybe, I went with the low hanging fruit to hide the content handler from MediaWiki core ContentHandlerFunctionalTest [11:54:15] and for the validation, the schema is stored in EventLogging, hence the requirement [11:54:25] (and they must be validated before being sent to EventGate) [14:51:35] see https://phabricator.wikimedia.org/T317793 [14:58:10] so I suppose replace EventLogging::schemaValidate() with (new JsonSchema\Validator())->validate(), inject additionalProperties=false, cross fingers that nothing explodes in production? [15:21:54] tgr_, https://bash.toolforge.org/quip/SL2aIpsBffdvpiTrbOvp [17:09:05] Krinkle: disk space pre-gzip. they should still compress almost as small with the strings expanded, and i *suspect* the network/disk slowdown on deploy comes from pushing all 540+ files when they don't compress well. i'll run some tests this weekend or over break [17:11:26] so if i take out the key indirection we still get: massive savings in uncompressed file size (1/6-1/10 total) vs both cdb and old lcstorestaticarrays, should still compress very well, and it's less fragile than with the key indirection [17:11:31] i think that's worth doing [17:12:04] and massive ram savings vs previous lcstorestaticarrays [22:04:39] when i have a chance this week i'll do another quick perf test, compare ram, perf & raw disk & .tar.gz size with and without the indirection. if ram isn't penalized and compressed size remains good, i'll probably go ahead and remove the key indirection, getting most of the resulting ram, generation time, and disk size improvement from de-duplications from late fallback [22:05:06] it's just fragile enough for a critical path i'm not confident in it :D [22:05:56] "perfection is reached not when there is no more code to add, but when there is no more code to take away" [22:15:26] sounds good to me [22:15:42] we can always go further later