[00:41:44] TimStarling: I landed a change from Aaron yesterday / some hours ago that esssentially removes `outputResponsePayload` [00:42:02] wasn't sure whether you noticed that re: last comments on T278579 [00:42:02] T278579: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 [00:42:04] yeah, it's reverted in my working copy [00:42:11] I had to revert it to reproduce the bug [00:42:31] ok [00:44:12] I'm not sure what direction we want to go in, but if we're happy with not doing content-length/connection-close tricks for stock installs without fpm, then we may want to also remove some of the other content-length and output buffer logic added in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/538362 [00:44:18] well, specifically I reverted it to make the previous reverts apply cleanly, but as I said on the task, the bug is actually in wfResetOutputBuffers() [00:44:48] and perhaps even OutputHandler more generally if we're happy with apache/nginx/etc taking care of compression for all cases other than 'php -S' [00:45:45] In prod that is active only to output Content-Length which I didn't quite understand why we need to do that (apart from the apache post-send use case, that is, but this code predates that by over a decade afaik) [00:47:20] (for php -S, compression is handled by router.php) [00:47:46] outputResponsePayload() is assuming that OutputHandler will send Content-Length in the normal case in which an output buffer is active [00:48:37] my solution is to remove Content-Length handling from OutputHandler, putting it in outputResponsePayload() instead [00:49:09] I'm dropping the HTTP/1.0 feature which motivated the introduction of the Content-Length header to OutputHandler [00:49:20] yeah, outputResponsePayload() only exists afaik since the 1.35 patch to support post-send on mod_php, as an attempt to collect all output centrally, which seems to not work well in practice since there's so many other ways to output things in core and extensions "out there". I mean, maybe profiling text and SpecialExport are the only ones, but it seems unlikely. [00:49:47] but if that instead interacts with the output buffer to get the output, I suppose that could work. [00:50:04] the reason for sending Content-Length in OutputHandler is because at the time, that was the only way to make Squid do keepalive [00:50:29] look up HTTP/1.0 keepalive if you're curious, but since we don't use squid anymore there's no reason to keep it [00:51:06] ah okay. I guess at the time neither apache nor php did anything to add content length? [00:51:14] no [00:51:27] apache and PHP assume HTTP/1.1 [00:51:45] HTTP/1.0 was very ancient by the time PHP was invented [00:52:00] squid was the only thing left on the internet still using it [00:53:20] oh I see. I thought I saw plain apache/php setups produce CL headers but maybe I imagined it. [00:53:37] but yeah, I guess it's moot, it's just that applications don't need them rather than that it gets added. [00:54:24] HTTP/1.1 introduced Transfer-Encoding:chunked, and PHP has that baked in [00:54:46] with TE:chunked, the theory was that you wouldn't need Content-Length [00:55:49] actually dropping non-fastcgi deferred updates is tempting, I am hitting complications [00:56:06] I've not seen e.g. WordPress or Laravel send such header by itself. but I may have missed it, didnt' look very deeply [00:56:29] I think the main thing to support perhaps is non-blocking job execution [00:56:46] but afaik we have that either way thanks to the sub-request trick which then ignores abort. [00:58:16] I thought non-fastcgi postsend support was new in 1.35? [00:58:30] triggerAsyncJobs is disabled by default ($wgRunJobsAsync) [00:59:03] ah, I totally missed that. [00:59:50] the idea of using Content-Length for postsend deferred updates is new with this train, that's why we are seeing this now [01:02:19] not exactly, I think, it was new with 1.35 a few months ago but was limited to a condition we did not trigger in production. It did expose issues for third parties which piled up in https://phabricator.wikimedia.org/T235554. [01:03:46] Last week we tried to fix that in a way that, as part of the attempted fix, seems to have moved more things to the output buffer and its CL header and yeah broke Special:Export and ohter things that bypass OutputPage and/or echo things after we stop looking. [01:04:31] which could be fixed by de-duplciating the logic and matching the logic in wfResetOutputBuffers [01:05:44] but I'm wondering, if we don't care for Squid and don't want to introduce postsend support in 1.36 for non-fastcgi; what's keeping these buffers in place? [01:05:46] squid is only used for webproxy/url-downloader these days [01:08:14] yes OK it was in 1.35 [01:11:47] squid claims to support chunked encoding since 3.2: https://wiki.squid-cache.org/Features/HTTP11 [01:12:10] I think we were using 2.6 -- they did this big rewrite for squid 3 and the branch was stalled for ages [01:19:02] do we still need mangleFlashPolicy()? [01:19:26] I guess it depends if the Chinese Railways are using MediaWiki [01:24:32] reading the "Cross Domain Policy File Specification", what a stupid idea [01:24:41] they thought they owned the web [01:25:57] probably we need it as long as any adobe product still exists: "grants a web client—such as Adobe Flash Player, Adobe Reader, etc.—permission to handle data across multiple domains" [01:26:14] so maybe the death of flash doesn't let us off the hook [01:27:23] "A domain should always host a master policy file to enforce its intended meta-policy" -- otherwise you will be owned [01:27:44] a /crossdomain.xml file must exist or your site will be owned by any adobe product [01:32:26] there's a header we can set though [01:33:15] sending "X-Permitted-Cross-Domain-Policies: none-this-response" means the current response cannot be used as a cross-domain policy [01:35:05] but testing this is going to be gross [02:06:46] I filed https://phabricator.wikimedia.org/T279496 [02:11:55] (my comment about Chinese railways and flash, if people didn't see the news: https://arstechnica.com/tech-policy/2021/01/deactivation-of-flash-cripples-chinese-railroad-for-a-day/ ) [03:28:36] TimStarling: Adobe released a flash update a 1-2 years ago that instructed the code to become inert by a certain date, which has since been reached and they no longer offer it for download. [03:29:19] that's not to say there isn't some way someone could have an older version predating that or by other means manage to get flash running, but adobe has certainly done fairly hard at making that "hard". [03:33:37] probably the main way to run flash now is https://en.wikipedia.org/wiki/Ruffle_(software) [03:34:00] which can't bypass CSRF protections [03:35:14] Rust compiled to WebAssembly -- that's the architecture of the future right? [03:42:21] FlashDevelop is also still being maintained judging by their github