[03:35:36] hey tech, someone posted a message on the help board about a strange bug with the createwiki bot. probs something minor but just a heads up [03:35:36] https://meta.miraheze.org/wiki/Help_desk?diff=prev&oldid=543584 [04:11:47] It was a known problem that was fixed earlier [20:22:16] [1/2] mostly for other techies, but input from others is appreciated as well: [20:22:16] [2/2] I've opened https://issue-tracker.miraheze.org/T15537 for all the different caching issues that plague us. Please do fill in for anything I've missed. [20:23:18] ideally we get a list together and then we get out heads together and maybe, just maybe, the average editor will be able to edit mediawiki:common.css and have it show up instantly [21:09:16] iirc WO have an extension that allows admins to purge the cloudflare load.php cache, theoretically updates to common/skin .css or .js pages could just purge that [21:09:52] cf caching is almost entirely unproblematic on our end fyi [21:16:03] do you do cf -> varnish [21:16:14] I cut out the varnish layer and just have cloudflare [21:16:30] works out significantly cheaper in both technical debt and running servers [21:24:56] will defer to infra for an answer on the feasibility and performance of that [21:26:09] or anyone who was around when either one was put into use [21:26:28] cause i swear at some point cf wasnt in use at all but i might be completely wrong on that [21:45:38] is CF used to cache articles [21:45:55] I'd assume it is but I haven't looked too deep [21:46:25] actually maybe it's not lmao [21:46:36] nope [21:46:51] i think my fun experiment of caching fischwiki's main page might still be going [21:47:12] could probably expand it and see how much we explode [21:49:34] my solution to it was to make a job that purges from cloudflare when pages are edited [21:49:38] so far things haven't kaboomed too far [21:51:05] apart from the time I forgot some prick made centralauth cookies have a capital S which caused a ton of csrf token leaking but like [21:51:18] add it to the list of centralauth complaints [21:57:33] We already purge varnish cache when a page is edited, so to cache HTML responses with Cloudflare we'd need to do the same. Since both WGL and WO are doing it, I'd imagine it's not a huge hurdle and can potentially reduce page load times significantly, especially for non-US folks. [21:58:01] iirc telepedia do it too [21:58:16] assuming the cache invalidation is handled decently it's not impossible [21:58:44] I don't particularly like the way cache invalidation is handled right now but I don't see a better alternative that doesn't involve deferred updates which is also not ideal [21:58:50] If OrangeStar is around he would probably oppose because it's a nontrivial dependency on Cloudflare, though at this point we already have quite a few nontrivial dependencies on proprietary software. [21:59:24] the way I see it cloudflare already offer it for free so it's not like it's going to cause a massive bill [22:00:19] I've actually ended up going quite far into the cloudflare rabbithole and decided to get R2 on the move which is interesting [22:00:33] there's a few bits that need sorting out still with that but it works better than the previous alternative [22:01:58] [1/3] in this case he can eat an egg since [22:01:59] [2/3] a) usual cf downtimes dont affect us [22:01:59] [3/3] b) cf cache dies entirely? ok we'll just hit varnish then [22:02:36] at least if we continue upkeeping varnish [22:02:42] realistically any cf downtime would still take mh down [22:02:57] if it's bad enough to take caching down then it would probably take down the rest of it [22:03:27] 100% uptime SLA 😊 [22:03:29] these days they don't usually go down aside from localised datacenter reroutings but then it's an edge issue which would again happen anyway [22:03:37] I'm p sure it's not 100% lmao [22:03:44] oh shit no it is [22:04:23] I mean I wouldn't care anyway because some cloudflare > no cloudflare [22:04:38] without cloudflare fending off any amount of scrapers would be impossible [22:04:49] all gain for no pain [22:05:47] the funny thing is [22:05:57] we only have 99.9% uptime SLA with ovh lol [22:06:04] so realistically the weakest link is them [22:06:09] although it's probably something below that tbh [22:06:31] cough me cough [22:06:41] The main concern with CF is that they do not have transparent pricing and is known for raising prices abruptly (I saw a blog about this a while ago; can't find it now). Unfortunately we lack a good alternative if we ever need to switch. [22:07:27] I'm not a huge fan of their pricing models but [22:07:32] they are the best at what they do [22:08:38] i think we'll be good as long as https://www.cloudflare.com/galileo/ lasts [22:08:42] Yeah CF is very nice for smaller-scale website for which I am very grateful, though they have a worse reputation when it comes to larger corporations. Fortunately MH still gets enterprise stuff for free for now. [22:08:55] WO is particularly locked into CF [22:09:09] cloudflare registrar was not a decision I made but [22:09:15] I don't care enough to change it now [22:09:44] best do it before it bites your ass [22:09:49] the one pain in the ass will be R2 if they decide to suddenly raise the price on that [22:10:04] the rest I can either live without or hope galileo sticks around [22:10:36] anyway to summarize this conversation a bit: introducing page caching wouldn't increase our dependency on CF [22:14:03] ```(http.request.uri.path wildcard r"/wiki/" and not http.cookie matches r".(s|S)ession=.") or (http.request.uri.path matches r"/w/(index\.php|thumb\.php)" and not http.cookie matches r".(s|S)ession=.*")``` is the expression we use, and https://github.com/WikiOasis/WikiOasisMagic/blob/main/includes/Jobs/CloudflarePurgeJob.php is roughly how it works but ymmv, [22:15:35] I have no idea what the failure rate of purge jobs is nor the rate limit because I can't find those logs for whatever reaon but it seems to purge some stuff from cf [22:16:00] although I think you might have issues with the rate limit at your scale [22:16:57] Thanks! I think as a start we could try to purge load.php so that the worse case scenario of 5 minutes + 5 minutes for Common.css purges doesn't occur. [22:17:43] that would probably help, I need to look at the load.php purging but it'll probably take the form of enqueuing some sort of purge for load.php if it's a MediaWiki namespace .css file [22:17:45] or .js [22:17:55] although I don't think .js is such a big issue as css [22:17:59] although we have really high TTLs [22:22:15] I remember trying to have a cache key depend on multiple pages when I wrote RenderBlocking. That never happened and I just set a high TTL. [22:22:51] "depend on multiple pages" as in invalidate the cache when any of the pages is edited