[20:38:28] I've been having a strange problem with our database seemingly out of nowhere. It's locking up constantly and going over capacity, and I noticed that the following queries seem to consistently top the load list: https://pastebin.com/EcFkV8a9 https://pastebin.com/AnjGeUrB [20:39:32] Seems to mostly be with actor, image, and user_groups. I checked their schema and found that they match the manual. I'm flabbergasted, because this happened out of nowhere. [20:44:16] "WHERE `ug_group` IS NULL". That's interesting. Get all images and their uploader, but only if the uploader *is not in any user group* [20:44:59] Also, a paginated query. Probably from an api query or a special page [20:47:36] I wonder if this is from GlobalUsage... [20:48:01] (not in any user group includes normal users and autoconfirmed) https://www.mediawiki.org/wiki/Manual:User_groups_table [20:49:10] What I don't understand is, why would you want to exclude images uploaded by current sysops, or bots, or any other explicit user group? [20:50:51] Oh, wait, there's also "ug_group` = ?" in the join clause... It's excluding uploads from a single specific user group, not all groups [20:51:49] It's probably excluding bots, then. It's probably [[Special:NewFiles]] [20:53:43] You're right, I can see it in the sample queries I have. [20:53:47] NewFilesPager [20:54:10] Yeah, that probably is... I'd expect this page use the recentchanges table like Special:NewPages, but it isn't... that's potentially expensive [20:54:52] Maybe a rogue bot crawler is following every link of your wiki... [20:55:10] I wouldn't even be surprised. They are the bane of my existence. [20:55:27] Any suggestions? Hell, I'll shut that special page off if I have to. [20:58:26] Actually, I guess I can put a managed challenge on it. That might help. [20:59:33] be reactive and ban the offender's IP (or range)... or install some rate-limiting controls. The nginx implementation documented here is what I was using, before switching to anubis, and worked pretty well https://www.mediawiki.org/wiki/Handling_web_crawlers [21:03:37] Actually, it doesn't look like there's a ton of traffic to it. Only about 500 or so hits over the past 3 days. [21:36:26] I wonder if anything else is specifically using that NewFilesPager.