[05:07:10] this is so fucked [05:07:38] some random article revisions are being fetched from memcached hundreds of times a second [05:07:55] i hacked some debug logging to try and figure out wth is going on [05:08:09] all i can say so far is that it is related to Module:Yesno and Module:Category_handler on enwiki [05:14:24] AaronSchulz: [05:14:46] #0 MemcachedPeclBagOStuff->get(enwiki:revisiontext:textid:664533748) called at [/srv/mediawiki/php-1.26wmf7/includes/Revision.php:1515] [05:14:46] #1 Revision->loadText() called at [/srv/mediawiki/php-1.26wmf7/includes/Revision.php:1071] [05:14:46] #2 Revision->getContentInternal() called at [/srv/mediawiki/php-1.26wmf7/includes/Revision.php:1027] [05:14:48] #3 Revision->getContent() called at [/srv/mediawiki/php-1.26wmf7/includes/parser/Parser.php:3985] [05:14:50] #4 Parser::statelessFetchTemplate(Module:Yesno, Object of class Parser could not be converted to string) called at [/srv/mediawiki/php-1.26wmf7/includes/parser/Parser.php:3901] [05:14:52] #5 Parser->fetchTemplateAndTitle(Module:Yesno) called at [/srv/mediawiki/php-1.26wmf7/extensions/Scribunto/common/Base.php:153] [05:14:54] #6 ScribuntoEngineBase->fetchModuleFromParser(Module:Yesno) called at [/srv/mediawiki/php-1.26wmf7/extensions/Scribunto/engines/LuaCommon/LuaCommon.php:509] [05:14:56] this is typical [05:15:10] any clue wtf is going on? [05:16:07] the request URI never has anything to do with that revision [05:17:40] i wonder if it has anything to do with [05:17:51] # Give extensions a chance to select the revision instead [05:17:51] $id = false; # Assume current [05:17:51] Hooks::run( 'BeforeParserFetchTemplateAndtitle', [05:17:53] array( $parser, $title, &$skip, &$id ) ); [05:19:28] that hook is used by flagged revs iirc [05:19:34] what makes you think the revision is random? [05:19:44] legoktm, not on enwiki [05:20:20] i didn't mean random-random [05:20:37] i meant random as in "bizarrely specific yet out-of-place" sense [05:21:02] what is the text of text id 664533748? [05:21:30] AaronSchulz: really? it's in the unconditial setup part [05:21:53] legoktm, the enwiki configuration does not use template stabilizing [05:21:58] all those hooks do nothing there [05:22:00] it's https://en.wikipedia.org/wiki/Andrea_Pirlo?oldid=664533748 [05:22:11] ori, you are confusing rev ID and text ID [05:22:21] ok [05:22:21] oh [05:23:19] it's the content of Module:Yesno [05:24:13] still the 100s/sec rate is weird [05:24:18] is it coming from apaches or job runners? [05:24:21] https://dpaste.de/OxYd/raw [05:24:26] apaches [05:24:49] variety of requests from a variety of users [05:24:56] no pattern that i can discern [05:25:14] any other templates getting hit this much? [05:25:56] yes. this happened to be at the top of mc1001, but there were others, and the other memcached servers each had a few revisions at the top when sorted by bandwidth [05:26:52] here's another one [05:26:53] commonswiki:revisiontext:textid:100141736 [05:27:26] contents: https://dpaste.de/yU5X/raw [05:28:27] enwiki:revisiontext:textid:662607919 gets 200 reqs/sec on mc1002 [05:28:33] more like 250/s [05:28:47] is there a phab task? [05:29:12] contents: https://dpaste.de/2erL/raw [05:29:15] no [05:29:28] i actually have to go, promised simona i'd watch a movie with her [05:29:30] file one? [05:29:37] (that's what you get for asking :P) [08:02:31] anomie: re "I just noticed that OO.ui.InputWidget.prototype.getInputElement is marked as @private, even though all the subclasses have to override it.", clearly a documentation bug, fixing [08:10:49] anomie: https://gerrit.wikimedia.org/r/214791 [22:15:38] ori: I got a deb built for ggml and installed it on logstash1001