[17:49:11] Reedy, RainbowSprinkles: what cron updates site_stats? -- https://phabricator.wikimedia.org/T169488#3410469 -- the ukwikimedia that I added to deleted.dblist got an update and its making the DBAs unhappy. I'm guessing there is some other config change I missed. [17:49:34] We don't have a cron for that... [17:49:40] You can run it by hand [17:49:45] updateSiteStats or somesuch [17:49:51] siteStatsInit? [17:49:53] * RainbowSprinkles can't remember [17:49:56] initSiteStats [17:50:17] But he's asking the reverse [17:50:19] Not how to update it [17:50:27] But where it might be being updated in a cron [17:50:29] what made it change [17:50:41] I thought it was only run manually too... [17:50:49] Because people often complain about the counts being wrong [17:50:56] The counts haven't been right in years [17:51:05] But anyway. Hmm. Not sure what would've hit ukwm [17:51:12] https://tools.wmflabs.org/sal/log/AV0U77cxZYjxCLpCzQc9 -- this looks like it was just commons [17:52:28] hmmm... so it looks like maybe I was supposed to take the db out of all the other lists? [17:52:39] Probably [17:52:46] But I'm not sure why/what would've been running on it [17:52:53] `git grep bawiktionary` only shows in deleted.list [17:53:15] `git grep ukwikimedia` shows all over the place [17:53:50] Yeah [17:53:55] We wanna nuke it from everywhere [17:54:01] deleted.dblist is really a hack [17:54:03] It doesn't do much [17:54:06] I can make that patch :) [17:54:38] deleted.dblist is basically "we removed it from everything, but *something* might still think I exist" [17:55:10] https://github.com/wikimedia/puppet/blob/e959321aa620b77403cc9379db2e86080323c6e8/modules/mediawiki/manifests/maintenance/update_article_count.pp [17:55:14] But most things don't speak dblist, so who knows how useful it is [17:55:35] Heh, so does that mean commons was a waste of time? [17:55:38] https://github.com/wikimedia/puppet/blob/e959321aa620b77403cc9379db2e86080323c6e8/modules/mediawiki/files/maintenance/update-article-count [17:55:38] And would've fixed itself? [17:55:47] o [17:55:48] No [17:55:49] for set in wikinews wikipedia wikiquote wikisource wikiversity wikivoyage wiktionary; do [17:55:57] Cause afaik, commons isn't in any of these list [17:56:01] What is this shit, seriously [17:56:03] It's in special.dblist [17:56:43] # All language subdomains of content projects, unless they use comma count [17:56:53] Heh, yeah, just listing the project domains is gonna detect that... [17:57:09] https://github.com/wikimedia/puppet/blob/1d673ee97958522731801163d52d452efc0c1759/modules/profile/manifests/mediawiki/maintenance.pp#L30 [17:57:27] That's done in SiteStats now [17:57:48] if ( $wgArticleCountMethod == 'link' ) { [17:57:48] $tables[] = 'pagelinks'; [17:57:48] $conds[] = 'pl_from=page_id'; [17:57:48] } elseif ( $wgArticleCountMethod == 'comma' ) { [17:57:48] $conds[] = 'page_len > 0'; [17:57:49] } [17:57:54] Yeah [17:58:03] bd808: So yeah, just delete it from anything but deleted.dblist :P [17:58:09] (tbh, that is a dumb dumb dumb config setting) [17:58:14] (I was looking at it yesterday) [17:58:25] I think it was introduced for wikinews or someshit [18:00:58] Anyone object if I just make that script use all.dblist? [18:01:03] Cause I've no idea why it only does osme projects [18:01:19] Just do foreachwiki [18:01:25] Skip the extra logic [18:01:32] Well, yeah, that's what I was gonna do [18:01:34] :) [18:01:34] Replace most of it with [18:01:37] /usr/local/bin/mwscriptwikiset updateArticleCount.php all.dblist --update [18:01:42] dkwikimedia is still all over the config too :/ [18:01:50] Reedy: Which is what foreachwiki does ;-) [18:01:51] but that doesn't shit itself? [18:02:00] but it already uses mwscriptwikiset [18:02:13] That's my point :p [18:02:23] Oh well, 6 of one half a dozen.... [18:02:30] We like inconsistency [18:02:35] Wait, I'll reinvent the wheel here [18:02:42] I'll make it look at all dblists in the dblist folder [18:02:56] concat and uniq the resulting list [18:03:32] We could write a scap plugin [18:03:40] Which is my answer to everything these days [18:06:21] eh. bad grep dkwiki is in deleted, not dkwikimedia [18:09:17] Reedy, RainbowSprinkles: I present https://gerrit.wikimedia.org/r/#/c/363640/ for your nuke all the things pleasure [18:09:30] heh [18:48:52] I'll do that now [19:13:01] anomie: could you look at https://gerrit.wikimedia.org/r/363656 ? more debug logging for the TOC being disabled [19:15:17] legoktm: When I looked this morning there were 7 or 8 log entries for the past week, all for a mobile enwiki FlaggedRevs code path that doesn't seem likely to be actually causing this. Those might be caused by FlaggedRevs hitting existing cache entries and resaving them in its own cache, although it may be suggestive that they're all mobile hash keys. [19:16:02] hmm, I was also concerned that they're all enwiki log entries [19:16:26] do you have any suggestions on where to look next? we could try logging on view if there's no TOC? [19:17:21] Not really. It's possible the bug has already been fixed and all the reports are just people discovering existing bad cache entries, plus whenever FlaggedRevs happens to hit one. [19:19:22] if that were the case we should add a hook to reject the bad parser cache values [19:20:22] T168337 [19:20:22] T168337: PHP Fatal error: Uncaught Error: Call to a member function canExist() on null - https://phabricator.wikimedia.org/T168337 [19:20:29] T168723 [19:20:29] T168723: LinksUpdate totally broken when JobQueueDB is in use - https://phabricator.wikimedia.org/T168723 [19:20:32] That's not a bad idea. If nothing else, rejecting the existing bad cache entries might trigger new ones to be saved. [19:20:37] * RainbowSprinkles link drops and walks away [19:24:57] do you think we can safely reject anything cached with TOC disabled? [19:26:17] * legoktm bbl lunch [19:26:22] Should be safe, since we're seeing very few logs going in that way. [19:26:50] Include a log entry so we can keep an eye on it, maybe. [19:34:19] legoktm: Hmm. enwiki:pcache:idhash:27072613-0!canonical!responsiveimages=0 is a non-FlaggedRevs cache entry, with getTOCEnabled() returning false, and mCacheTime === '20170705223222'. There should be a log entry for when that was saved, I'd expect, but I don't see one. [19:36:11] tgr: Blah https://phabricator.wikimedia.org/T168337#3413100 [19:36:13] :\ [19:37:18] RainbowSprinkles: MWException subclasses do their own output handling [19:37:28] grrrr, you're right [19:37:30] MWExceptionRenderer is never even invoked [19:37:53] .... [19:37:56] This is a ball of mud [19:38:13] it sure is [19:38:31] Balls of mud roll better than square ones. [19:39:21] Basically the problem is ErrorPageError assumes it's always going to be in the web UI and not on the cli, ever [19:39:34] So LocalFileLockError should probably inherit from MWException directly [19:39:36] Or something [19:39:49] ErrorPageError makes some crappy assumptions [19:39:55] Or maybe I fix that to work on the CLI too? [19:44:24] tgr: WIP, untested... https://gerrit.wikimedia.org/r/363660 [19:44:27] But I think that might do it [19:44:32] (it's really kludgy) [19:46:07] RainbowSprinkles: if you are looking for a quick hack, this should work [19:46:23] Mainly I want to close the bug so I can release the damn 1.29 tarball finally [19:46:31] (it probably wasn't a blocker, but meh) [19:46:47] And yeah, it's a hack. That whole thing needs refactoring [19:46:49] the nice solution IMO is to give a getText + getHTML method to all MWException subclasses, and have MWExceptionRenderer do the rest of the work [19:47:05] We also don't need so many subclasses :) [19:47:17] Our exception hierarchy is ugly [19:47:24] I'll do that some time, needed for T111731 [19:47:24] T111731: Integrate a modern debug/error display tool into MediaWiki - https://phabricator.wikimedia.org/T111731 [19:49:46] If you think it's ok, I'll remove the WIP for a real review :) [19:51:05] I broke a unit test! [19:51:09] https://integration.wikimedia.org/ci/job/mediawiki-phpunit-hhvm-jessie/9377/console [19:51:18] (color me surprised there's actual tests here) [19:51:19] RainbowSprinkles: wanna check for MW_API as well? file lock errors can be triggered there too, probably [19:51:24] otherwise looks OK [19:51:37] Yeah, can check for MW_API || cli [19:51:50] (really, we should use a define for that instead of a global like MW_API) [19:51:56] (nb: MW_CLI) [19:54:45] Amended to also check MW_API, plus a fix for the unit test [20:00:09] Grrr still borked [20:03:59] need to set it to false I think [20:04:17] Oh, duh [20:04:19] Backwards [20:05:40] Heh, backwards logic is why I told my friend I'm afraid of AI in like weaponry. Imagine if threatLevel > 10 is swapped for threatLevel < 10 [20:05:59] You went from an awesome battlefield device to a scary terror machine that kills whole villages [20:07:27] legoktm (also Krinkle): https://phabricator.wikimedia.org/T168040#3413244 [20:08:56] anomie: I like the theory, sounds like something we should address regardless. [20:09:12] RainbowSprinkles: Reminds me of https://xkcd.com/534/ [20:09:28] OutputPage should probably send a clone to ParserOutput [20:09:38] anomie: True :) [20:10:24] ParserCache* [20:21:00] tgr: Backporting to REL1_29 as well [20:24:58] anomie: were you going to write a patch for that? [20:27:43] legoktm: If you or Krinkle want to beat me to it, feel free. [20:28:08] * anomie probably won't get to it this afternoon [20:35:52] Ok kids, last blocker: T168723 [20:35:52] T168723: LinksUpdate totally broken when JobQueueDB is in use - https://phabricator.wikimedia.org/T168723 [20:36:15] Ah, wait [20:36:17] "Given these elements, imho, this bug should no more be a 1.29 blocker, or at least a RC1 should be issued to get more feedback (is it a vanilla MW bug or related to interactions with specific extensions?) and do not block too much the release itself." [20:37:20] I don't think that's right [20:37:27] I'm pretty sure it's broken [20:38:30] Well, it's clearly broken :) [20:38:35] I could maybe do an RC1 tho [20:38:42] If that's the only remaining blocker [20:54:36] RainbowSprinkles: in the meantime wanna review the https://gerrit.wikimedia.org/r/325081 https://gerrit.wikimedia.org/r/325080 backports I just updated? :) [20:56:03] Done [20:56:05] * RainbowSprinkles runs to store [20:59:34] thanks :) [22:10:42] if anybody knows where all the different language versions of #REDIRECT are, that would help with some historical parsing of revision text we need to do, question is being asked here: https://phabricator.wikimedia.org/T161146#3381362 [22:11:33] milimetric: I do [22:11:55] languages/messages/Messages*.php in mw core [22:12:00] Then auf Deutsche [22:12:01] awesome, thx [22:12:01] $magicWords = [ [22:12:01] 'redirect' => [ '0', '#WEITERLEITUNG', '#REDIRECT' ], [22:12:36] We really should move magic words to json or some shit too [22:12:51] Most of those files should probably go [22:16:32] WIKITEXT IS A PROGRAMMING LANGUAGE, IT SHOULDN'T BE LOCALIZED [22:22:18] wasn't applescript localizable? [22:24:12] hmm apparently they dropped that long ago in like macos 8.5 [22:26:15] I mean, all these thoughts are lovely, and I support them, but I still have to deal with the history which is written in the stone tablets :) [22:27:46] but looking up the magicwords thing is great, we should be able to just run PHP and output all those variations, and look for all of them. Now, I assume that if we found like #WEITERLEITUNG on French Wiktionary, it wouldn't be a redirect, but I'll test that just to make sure :) [22:38:32] milimetric: yeah it should trigger only on the matching language(s), though it likely won't hurt to interpret them all ;) [22:44:10] * bd808 replaces MaxSem with some obfuscating pre-processor commands [22:45:53] * MaxSem ports ~~Labs~~ The Cloud into 1C: Accounting for bd808's enjoyment of localized languages [22:49:16] Yuvi was playing with a python variant with Tamil keywords for a while [22:50:13] and it didn't result in anything [23:09:42] MaxSem: true [23:18:58] milimetric: Yes, other localized languages won't work on a wiki. (FR not on DE and vice versa) [23:19:04] With the caveat that English works everywhere [23:19:11] And languages *might* have a fallback [23:19:33] So "foo" might not define it, but it inherits from "bar" and bar does [23:19:41] (ultimately english is the ultimate fallback) [23:20:01] Strike one of those ultimates [23:20:03] It's not that cool [23:23:37] The Ultimate Fallback, coming to servers geographically near you [23:34:29] You could probably spit out something pretty easily from MW itself. Like iterate the wikis (foreachwiki) and grab the magic word from the language object