[02:14:19] TimStarling: are you working today? [02:14:28] yes [02:14:42] haven't opened my email yet though [02:18:48] TimStarling: https://gerrit.wikimedia.org/r/#/c/307105/10 & https://gerrit.wikimedia.org/r/#/c/308172/ would be nice for someone to look at [04:09:30] TimStarling: posted some comments on the 308172 [04:15:55] need to horizontally scroll to read those comments in my browser [04:16:06] better to read the notification email [04:16:30] incredible that they could have made the layout even worse than it was [04:16:54] the related patchsets part of the GUI should be under the "file list"/diff links section [04:17:02] instead they made it to the right of everything [04:17:19] so it looks terrible, and requires scrolling on most monitors [04:17:32] I mean...my 4K 27" monitor doesn't need scrolling, but... [04:17:51] I've been using my 13" laptop mostly these days [04:18:25] and by "4K" I mean "UHD" ;) [04:18:34] it's all lies [04:18:49] I use portrait mode, 1050 width by 1680 height, can fit 100 lines of code on it [04:19:32] browser is never actually the full 1050 though, since I use horizontal position on the desktop to find windows [04:20:09] I guess you are right about clearLink etc. being broken already [04:21:14] it would be nice if we had some kind of fake lagged slave wiki to test this on, instead of it all being theoretical [04:26:11] TimStarling: are you thinking lag for subsequent requests or within a request? [04:26:25] I find the trick I mention to Matt F useful for the later. [04:26:46] were you DB_SLAVE gets a second master connection and keeps the stale snapshot [04:27:01] then you say see some of the bad assumptions actually break [04:27:25] yeah, I used to have such a setup [04:27:29] like clearing something from process cache and hoping it loads from a slave even though the COMMIT hasn't happened yet [04:27:44] someone was doing that and wfWaitForSlaves() in Flow [04:27:56] so it would just wait for random other people's transactions and then read the same stale data [04:28:30] at one point I even had full on mysql replication on my laptop [04:28:43] but you don't bother rebuilding such things unless you need them [04:28:59] I use https://www.mediawiki.org/wiki/User:Aaron_Schulz/Quick_Vagrant_DB_Replication when I feel like it [04:29:42] but to answer your question, both, I guess [04:29:57] hmm, I duplicated two steps [04:29:59] * AaronSchulz fixes that page [04:30:16] for example whether a link will go red when the page is deleted, or if refreshLinks will store stale HTML to the parser cache [04:30:18] I no I didn't [04:30:22] one was for the slave [04:30:49] I know Database::getCacheSetOptions() is meant to avoid that [04:31:21] my hope with WAN cache is too have a thing that Just Works (tm) ideally [04:31:35] but it's complicated, and there's no automated testing for it [04:31:41] and for no one to ever write another cache manager in an extension again [04:32:38] well there are lots of unit tests, some of them checking mock lag return values, but nothing with actual DB connections [04:33:04] how much faster is WANObjectCache compared to a slave DB query? [04:34:09] part of the reason I always advocated batch queries over memcached was because the performance difference between a simple DB query and a memcached query was not so amazing [04:35:51] and with memcached you have complexity overhead [04:35:57] easy to screw up [04:36:01] single row queries are useful for load-shifting and if a key gets hot [04:36:17] looking at memkeys, grafana, and ganglia, we are not really using mc that much [04:36:39] I'd rather the DBs being spending resources on more useful things, like replication and complex queries [04:37:03] but for the common key, I don't think it will be amazing either [04:37:41] though Revision got a bit more complex over the years with all the JOINs [04:39:41] heh, I've heard of single mc boxes getting the combined hit rate of our 18 [04:44:03] TimStarling: note that patch is marked as dependent on another [04:44:27] yeah, it's obvious once you've +2'd it that that is the case [04:44:36] though you probably can't see that part ;) [04:45:22] it used to be obvious which direction dependencies were in [04:45:36] now I have to look at hashes and see which "related change" patches the parent [04:46:06] it's ridiculous [04:48:05] *matches [04:59:14] I think a lot of things would be a lot easier if I didn't have NewParserTest inherit from MediaWikiTestCase [05:22:57] TimStarling: are you doing that? [05:23:54] tried it, but nothing is ever quite that simple... [05:25:31] when I say NewParserTest, I mean the 30 lines of code remaining after I deleted the whole class [05:26:05] and it's now called ParserTestCase [05:26:29] the thing that used to be called ParserTest is now ParserTestRunner, all the good bits from both previous classes are in ParserTestRunner now [05:27:12] the current challenge is getting ParserTestRunner to run under phpunit [05:28:41] most of that work is done, for example, abstracting things like skip results and warning output [06:04:20] TimStarling: I added you to the parent change https://gerrit.wikimedia.org/r/#/c/307105/ ; a sort of risk-based insurance, though not a hard dependency. [17:32:37] aude: Do you know if a fix for Wikibase's implicit transactions is going to land? Still trending high on my graphs :) [17:34:37] Unrelated: I'm convinced the "Duplicate get(): "{key}" fetched {count} times" messages must be unimportant. [17:36:15] anomie tgr|away i moved the weekly meeting to 30 minutes long and starting in 25 minutes. again. [17:47:22] ostriches: they mostly are, yes. That's logging to help track down BagOStuff repeated fetches that could be cached in-process. [17:47:35] So it's only a perf warning [17:48:19] is anyone fixing them or are we just logging them for shits and giggles? :) [17:48:43] Cuz we get like 400k an hour! [17:49:03] heh. Krinkle was fixing some I think. Not sure if he got to a point where he stopped or not [17:50:06] we were excited about such things in February or so when we saw SessionManager/AuthManager being a bit wasteful for repeated lookups [17:50:53] amplifying a perf problem with extra logging is maybe not the most awesome thing :) [17:52:49] most of it seems to be Echo, just file a bug and ask them to look at it? [17:53:33] something like 10K dup lookups per minute, that's nontrivial perf-wise [17:54:18] maybe INFO would be better than WARNING though [17:55:20] Well, if it's something that should be ostensibly looked at and possibly fixed, warn isn't bad. [17:55:30] But if it really is just info because Somebody Might Care, then yeah info [17:55:30] :) [17:55:52] ostriches: bug the perf team! ;) [17:57:22] ostriches: when the rate is this high, it should be looked at IMO [17:59:09] T144836 [17:59:10] T144836: Redis constantly doing duplicate redis fetches - https://phabricator.wikimedia.org/T144836 [17:59:17] Whoops [17:59:47] s/^Redis/Echo/ [19:42:59] anomie: any thoughts on how to split the Q2 API tasks? [19:45:47] tgr: I'll likely wind up writing at least the first pass at code for the ORES stuff. We haven't heard anything on the pageviews stuff as far as what their side of things looks like. [19:46:19] bd808: https://gerrit.wikimedia.org/r/#/c/308808/ [19:51:28] Heh. https://phabricator.wikimedia.org/P3984 [19:51:29] AaronSchulz: ^ [19:51:40] anomie: if you mean the REST API side, that's https://wikimedia.org/api/rest_v1/#/Pageviews_data - you can query a single page for a list of daily view counts [19:52:28] ostriches: sampled logs are excluded from Logstash via global stuff in wmf-config/logging.php. There was no way to sanely sample two different output streams. [19:52:48] Ah, so I'd have to remote the sampling? [19:52:53] so you can have sampling or logstash but not both [19:52:58] yeah [19:53:17] Eh, I've probably got enough data from fluorine anyway [20:23:05] bd808: why not sample them? You mean because the aggregate of multi-channel logs would have a meaningless rate? [20:23:09] or some interface limitation? [20:24:37] AaronSchulz: I/we couldn't come up with a single sample variable to share across the udp2log and logstash outputs. Getting different logs on each side is too confusing to be useful. [20:25:15] it would have to be done with a filter that was common to both outputs and ignore the built in stuff in the legacy udp2log output handling chain [20:26:17] and at the time I made that config change logstash was melting regularly under the log volume we already had, so adding more wasn't an exciting prospect [20:29:50] AaronSchulz: With current sampling on that log, I'm getting 7927 / 12h [21:24:53] Updated https://phabricator.wikimedia.org/P3984 with counts [21:25:07] 1411 Linker::formatTemplates/Message->parse/Message->toString/Message->parseText/MessageCache->parse [21:25:07] 1896 ChangeTags::tagDescription/Message->parse/Message->toString/Message->parseText/MessageCache->parse [21:25:11] The absolute worst ^ [21:31:55] I think formatTemplate shows up on most edit views. Also, all callers should have a context available [21:32:58] formatTemplate() doesn't have a Context. [21:33:12] And RequestContext::getMain() just pretends its not a problem [21:34:07] I meant that the callers of formatTemplate are InfoAction and EditPage, which both have a context [21:34:35] * ostriches looks for any extensions [21:34:36] :) [21:34:41] If not, I'll refactor that whole function [21:34:56] If you move it out of Linker, that would be even better :) [21:35:50] Yay no extensions [21:56:08] legoktm: is there a username usurpation policy? Outside of what happened for SUL finalization is that even a possibility? [21:56:49] bd808: still in discussion https://meta.wikimedia.org/wiki/Requests_for_comment/Global_usurpation_policy_2 [21:57:05] https://meta.wikimedia.org/wiki/Global_rename_policy#Usurpation "case by case basis" [21:57:08] awesome. thanks [21:57:28] * bd808 is researching things before trying to reboot https://meta.wikimedia.org/wiki/Requests_for_comment/Abandoned_Labs_tools [22:24:08] I felt that that discussion was overly bureaucratic [22:26:15] the most important things to consider for me are 1) will giving access to the tool also give access to an on-wiki account 2) is the person who is getting access sufficiently trusted and competent and balancing that with new volunteers who just want to rescue a tool they depend upon [22:27:55] maybe a starting point could be having a way for tool owners to say "I authorize tool labs admins to use their judgement in granting access to my tool to other users in cases of need blah blah" on a per-tool basis [22:28:46] (AIUI that is the de facto process anyways) [22:30:41] bd808: ^ [22:37:41] legoktm: *nod* there are a lot of layers to this sadly (or not sadly as the case may be) [22:38:38] the shared credentials thing is the big potential problem. Giving someone control of a SUL account is wacky [22:49:15] bd808: https://gerrit.wikimedia.org/r/#/c/308892/ quick IDEA warning fixes :) [23:06:39] legoktm: Linker::formatTemplates() is a rabbit hole of nasty [23:06:56] how? [23:07:48] (also, that's true of all of linker :P) [23:08:58] message calls far as the eye can see :p [23:09:01] All needing a title [23:09:06] Because parse()! [23:09:09] Or escaped() [23:09:10] :D [23:09:43] Use $context->msg() ? [23:15:42] Perhaps Linker things should require contexts all over the place [23:21:51] Well I'm currently working on splitting things out of linker [23:22:21] So we could just split formatTemplates into its own class that requires a context [23:52:25] ostriches: we are deploying new wikibase code tomorrow, which i think contains some changes that will help [23:52:51] yay