[00:00:33] umm [00:00:41] you should see the traceback in your terminal? [00:10:16] Nope [00:10:18] (gotta go) [02:31:12] anyone want to review a couple of extra wfIncrStats() calls before I self-merge and deploy it? https://gerrit.wikimedia.org/r/#/c/243094/ [02:33:24] TimStarling: {{done}}; deploy away [02:33:32] thanks [16:01:34] * anomie is feeling curmudgeony this week [16:14:49] anomie: Watch the movie Cocoon and imagine yourself infused with youthful vigor ;) [16:17:20] ori: I have a patch to backport the hidden series bug fix to our grafana. Any good idea on how to get that reviewed, tested and applied in prod? It looks like that repo has mostly been you importing the master at various points and then building the combined js files. [16:17:49] i could do it for you if you like [16:18:19] rebuild grafana with the backported patch, i mean -- not watch the movie Cocoon [16:18:27] once was enough [16:19:06] if you feel pretty confident about it, you can also just push it directory [16:19:10] awesome. How sould I get the patch to you? Phab paste? [16:19:15] that works [16:20:46] https://phabricator.wikimedia.org/P2144 [16:29:47] bd808: done; try it? [16:30:50] hmmm... https://grafana.wikimedia.org/#/dashboard/db/authentication-metrics?panelId=14&fullscreen still looks messed up [16:31:50] is there a button to get grafana to show you the graphite query it is building? [16:32:17] well, you can check the network pane in your browser dev tools [16:32:22] and reverse-engineer it from the query string [16:32:39] what is the bug exactly? [16:33:09] also i don't know the caching behavior exactly, it is behind varnish [16:33:53] If you open the edit view for that panel I linked, you can see that there are a bunch of series that are built up but hidden so that they can be used in asPercent() series [16:34:42] the bug is that the hidden series are sent to graphite as targets which makes grafana compute the wrong data in its graph [16:35:18] if you unhide the hidden series and set them to the right axis you can see that the percentages end up calculated correctly [16:35:44] that patch is supposed to stop send the hidden things as separate targets to graphite [16:36:28] fwiw, if you want this done for the end of the quarter and don't want to troubleshoot, there is always tessera, which is still provisioned [16:36:39] though i imagine it'd be more satisfying to get grafana to just bloody work [16:39:25] I'll poke at it a bit and see if I can guess what's wrong [16:40:04] as we've talked about before, all graphing tools seem to be horrible [16:40:13] much like all bug trackers adn all code review systems [16:40:15] yep [16:41:08] AaronSchulz: instead of having configuration variables like $wgLanguageConverterCacheType = CACHE_ANYTHING, etc., wouldn't it be simpler to have DefaultSettings.php $wgObjectCaches['LanguageConverter'] = array( 'factory' => 'ObjectCache::newAnything' ), and then you can just override it? [16:51:16] ah ha. it is not expanding the "#A" series references into the actual series definition when the series is hidden [17:05:03] bd808: Cocoon! it's been so long.... That and "Batteries Not Included". I put those in the same bucket for some reason. [17:08:18] "crappy 80s movies that you saw on late night tv" ? [17:43:08] bd808: I was a youngin, so, maybe on HBO during the day [17:44:04] also, batteries not included wasn't crappy! if it was crappy so was Coming to America [17:44:08] (yes, yes it was) [17:48:36] I'm not sure if Eddie Murphy has been in a non-crappy movie other than Trading Places [18:21:39] bd808, tgr|away: https://gerrit.wikimedia.org/r/#/c/243223/ ! [18:22:12] cool [18:22:47] is there a phab task for that sub component? If not we should probably make one [18:23:09] (see random discussion with TPG about "tracking things") [18:23:13] Not that I know of, but I haven't really looked. [18:23:36] gergo made a huge pile of phab tasks so there might be one in there somewhere [18:24:21] Feel free to add the task to the commit message once you find/make one, I'll pick it up. [18:24:31] *nod* [18:24:56] Also see the last bullet point under TODO there, I could use input on that one. [18:25:37] yeah. such bikeshed. [18:28:24] * anomie tried to run phpcs locally, but it whined about vendor/mediawiki/mediawiki-codesniffer/MediaWiki not existing [18:41:45] https://integration.wikimedia.org/ci/job/mediawiki-phpunit-hhvm/16566/console is informative [19:13:56] no task for this specifically although T111305 is somewhat related [19:20:06] woah, "Batteries Not included" and "Cocoon". I put those in the same bucket as "Short Circuit" and "Fried Green Tomatoes at the Whistlestop Cafe" and "Driving Miss Daisy". (Things that I enjoyed when I watched them with mom or grandad at a fairly young age, and don't touch again for fear of changing the vague memories) [19:20:50] (also a suspect a lot of actor crossover, between those 5) [19:30:01] so, to keep off topic: Please recommend a good movie I should watch this weekend, my wife and kid are out of town and I'm bored/don't know what to do with myself. [19:36:45] bd808: can you peek at https://gerrit.wikimedia.org/r/#/c/242804/ ? That was driving me crazy [19:37:39] * bd808 would love to just delete that class [19:38:59] AaronSchulz: I'll test it out after I eat some lunch. too nasty to think about with low blood sugar [19:40:04] Reedy: ping https://gerrit.wikimedia.org/r/#/c/87269 [20:19:08] AaronSchulz: I can fix the message, and add the extra ping easy enough... [20:20:45] Not sure about getting hte right messages in the right places [20:31:11] * AaronSchulz made a comment [20:38:17] ori: I'll be in a bit after 2 [20:38:23] that's fine [20:56:57] AaronSchulz: heh. I was having trouble testing your patch and then I realized that my wiki is using monolog and that keeps it from calling MWDebug::debugMsg() at all. [20:57:10] * bd808 should file a bug about that I guess [21:24:29] anomie: guess I have missed you ( re https://gerrit.wikimedia.org/r/#/c/243223/ being killed in Jenkins ) [21:26:09] anomie: rerunning the test with Zend instead of the default HHVM [21:36:04] anomie: anyway I gave a bunch of comments on https://phabricator.wikimedia.org/T114522 [21:43:46] anomie: and maybe we could have used a 3rd party lib to replace session management but I am diverging :D [21:44:04] and it looks like a HHVM memory leak of some sort :-((((( [21:47:54] hashar: session management is very often a "library" thing as much as a "framework" thing. I haven't looked at the symfony component for it but I would be willing to bet that they will have a few narrow interfaces we could use and all the real work would need to be done from scratch [21:48:46] native php sessions are "easy" but anything else gets into tightly coupled implementation details of were you are storing things and how you talk to that datastore [21:49:29] s/is/isn't/ [21:50:34] bd808: yeah I am pretty sure there is a good reason for it [21:50:42] and we probably need unique features [21:50:55] + Symfony would definitely need a more recent version of PHP [21:51:30] I was merely preaching the false to get the trueness. But the real trouble is the code seems to dramatically slowdown the PHPUnit suite [21:51:38] they have old branches that support 5.3 still [21:51:42] and seems to hit a mem leak bug in HHVM :-/ [21:52:13] making phpunit even slower than it is normally is not fun :/ [21:52:48] oh that part does not worry me [21:53:03] but that might indicate a potential huge performance hit on prod sites :-\\\\ [21:53:19] that can probably be profiled anyway and the code is a first version [21:53:39] maybe it is just a single mistake [21:54:32] the core PHPUnit test suite we have been talking about it a bit in #releng . Not sure what to do with it but something will need to happen eventually. We will see :} [21:55:09] just let Chad start deleting crappy tests. He's good at that :) [21:55:40] s/tests/stuff/ [21:55:43] ^ more accurate [21:56:49] I bet we could delete 30% of mw-core without getting rid of anything that is actually used [21:57:03] for certain values of actually used I suppose [22:01:40] Find a target and fire at will! [22:04:43] well [22:04:53] we could get rid of the parsertests wrapped in PHPUnit [22:05:09] and fallback to the good old parser testing script which is still around [22:05:15] I filled a task about it [22:05:39] but yeah that would probably need a nice focused team to clean it up [22:05:40] :-/ [22:06:33] I was thinking more about things that are in core only to support WikiBase or the education extension [22:07:09] plus there are still a ton of things we should break out into separate libs [22:07:23] which makes them oh so much more testable [22:09:21] well [22:09:23] Yeah, there's the ORM stuff that should go from core [22:09:46] the ORM stuff was merged in core because it was used by several of Jeroen extensions [22:09:54] Yeah. That's my fault [22:10:03] I am not sure whether anyone relied on it, but the decision made sense at the time [22:10:05] includes/site [22:10:08] It was to prevent some head bagning against the wall [22:10:10] I don't like the old parser test, it would be nice to get it running not insanely in PHPUnit instead. [22:10:32] now that we have composer / vendor ... the ORM stuff could be made a standalone stuff and I am sure Jeroen will be happy to do it [22:10:41] ostriches: what is wrong with the old one ? [22:11:03] I just don't like having our own DIY framework for testing [22:11:08] well [22:11:09] And having to test things in 2 places [22:11:25] the PHPUnit wrapped one is a merely copy pasting of the old one. So it is still more or less a DIY :-} [22:11:32] but I got your point indeed [22:11:37] Yeah, because nobody would let me get rid of the old one :p [22:12:02] there is surely a bunch of features in it that are of no use, such as the ability to change the filebackend from the command line [22:12:53] if you have ideas, we can fill them in Phabricator under some tracking task [22:14:14] thanks AaronSchulz :) [22:14:53] We should file some tasks about removing some of this stuff from core... [22:17:17] ORM task is https://phabricator.wikimedia.org/T114538 [22:18:15] baby steps :-} [22:20:04] Another thing ppl can do is find unit tests that don't actually rely on weird global manipulation or databases and make them subclass PHPUnit_Framework_TestCase or w/e it is instead of our silly wrapper stuff. [22:20:13] Saves a ton of overhead and makes it much more...unit-y [22:20:49] yeah [22:21:02] I had a patch a year or so ago after a nightly discussion with bryan [22:21:23] the idea was to introduce /tests/phpunit/unit and a /phpunit.xml at the root of the repo [22:21:37] then migrate anything that did not rely on mw being installed [22:21:50] such as the unit tests for some global functions or the IP class [22:22:27] * AaronSchulz hands bd808 https://gerrit.wikimedia.org/r/#/c/243054/ [22:22:46] ostriches: https://gerrit.wikimedia.org/r/#/c/178696/ https://phabricator.wikimedia.org/T87781 [22:23:59] I don't think we need a 2nd phpunit.xml or a new directory [22:24:03] Just clean up the tests we have [22:24:28] bd808: not sure I like how that trip DB connections. Maybe we should have a separate readonly method that only checks the globals and the file [22:24:48] ostriches: well that was some midnight still commit so ... [22:25:19] ostriches: I guess we can define a unit test suite in /tests/phpunit/suite.xml and introduce a new unit sub dir there [22:25:27] then it is all about copy pasting [22:26:33] anyway sleep() have fun! [22:26:49] AaronSchulz: you lost me somewhere [22:27:24] * bd808 looks at FileBackendGroup for possibly the first time [22:28:59] heh [22:29:33] bd808: wfReadOnly gets a DB_SLAVE connection if needed [22:30:08] but for that case I really just want to check the actual config probably [22:31:32] so what you have added there will pretend that a backend was configured as readonly if wfReadOnlyReason() returns anything other than false? [22:31:48] what does that fix in core? [22:31:58] * bd808 really doesn't know about this stuff [22:32:48] readonly can happen from a touched file, or too much lag it looks like [22:33:04] yep [22:33:06] don't we have a $wg for it too? [22:33:24] $wgReadOnly should make file backends read-only too [22:33:24] yeah $wgReadOnly [22:33:41] ah ok. but many not the slave lag path? [22:33:47] *maybe [22:34:19] hence your first cryptic message that got my attention [22:35:26] so would it be better to just populate from $wgReadOnly directly after calling wfReadOnly() to ensure that it is populated? [22:35:46] oh wait that sill sets for the slave lag issue [22:35:57] *still (friday typing) [22:36:12] * AaronSchulz could just add a wfConfiguredReadOnlyReason() [22:37:44] which would just do the first if/else block from wfReadOnlyReason() right? [22:38:14] which you could static cache in the method to keep from making yet another damn global [22:38:28] and then use in wfReadOnlyReason() [22:38:54] yep [22:38:59] all of which could be used in non-master DC to keep things from accidentally writing [22:39:10] * bd808 guesses [23:27:43] ori: that patch did fix the dashboard! Apparently I got cached js from varnish when I tested this morning [23:52:10] tgr: the good news for AuthManager is we could hardly make the failure rate for account creations worse :/ [23:52:22] https://grafana.wikimedia.org/#/dashboard/db/authentication-metrics?panelId=14&fullscreen [23:53:36] 600% error rate? [23:53:45] what does that mean? [23:54:01] you need to force reload grafana to get a bug fix for the js [23:54:07] it's ~85% [23:54:29] heh, ok, but golly that's still high [23:54:50] yeah, but maybe there are that many dumb spam bots I guess [23:54:54] it's mostly captcha errors [23:55:40] login error is wrong user/pass? [23:55:52] and there is a magnitude more of "user was shown a captcha and went away" which doesn't register in the error rate, so actually it's more like 98% [23:55:59] ah. so we are kind of double counting captcha failures in that report? [23:56:09] * greg-g is trying to make sense of that [23:56:32] but yeah, could be easily spambots trying to solve the captcha and failing [23:56:50] we are not double-counting them AFAIK [23:57:14] the curves look really really similar [23:57:23] but compare captcha display with captcha success/error in the overview graph [23:57:26] but that may just be pattern bias [23:57:48] they are counted as failures if that's what you mean [23:58:03] and make up for the overwhelming majority of failures [23:58:15] that's what I meant, yeah [23:58:26] everything where you submit the usercreate form and the user is not created counts as error [23:58:51] for the API that gives weird results because you are supposed to fail a few times [23:59:07] for web login, ideally you would immediately succeed