[03:34:30] bd808: do you think it would be doable to create a log channel for all events that happen on mw1017? [03:34:35] legoktm: https://github.com/wikimedia/composer-merge-plugin/pull/84 is blocking my grand refactoring patch which is blocking your new feature request :) [03:34:42] I guess that's not really a 'channel' [03:35:13] hmmm... I'm not sure how we would hack it into the config [03:35:43] I could see it being useful but "all events" would be a metric ton [03:36:19] or is that server only hit when you use the debug header? [03:36:52] we have a "log all the things" file for testwiki and test2wiki [03:37:16] but not into logstash because it really has enough junk events already [03:38:58] wrap everything in a GroupHandler? [03:39:24] I don't know if it's included in normal load balancing [03:39:51] it would be fine if it is a file like testwiki.log [03:40:09] There's no normal way to set config settings based on physical host [03:40:10] but some stuff cannot be tested on testwiki [03:41:15] we could probably figure out how to hack it in. we might even be able to tie it to the special debug header if needed [03:42:11] CommonSettings.php seems to use gethostname() already [03:44:14] I wonder if unique debug IDs could also be tied to the debug header [03:44:35] user tracking!!!! [03:44:38] that would avoid the privacy aspects and make debugging quite a bit less painful [03:45:06] well, as long as you provide the debug header you can be reasonably assumed to consent to it [03:45:17] :) agred [03:45:28] +e [03:46:28] I will take a shot at it over the weekend if you think it is a sane plan [03:46:48] for now I think I figured out a workaround [03:46:50] the way that testwiki.log is created is by setting the default logging location to a udp2log route rather than the null handler [03:47:03] so it's just one global to change [03:47:22] yes, but that still doesn't include the dedicated channels, right? [03:47:28] right [03:47:40] they go whevere they normally go [03:47:57] changing that would be ... tricky [03:48:19] unless you just unset all the other channels I guess [03:48:24] wrapping everything in a GroupHandler when some global is set does not seem horrible [03:48:59] s/everything/some things/ [03:49:20] you lose all the nonstandard processors/formatters, but that does not sound that bad [03:51:20] just adding another handler to each logger would probably be easier [03:51:50] $wmfLogAllTheThingsHandler [03:52:46] actually I should fix it so that the testwiki.log does that too [03:53:03] rather than just getting the non-handled channels [03:54:00] make a task about it if you haven't already [03:54:06] * bd808 will forget [04:25:18] bd808: https://phabricator.wikimedia.org/T117019 https://phabricator.wikimedia.org/T117020 https://phabricator.wikimedia.org/T117021 [04:25:37] probably not the right project, did not find anything non-MediaWiki-specific [04:25:52] it's good enough [04:26:36] the right project I guess would be site-requests but meh [04:27:53] the id thing... we actually have that in logstash logs it's just not passed back to the browser [04:28:06] and it doesn't propigate [04:29:06] passing it back to the browser is important though [04:29:26] or taking it from the browser in the first place, that would work too [04:29:48] without that you still have to locate at least one record by hand [04:29:58] *nod* [04:29:59] not that hard but time-consuming [04:30:46] the "uid" we get on logstash logs isn't very unique either really [04:31:19] it's just this -- https://github.com/Seldaek/monolog/blob/master/src/Monolog/Processor/UidProcessor.php [04:32:03] 7 random hex chars [04:33:56] if we did the header outside of the opt-in debug requests we would need to strip it before caching in varnish I suppose [04:35:39] if you do it for cached requests, you might want it to be cached [04:35:49] otherwise it might be hard to access [04:36:06] but X-Wikimedia-Debug requests avoid varnish anyway [05:47:19] legoktm: I have a fix for your require-dev issue, but I did a bunch of other things too. I have 3 PRs worth of stuff (one submitted already). I can either wait for you to review and merge each one in series or I can make one mega PR for all 3. Which would be less painful for you to review? [05:47:49] smaller ones [05:47:53] k [05:48:17] It really sucks how github doesn't let you do dependencies between PRs [05:48:17] then you have one now and I can post the others as you merge [05:48:22] yeah [05:48:26] if only we had a code review system that let you do that... [05:48:41] I want to move CR for this to phab [05:49:03] but I haven't talked to anyone about that yet [05:49:13] * bd808 makes a note to do so [05:50:18] is CI working with phab yet? [05:50:39] yes, in stealth mode [05:50:49] scap3 is using it as dogfooding [05:51:06] oh... ci I actually don't know [05:51:26] They may be testing manually [05:52:25] merged [05:56:37] next one posted. [05:57:41] I need to head to bed. You feature request is after the refactor PR I just posted [13:10:28] https://gerrit.wikimedia.org/r/243945 is up for merging [13:12:55] * hoo presses +2 and runs away [13:13:41] Well, gerrit doesn't even load for me right now, so no idea what that actually is... mobile internet [13:14:08] ah, it's the ORM thing! :) [13:14:35] lolol [14:17:01] Reedy: is the patch to the education extension already merged? [14:17:09] bd808: Nope [14:17:21] probably should do it first eh? [14:17:32] I was waiting for the core one to be done, force recheck unit tests, and then merge that if it passes [14:17:44] ok [14:19:52] Reedy: poke me when the other one is ready [14:19:54] Probably doesn't make a big difference, just need both merging in same WMF deployment cycle [14:20:11] bd808: It should be ready to go bar the recheck and merge. Doesn't need rebasing [14:20:41] I wonder if we have anything that uses EducationProgram in the beta cluster? [14:21:23] * bd808 admits to not knowing exactly what it does, just that the code is hated [14:21:57] I think there's some test instances, but I think they were seperate dev intances [14:22:00] Sage had one IIRC [14:22:44] *nod* poking Sage to test might be a good idea [14:23:22] * bd808 twiddles thumbs while the zend core tests poke along [14:23:25] I used the phpstorm static analysis to make sure any class references were properly globalised, or changed for imports etc [14:23:46] So as long as there's not anything overly dynamic (which the unit tests did find a couple) [14:23:59] Gotta go out, back in probably an hour or so :) [14:24:00] Thanks! [14:24:02] o/ [15:47:50] Reedy: both merged! I had to fix one dynamic class reference in the EducationProgram patch to use the new namespace [16:34:37] bd808: woo, cheers [17:43:23] AaronSchulz: wanna +2 https://gerrit.wikimedia.org/r/#/c/249475/ ? [17:49:31] after I faff around with some stuff I'll look [18:10:58] AaronSchulz: Do you know if lowerCdnMaxage( 0 ) is the right way to disable all frontend caching for a page? Or who would know nowadays? [18:15:46] csteipp: usually setSquidMaxage( 0 ) [18:58:08] bd808, tgr|away: Did we decide on what namespace to use for AuthManager, and if we're going to do anything to shorten up the class names? I forget. [19:50:21] anomie: I don't think we had absolutely decided... \MediaWiki\Auth or Authentication would work I think. Auth if you feel that the eventual authz stuff could slide into the same namespace [19:50:31] ok [19:50:45] * anomie is massively rebasifying [19:50:53] if you used Authentication then you could probably drop that from most of the class names [19:52:16] anomie: by the way, I merged Reedy's patches that moved ORM to the EdProgram extension today \o/ [19:52:21] \o/ [20:10:32] bd808: is there a way to get an unsanitized version of an exception trace? [20:10:59] not if we did things correctly, no [20:11:00] in production I mean [20:11:06] in a log I assume you mean [20:11:18] yes [20:12:48] You could hack one in by not putting the exception in the context and instead directly generating the trace, but it wouldn't be a great idea. [20:15:26] can you add some logging somewhere else to get the data you want? [20:17:18] I'm sure I could, I just don't want to mess around with production code [20:17:41] but I can probably figure it out without that, it would just have made things faster [20:38:57] bd808: where do PHP warnings end up? the plain debug log? [20:39:19] hhvm.log [20:39:43] with no stack traces much to teh sadness of Krinkle and Reedy [20:40:27] doesn't seem to happen in this case [20:40:50] that, or some preg_match_all error conditions don't result in a warning [20:41:01] hardly any of them do [20:42:59] the ones I could think of seem to have decent error messages at least in HHVM [20:43:22] e.g. [20:43:23] > var_dump(preg_match_all('/x[/iuS', "\xc3\xb1", $matches)) [20:43:24] Warning: preg_match_all(): Compilation failed: missing terminating ] for character class at offset 2 [20:44:32] aren't you supposed to check for false and the hit preg_last_error? [20:46:36] you are supposed to but we don't [20:46:54] but yeah that's an easier route than trying to guess what the error is [20:50:09] Sigh. Namespacing breaks all the unit tests... [20:51:23] I remember somebody talking about a lib to make regex stuff work nicely... like that we should make one and use it everywhere [20:52:47] legoktm, about using something like Wikimedia\preg_match, I believe [20:53:05] safe_preg_* yes [20:53:10] * ori raises hand. [20:53:14] legoktm: can you look at https://gerrit.wikimedia.org/r/#/c/249873/ ? [20:53:17] https://phabricator.wikimedia.org/T115639 [20:53:49] A.aron distracted me with wfBaseConvert(), it's next on my list [20:54:29] tgr: reading the php.net docs (which is not as definitive as reading the hhvm source), it looks to me like the wanrings are emitted for regex compilation fails and the false + error code are for errors running the compiled regex [20:55:11] that's probably right, I tested the regexps with eval.php and there was no compilation problem [20:55:31] the error codes a practically useless too [20:55:38] tgr: oh, your lines are too long [20:55:55] backtrack, recursion, utf8, and "internal" [20:55:56] duh [20:56:00] give me a sec [20:56:15] I should really put phpcs into a local git hook [20:56:33] or wire it into your editor [20:56:40] * bd808 hasn't done that yet [21:02:00] * anomie set up vim to highlight long lines, but it doesn't do the full phpcs thing [21:04:00] I have the syntastic vim plugin setup for pep8 in python on save but only lint for PHP right now [21:05:15] can that limit the tests to changed files? a full phpcs test run on core takes minutes [21:05:31] yeah it just runs the linter on buffer save [21:05:36] for that buffer [21:06:35] * bd808 might make tomorrow afternoon a tools tuneup time [21:20:32] bd808: What's blokcing having stacktraces for notices and warnings [21:20:35] I thought we fixed this months ago [21:20:38] logstahs is back up [21:20:44] Things are good now. What's breaking it? [21:21:11] Even more so for fatals/exceptions, do we have those back? [21:21:15] Not sure, the log size on fluorine? [21:21:29] fatal.log is good now [21:21:34] it has full traces [21:22:14] as I recall the warning thing was just that it was too much data [21:45:56] legoktm: https://gerrit.wikimedia.org/r/#/c/246719/ is throwing an exception when there is broken UTF-8 in the input [21:46:04] any objection to reverting? [21:46:06] hmm [21:46:09] no objection [21:46:26] is broken utf-8 supposed to make it that far? [21:46:39] also I wonder why we get a crappy error page like https://commons.wikimedia.org/wiki/File:Georg_August_Samuel_von_Nassau-Idstein.jpg [21:46:49] at least the exception id should be included [21:47:46] is it a non-MW exception? [21:48:00] yeah, bare [21:48:27] the stack trace does include the id though [22:03:42] bd808: could you set up https://github.com/wikimedia/base-convert/settings/hooks/new?service=packagist please and register it with packagist? [22:04:00] yup [22:05:09] legoktm: did you look at https://github.com/ocubom/base-convert ? [22:05:25] packagist is warning me about the similar names [22:05:30] I did not [22:05:41] I just moved the existing code out [22:05:51] *nod* [22:06:31] that doesn't have a pure php fallback :/ [22:07:08] https://packagist.org/packages/wikimedia/base-convert [22:07:21] I'll do travis while I'm here too [22:07:43] thanks :D [22:09:02] https://travis-ci.org/wikimedia/base-convert/builds/88225177 [22:09:07] build just started [22:09:17] How do we see the list of all functions that are run for a hook nowadays? var_dump( $wgHooks ) doesn't look right... [22:09:37] legoktm: ... we should add php7 to the .travis.yaml file(s) [22:10:10] csteipp: everything should still be in the $wgHooks global after MW initializes... [22:11:09] bd808: just 7.0? [22:11:13] yeah [22:11:21] that's how travis labels it now [22:11:29] it used to be "nightly" [22:11:56] * bd808 is waiting for the OMG PHP7 thread on wikitch-l [22:12:12] bd808: https://gerrit.wikimedia.org/r/249904 [22:12:18] is that librarizification of wfBaseConvert() ? :-) [22:12:33] yep :) [22:13:24] legoktm: please to IP next [22:13:25] die die GlobalFunctions.php ;-} [22:13:47] the IP class is a great one as well. I reused it a lot [22:15:05] AaronSchulz: where should the trusted proxy/configured proxy functions go? [22:15:48] wfSleep( $hashar ); [22:15:59] I assume they'd stay in the core /IP and the new class would be a namespaced/differently-named [22:16:41] I guess they could find a more permanent home, but I mostly just care about getting more stuff in /libs or repos and lots of stuff needs a thing or two from IP right now [22:17:53] eh, I forgot the .gitattributes file. [22:18:14] shame [22:18:26] it's always something [22:18:51] * AaronSchulz wonders if php 6 will have windows 9 support [22:19:46] php6 will never exist [22:19:51] they jumped to php7 [22:19:56] you missed his joke [22:20:20] windows 22 will be written in php6 [22:20:23] that I'm sure of [22:20:24] * bd808 imagines it was some M$ thing [22:37:27] windows 9 will never exist either ;) [22:38:35] oh yea of little faith -- did you forget nostalgia windows they will roll in 5 years or so [22:39:21] when mullets and 80's rock is the hipster fashion and president bieber is waging war on antarctica [22:39:41] obv the age will be lowered so he can take office [22:43:18] DUKE NUKEM FOREVER [22:43:45] that actually shipped at some point thought right? [22:44:39] hell perl6 might even ship [22:46:19] in some dimension it already has [22:56:50] TimStarling: I have a very small follow-up for Scribunto to make the percentile threshold for logging configurable via $wgScribuntoSlowFunctionThreshold -- https://gerrit.wikimedia.org/r/#/c/249915/ . This is the last nag, I promise. [23:54:53] hey dapatrick, I'm reviewing the revscoring stuff. Do the ores celery workers read pickled stuff from redis and load them? [23:55:49] My understading of the code is that pickled models are only read from the filesystem. [23:56:16] That was also my understanding from the orientation phone call.