[02:27:35] ori: https://gerrit.wikimedia.org/r/#/c/205998/ [03:19:52] lots of good ideas here: http://appserver.io/get-started/documentation.html [03:30:20] ori: https://gerrit.wikimedia.org/r/206034 & https://gerrit.wikimedia.org/r/205989 :) [03:33:04] hmm [03:39:32] Krinkle: maybe we should just make EventLogging shut up for qunit test runs? [03:40:09] This is fine too, I guess. But why did you choose this route? [03:40:31] Also, I was going to spend a bit of time poking at the unit test execution time. How are you profiling individual tests, again? [03:41:45] ori: Well, it seems desirable to be more explicit about what to hide. [03:41:47] * ori is checking out https://github.com/phpunit/phpunit-testlistener-xhprof/blob/master/src/XHProfTestListener.php [03:41:51] Errors in the output don't hurt [03:41:55] but if you don't expect them.. [03:42:24] I don't know EL well enough to know that if any other test emits an error it will show in the unit tests themselves. [03:42:38] If that's teh case, we can surrpress/restore from setup/teardown in the EL test module [03:43:55] yeah, let me do that [03:44:00] i think i'd want that regardless [03:44:06] OK [03:44:12] assuming that works, what would you want to do with the two patches you linked to above? [03:44:46] ori: merge the mediawik-core one first, then amend the EL one to have the warning suppression where you want it. [03:46:56] we could also decide that extensions should not interact with the console directly (as EL is currently doing) and provide an mw.track-based means for extensions to log some message and express the intent that it should be shown in the console, if MediaWiki decides it's not inappropriate [03:47:18] in other words, something like mw.trackSubscribe( 'eventlogging.error', function ( topic, error ) { console.error( error ); } ); [03:47:30] except a little subtler and not coupled to eventlogging [03:48:47] basic have a topic pattern that mediawiki flushes to the console by default [03:48:50] *basically [03:49:26] as the saying goes: "All problems in computer science can be solved by another level of indirection" [03:50:05] mw.log.warn? [03:50:09] in fact the topic names could simply be 'console.log', 'console.error', 'console.warn', etc. [03:51:21] If EL emits "eventlogging.error" and binds it to the console, how do we prevent that in test runs? [03:51:32] mock mw.track? [03:51:40] it wouldn't bind it to the console [03:51:50] maybe it should use mw.log.warn instead [03:52:15] console.error is special in that it handles Error objects and stacktraces [03:52:39] mw.log.warn uses console.warn (string based) and calls console.trace which has a different stack [03:52:46] what I was suggesting a moment ago was something like this: [03:53:08] mw.trackSubscribe( 'eventlogging.error', function ( topic, error ) { mw.track( 'console.error', error ); } ); [03:53:32] Right [03:53:40] Hm.. [03:53:48] that keeps everything in memory though [03:53:51] and then in mediawiki you'd have something that calls mw.trackSubscribe( 'console.error', ... ) [03:54:12] well, if mediawiki had this console.error channel [03:54:16] eventlogging would use that directly [03:54:20] rather than have eventlogging.error, I think [03:54:24] actually, no, that's not true [03:54:38] there are snippets around for binding custom subscribers to eventlogging.error and displaying them prominently [03:54:58] Hm.. let's add mw.log.error maybe? [03:55:00] and use that [03:55:13] yeah, wfm and consistent with existing code [03:55:38] OK. I'll update the core patch [03:55:43] thanks [04:07:58] ori: Done. [04:08:29] ori: It still logs to console.error properly (after the core patch and the first EL patch) and with the second EL patch applied it logs nothing during qunit [04:08:34] Tested with wiki/Special:JavaScriptTest/qunit/plain?hidepassed&module=ext.eventLogging&debug=true [10:17:30] 6MediaWiki-API-Team, 10MediaWiki-extensions-CentralAuth, 10SUL-Finalization, 5Patch-For-Review: SULF renameuser jobs slow on mediawikiwiki and queries timing out - https://phabricator.wikimedia.org/T96489#1230846 (10Aklapper) So should https://gerrit.wikimedia.org/r/#/c/205070/ get reviewed/merged? Or doe... [14:42:39] <^d> manybubbles: https://gerrit.wikimedia.org/r/#/c/204310/ is pretty trivial :) [14:43:02] ^d: sorry - yeah [14:44:23] <^d> manybubbles: In retrospect, I'm wondering if snapshot/restore isn't quite going to work for our backups well. How does it handle index renames? [14:44:42] I suspect you know more about it then I do [14:44:42] <^d> ie: if we rebuild enwiki and it has a new name, does the old backup become useless to build a delta against? [14:44:53] I never really reviewed its code beyond making sure your plugin made sense [14:44:54] <^d> It supposedly tracks aliases [14:45:14] * ^d shrugs [14:45:34] <^d> $things_i_dont_have_time_for_right_now++; [14:45:36] <^d> :) [14:46:01] <^d> getting close to a buffer overflow there [14:46:02] <^d> :p [14:57:01] lol [15:20:01] 6MediaWiki-API-Team, 10Analytics, 6Analytics-Kanban, 10MediaWiki-Authentication-and-authorization, 5Patch-For-Review: Create dashboard to track key authentication metrics before, during and after AuthManager rollout - https://phabricator.wikimedia.org/T91701#1231302 (10Milimetric) [16:14:09] 6MediaWiki-API-Team, 10Incident-20150205-SiteOutage, 10MediaWiki-Debug-Logging, 10Wikimedia-Logstash, and 2 others: Decouple logging infrastructure failures from MediaWiki logging - https://phabricator.wikimedia.org/T88732#1231446 (10bd808) [16:19:51] 10MediaWiki-Core-Team, 7Performance: [draft] Performance Roadmap April - June 2015 (Q4 2014/2015) - https://phabricator.wikimedia.org/T93845#1231470 (10bd808) a:3ori [16:21:44] 6MediaWiki-API-Team, 10Incident-20150205-SiteOutage, 10MediaWiki-Debug-Logging, 10Wikimedia-Logstash, and 2 others: Decouple logging infrastructure failures from MediaWiki logging - https://phabricator.wikimedia.org/T88732#1231480 (10bd808) I have a new an improved version of the mediawiki-config patch ( dancecat: https://phabricator.wikimedia.org/T97046 might be interesting to you :) [17:53:50] 6MediaWiki-API-Team, 10MediaWiki-extensions-CentralAuth, 10SUL-Finalization, 5Patch-For-Review: SULF renameuser jobs slow on mediawikiwiki and queries timing out - https://phabricator.wikimedia.org/T96489#1231843 (10csteipp) The majority of renaming on mw.org is done now, so the impact won't be huge. If it... [18:50:18] bd808: git reset --hard fixed my missing vector pngs! [18:50:27] * dancecat wonders wtf happened [19:00:58] dancecat: weird. reset on mw/core.git or on the vector skin? [19:04:10] in vector skin [21:20:59] bd808: > Meaning that there should always be a blackhole/null handler added magically by the Monolog SPI? [21:21:06] Yeah. Any reason not to? It seems generic and useful. [21:22:01] Just another thing to document I guess. [21:22:15] it's fine to do it in wmf-config, I think [21:22:18] was just a thought [21:22:25] *nod* [21:22:49] explicit is better than implicit ;) [21:23:45] i'm ok merging https://gerrit.wikimedia.org/r/#/c/191259 if you want [21:24:36] bd808 is all like "who are you and what did you do with ori" [21:25:55] Did we triple check that I only turned on group0 to logstash yet? [21:25:58] * bd808 triple checks [21:27:00] 'wmgLogstashServers' 'default' => false [21:27:04] LGTM [21:27:37] ori: if you wanna merge + sync I have the rest of the afternoon free to help watch it [21:28:10] It will change some file names on fluorine [21:28:13] sure. what about the fact that jgage is doing ES / LS stuff right now? [21:28:20] do you know if he's done? [21:28:54] I'm not sure. Maybe we should wait on the config change until the new nodes are in the cluster? [21:29:03] when is that expected to happen? [21:29:04] That should only take today and tomorrow I hope [21:29:29] robh/jgage are helping. [21:29:47] I say we do it now. That way, if there are problems that manifest under load, they will be manifest while ops are still on the task. [21:29:56] As opposed to having to pull them back in after they're done. [21:30:21] Sure. Plus this only send group0 to logstash [21:30:55] the bigger risk is things not getting to fluorine or a fuckup in the initialisesettings itself [21:31:28] let's merge it and then do a manual sync-common on mw1017 [21:31:33] and poke around to make sure it looks sane [21:31:52] can use the X-Wikimedia-Debug thing to make sure both group0 and group1 wikis load OK [21:31:58] and then sync it everywhere [21:33:08] maybe rebase https://gerrit.wikimedia.org/r/#/c/201987/ first? [21:33:20] nm, i just did; it rebased cleanly [21:34:25] bd808: can you remove your -1 from the follow-up commit? [21:35:17] {{done}} [21:35:47] ori: https://gerrit.wikimedia.org/r/#/c/187074/ (see that last comments) [21:37:15] bd808: ok, merged and synced to mw1017 (but no other app servers yet) [21:38:49] ow do I tell if x-wikimedia-debug worked? [21:38:50] nothing obviously broken on testwiki or enwiki-on-mw1017-via-debug-header [21:38:57] view source and scroll to the bottom [21:39:00] and see the backend host name [21:39:19] or just type mw.config.get('wgHostname') in your js console [21:39:28] "wgHostname":"mw1017" [21:39:36] yeah [21:39:40] I should make a js that shows that [21:39:59] yeah that'd be useful [21:40:17] so this LGTM [21:40:24] any reason not to sync everywhere? [21:40:33] nope [21:40:38] light the candle! [22:15:34] Krinkle: startup module never gets browser cache hits on my vagrant [22:15:46] * dancecat sighs...some other time [22:18:30] bd808: wait do I need all the extensions in /mediawiki *and* /mediawiki/extensions ? [22:18:42] huh? [22:18:57] there's no extensions in mediawiki core itself... [22:19:03] the former seemed to pop up running the vagrant git pull method [22:19:18] * dancecat will have to nuke all that and go back to his sh script [22:19:19] are you on a branch? [22:19:25] ^ [22:19:30] master [22:20:06] so you have /mediawiki/SomeExtension now? [22:20:21] lots of those [22:20:43] I... don't know what would have done that. [22:21:32] anomie: Starting to fill in api-feature-usage again! -- https://logstash.wikimedia.org/#/dashboard/elasticsearch/api-feature-usage [22:21:39] only group0 righ tow [22:22:16] \o/ [22:22:31] dancecat: I'm running `vagrant git-update` now to see if I can reproduce [22:24:58] gah just lost a commit I was about to make [22:25:00] * dancecat remakes [22:27:33] dancecat: I can't reproduce `vagrant git-update` pooping extensions all over $IP [22:28:11] bd808, did you try on Windows? :P [22:28:27] I did not, but the script runs in the VM [22:28:29] * dancecat nukes vagrant again [22:28:43] In theory the host OS shouldn't matter [22:28:50] in practice... no idea [22:36:30] what connects to redis on fluorine? [22:36:47] from MW [22:36:58] "Warning: Failed connecting to redis server at fluorine.eqiad.wmnet: Connection timed out" [22:37:33] xenon [22:39:43] bd808: what was that magic force cpu count command? [22:40:23] vagrant config vagrant_cores N [22:40:47] Does the default not work on windows? [22:41:04] never for me, no [22:41:07] You should file a bug for marxarelli [22:41:57] it's odd that it tries to set it to 0, maybe some code could sanity check it's computed value and use max( 1, count ) or something [22:42:51] I thought we had that. I blame ruby [22:43:54] This is the calculation -- https://github.com/wikimedia/mediawiki-vagrant/blob/master/lib/mediawiki-vagrant/environment.rb#L32-L47 [22:44:26] Which gets used here -- https://github.com/wikimedia/mediawiki-vagrant/blob/master/lib/mediawiki-vagrant/settings/definitions.rb#L19-L24 [22:44:31] dancecat: file a bug please. i tested it on windows 7 but that's all i had :/ [22:45:18] I'm not seeing where the sane default comes in though... [22:45:27] o/ marxarelli :0 [22:45:34] * Reedy tests on server 2003 [22:45:35] howdy [22:45:46] lol, it doesn't work there [22:46:19] http://p.defau.lt/?WEI24tn0yIpU4SlkwE_5zw [22:46:50] perhaps i put too much faith in my cursory search for "windows cpu count command line" [22:46:53] * anomie is coming very close to blocking [[User:Verdy p]] on mw.org, for personal attacks and general trolling related to Scribunto's reference manual [22:47:18] WFM on win 7 [22:47:55] * marxarelli goes to boot up his crappy little windows box [22:47:58] anomie: :( I hate haters [22:48:13] and server 2012 r2 [22:48:19] anomie: i'd be inclined to concur if they keep it up [22:50:18] His basic problem seems to be that I won't let him insert kilobytes of whining about how MediaWiki's language tags aren't BCP47 language tags and about how Scribunto's sandboxed Lua 5.1 isn't exactly the same as standard Lua 5.1 or standard Lua 5.3. [22:50:33] Reedy: weird. works for me on win 7 pro [22:50:47] does for me too [22:51:16] dancecat: what version of windows? [22:51:19] running the command that is [22:51:29] https://phabricator.wikimedia.org/T97087#1232761 [22:51:56] dancecat: could you try this snippet and see what value it returns: http://blog.robseaman.com/2009/7/11/detecting-the-number-of-processors-with-ruby ? [22:54:53] default: Error: Cannot create /vagrant/mediawiki/skins/Vector; parent directory /vagrant/mediawiki/skins does not exist [22:55:01] so many notices, I wonder if any of that matters [22:55:06] jackmcbarn: I may ping you when he does it again, to avoid further claims that I'm somehow "abusing my authority". [22:55:15] dancecat: all of it matters, file tasks please [22:55:39] heh I just ignored that the first times and didn't notice anything [22:56:17] * Reedy imagines anomie doing a Cartman impression [22:57:21] https://www.youtube.com/watch?v=gx4jn77VKlQ [22:57:28] https://youtu.be/gx4jn77VKlQ?t=14 [22:58:20] kk [22:58:22] * anomie hasn't watched that show for about 14 years. Ah memories of downloading RealPlayer bootlegs. [22:58:36] dancecat: wtf. $IP/skins is in mw/core.git. How could it start doing skins before that is cloned? [23:12:41] ori: PS F:\Dev> ruby.exe .\proc_count.rb [23:12:43] ./proc_count.rb:20:in `number_of_processors': can't determine 'number_of_processors' for 'i386-mingw32' (RuntimeError) [23:12:44] from ./proc_count.rb:23:in `
' [23:16:36] ori: I tweaked your showBackendPerfomance() function to put the MW host in my header -- https://meta.wikimedia.org/wiki/User:BDavis_%28WMF%29/global.js [23:21:35] ori: most of the Error: notices go away if the host directory is clean (I guess the shared files pollute the new vm state) [23:21:42] is that still worth filing then? [23:23:52] dancecat: try this: [23:23:54] require 'win32ole' [23:23:55] wmi = WIN32OLE.connect("winmgmts://") [23:23:55] q = wmi.ExecQuery('select NumberOfCores from Win32_Processor') [23:23:56] puts q.to_enum.reduce(0) { |cores, processor| cores + processor.NumberOfCores } [23:23:59] dancecat: you should use the ruby embedded with vagrant [23:24:18] c:\HashiCorp\Vagrant\embedded\bin\ruby [23:24:18] bd808: neat, nice [23:26:23] ori: thanks for finding that. way cleaner if it works [23:26:55] i adapted it from http://stackoverflow.com/a/6420817 , but that answer only counts the cores on the first physical processor [23:27:10] mine gets all of 'em, i think [23:27:42] win32ole is a mystery to me [23:27:51] never touched it before [23:28:25] ori: that gives 4 [23:28:25] less fun than win32olé [23:28:34] dancecat: THERE. ARE. FOUR. CORES. [23:28:44] ooh look i made a star trek joke [23:28:48] haha. less spicy [23:33:22] dancecat: one more... [23:33:45] what about wmic CPU get NumberOfLogicalProcessors | more +1 ? [23:33:58] maybe it's the more that is screwing it up [23:34:41] ori: that could be. the output he posted in https://phabricator.wikimedia.org/T97087 looks ok [23:35:04] works fine in cygwin [23:37:41] relying on one command-line tool fewer would be better, imo [23:37:42] could do: [23:38:05] `wmic CPU get NumberOfLogicalProcessors`[/\d+/].to_i [23:41:47] might as well use win32ole. i believe vagrant uses it for the vb provider already, so it would be zero bloat [23:42:43] marxarelli: https://gerrit.wikimedia.org/r/206332 . posted that before i saw yr last comment [23:42:51] if you prefer to just do the latter feel free to amend [23:43:37] ori: \o/ i'll check it out [23:53:48] * dancecat keeps seeing Liuxinyu970226 show up in phab [23:54:38] marxarelli: shouldn't the '[1, cores].max' logic apply on all operating systems? [23:54:56] ori: durr... [23:54:58] yes! :) [23:56:03] i checked out windows 8.1 in a vm for the first time while trying out this snippet [23:56:06] it's pretty interesting [23:56:31] if you do something that makes it clear you want the metro interface to go away it goes away, but it does beg you to give it a shot otherwise [23:57:16] http://i.imgur.com/VSFWWlK.png [23:57:20] "WAIT WAIT DON'T LEAVE YET" [23:57:38] ori: do we need to update mwerrors.py for the new log format? [23:57:39] "JUST GIVE IT A FUCKING *CHANCE*, OK? JUST A CHANCE!" [23:59:17] marxarelli: LGTM, can merge if you like [23:59:29] but i haven't tested it all together, don't have vagrant in the vm [23:59:42] bd808: yeah, probably