[00:00:53] * AaronS was looking at http://unix.stackexchange.com/questions/108635/why-i-cant-escape-spaces-on-a-bash-script [00:03:01] AaronS: Ah, yeah, for MW_DB_PATH [00:03:04] but not * [00:03:06] (I think( [00:03:18] * can match spaces fine I think [00:03:24] but yeah, if MW_DB_PATH has a space that'd be a problem [00:03:32] though can be worked around by doing "$MW_DB_PATH"/* [00:03:39] I guess if bash is doing it then it's fine [00:03:42] because bash allows string concatenation of quoted and unquoted [00:03:50] which it has to be [00:05:05] AaronS: So it's working now [00:05:07] expanded to: [00:05:08] 23:55:59 + chmod 777 /mnt/home/jenkins-deploy/tmpfs/mediawiki-core-qunit-karma/build833.sqlite /mnt/home/jenkins-deploy/tmpfs/mediawiki-core-qunit-karma/wikicache.sqlite [00:05:24] no build failure [00:05:31] and no dberror, exception or error.log [00:05:46] no locks in debug-www.log [00:05:54] and no other db errors that I can see in https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-karma/833/artifact/log/mw-debug-www.log/*view*/ [00:05:56] Yay! [00:07:31] AaronS: I +1'ed but don't feel comfortable merging as I don't know much about the db and cache classes. [00:07:40] Got an extra reviewing in mind? Perhaps TimStarling [00:07:50] https://gerrit.wikimedia.org/r/#/c/192476/ [00:09:33] 6MediaWiki-Core-Team, 10CirrusSearch: inefficient work of CirrusSearch in Russian Wikipedia - https://phabricator.wikimedia.org/T88724#1068305 (10Manybubbles) So what ruwiki needs is for the phrase rescore to still work for searches like ```foo* bar*```. That _currently_ relies on Elasticsearch's behavior for... [00:16:24] csteipp: does https://phabricator.wikimedia.org/T74193#1067388 look good? [00:17:29] looking... [00:19:25] legoktm: lgtm [00:19:52] ok, I'll make it non-experimental then :) [00:28:00] greg-g: ping... I need a +1 from you [00:28:06] Not on a real change :P [00:33:34] 6MediaWiki-Core-Team, 10Librarization, 10utfnormal: Bring in utfnormal library with composer and create backwards-compat layer - https://phabricator.wikimedia.org/T90825#1068394 (10Legoktm) 3NEW [00:33:45] 6MediaWiki-Core-Team, 10Librarization, 10utfnormal: Bring in utfnormal library with composer and create backwards-compat layer - https://phabricator.wikimedia.org/T90825#1068401 (10Legoktm) [01:13:42] Krinkle / AaronSchulz : I did have a look at that patch already, the @todo FIXME is a bit depressing [01:25:43] TimStarling: I'll probably add a 'directory' param to the array [01:26:04] * AaronSchulz was trying not to do any lightbulb changing [01:26:24] well, I merged it [03:56:00] 7Blocked-on-MediaWiki-Core, 10Continuous-Integration, 6Scrum-of-Scrums, 5Patch-For-Review: MediaWiki installs in Jenkins frequently fail to access their sqlite database due to locks - https://phabricator.wikimedia.org/T89180#1068951 (10Krinkle) >>! In T89180#1068797, @gerritbot wrote: > Change 192480 merge... [03:56:22] 7Blocked-on-MediaWiki-Core, 10Continuous-Integration, 6Scrum-of-Scrums: MediaWiki installs in Jenkins frequently fail to access their sqlite database due to locks - https://phabricator.wikimedia.org/T89180#1068953 (10Krinkle) 5Open>3Resolved a:3Krinkle [06:20:50] AaronS: I do not know, I need to find out. [06:43:35] Nikerabbit: did you see https://gerrit.wikimedia.org/r/#/c/192963/ ? [06:45:13] AaronS: heh I was just replying to it when you pinged [06:45:24] AaronS: can you comment on my proposal? [09:56:52] 6MediaWiki-Core-Team, 10Possible-Tech-Projects, 10Wikimedia-Hackathon-2015: Create a system to store and query links to ISBN - https://phabricator.wikimedia.org/T90852#1069360 (10Ladsgroup) 3NEW a:3Ladsgroup [10:00:52] 6MediaWiki-Core-Team, 10Possible-Tech-Projects, 10Wikimedia-Hackathon-2015: Create a system to store and query links to books - https://phabricator.wikimedia.org/T90852#1069374 (10Ladsgroup) [10:05:12] 6MediaWiki-Core-Team, 10Citoid, 10Possible-Tech-Projects, 10Wikidata, 10Wikimedia-Hackathon-2015: Create a system to store and query links to books - https://phabricator.wikimedia.org/T90852#1069383 (10Qgil) #MediaWiki-Core-Team ? I would start with #Wikidata and perhaps #Citoid [10:13:40] I would use the help of someone more knowledge able than me on solving this urgent issue https://phabricator.wikimedia.org/T90704#1069403 [12:11:21] Nikerabbit: are you looking for a stack trace, too? [13:40:45] 6MediaWiki-Core-Team, 10GlobalUserPage: Wikilinks from global user pages should point to the central wiki - https://phabricator.wikimedia.org/T89916#1069664 (10He7d3r) Because when I want a link to a specific project I use @Pathoschild's [[https://meta.wikimedia.org/w/index.php?title=Synchbot&diff=prev&oldid=2... [15:48:21] 6MediaWiki-Core-Team, 10GlobalUserPage: Wikilinks from global user pages should point to the central wiki - https://phabricator.wikimedia.org/T89916#1069990 (10KTC) That's all fine and dandy if it weren't for the fact that "m:" was broken on global user page until change 191812 (see above) provided a temporary... [16:48:09] anomie: snowpocolypse NC hasn't taken out your power? [16:48:18] robla: Not yet anyway [16:48:43] I fear manybubbles wasn't so lucky [16:58:11] anomie, csteipp, legoktm: I smushed Brad's new words around a bit on https://www.mediawiki.org/wiki/Requests_for_comment/AuthManager ; generally I think I'm pretty happy with what we have there [16:59:14] the only part that is still a bit hand wavy is getting the session data [17:00:27] I think what's there makes sense but it is spec'd at a higher level than what we've ended up with for the core flow [17:00:36] which should be fine really [17:02:10] legoktm: on a completely unrelated note, I found out last night that the way I've been bundling the vendor directory for the scholarships and grants projects kind of sucks when you want to add a `composer test` entry point. :/ [17:03:18] I may have to figure out how to deal with the complexity of having a separate vendor repo for each of them and use submodules on release branches [17:03:48] or maybe just release branches and prepare the vendor repo there each time. [17:05:57] we could consider running composer in production (on deployment hosts, not every server) [17:06:08] and using satis to pull from wmf-hosted mirrors [17:06:41] (https://github.com/composer/satis) [17:08:15] bd808: yeah, you have to keep switching between --no-dev and --dev :/ [17:10:35] ori: That would be interesting. For something like scholarships that is deployed with trebuchet I could see that working. Fetch to tin; run composer; sync [17:11:30] why not mediawiki, too? [17:12:27] it's not a big deal either way -- the vendor repo works well enough, too [17:12:36] but composer is a nice tool, and we shouldn't be afraid to use it [17:13:50] The lack of signed packages is the current security concern I think [17:14:18] the pointers on packagist.org could change at any time to any other code [17:14:36] * bd808 will read up on what satis does [17:17:09] bd808: I like your smushed words [17:21:29] <_joe_> mmmh manybubbles is not here? [17:22:29] <^d> Power possibly out. Heavy snow [17:22:29] He might have lost power due to the snow here. [17:23:20] <_joe_> gee [17:57:41] <_joe_> SMalyshev: I won't make it to the meeting today, I'm feeling not-so-well and I have to lie down for a bit [17:57:53] _joe_: eeeeep, hope you feel better [17:58:03] _joe_: ok, feel better [17:58:19] <_joe_> ori: I suspect sleeping 3 hours tonight has something to do with it [17:58:33] go to sleep then! :) [18:29:56] csteipp: I already deployed the job, so https://integration.wikimedia.org/ci/view/Default/job/mediawiki-vendor-composer-security/configure has the configuration [18:45:00] anomie: for the SecondaryAuthenticationProviders, we specify that they are in an ordered list. Should PreAuthenticationFilters also be an ordered list? [18:46:12] ugh: Maybe, although it seems to me that it matters less since we're not allowing them to respond with "REDIRECT" or "UI". [19:26:03] anomie: hmm, if captcha is a preauthenticationfilter, doesn't it need to be able to request UI stuff? [19:27:00] ugh: That's in the latest version of the RFC: they can provide an AuthenticationRequest type, which is how we're passing around "I need this UI" [19:31:59] <^d> AaronS: Any thoughts on https://phabricator.wikimedia.org/T90796? [19:32:29] anomie: so we'd have a CaptchaAuthenticationRequest or something? [19:32:38] ugh: Yeah [19:32:44] hmm, alright [19:41:14] ^d: log the full URL? [19:41:22] <^d> will do [19:56:50] AaronS: Thanks for all the sqlite work. Looks like the mediawiki core qunit builds are now passing. [19:56:57] AaronS: Though the mediawiki-extension builds are still failing. [19:57:23] Thu Feb 26 18:22:09 UTC 2015 integration-slave1005 build480 MessageBlobStore::insertMessageBlob/single-row 5 database is locked INSERT OR IGNORE INTO msg_resource (mr_lang,mr_resource,mr_blob,mr_timestamp) VALUES ('en','Base64.js','{}','20150226182209') [19:57:52] Is the new transaction mode intended to be used for message cache? Or a different solution for this one like for objectcache? [19:58:31] Ah, right : https://phabricator.wikimedia.org/T90001 [20:03:10] aren't those errors caught? [20:03:27] ^d, AaronS: evan merged a couple of my trivial xhprof fixes today. Maybe he will look at the bigger one that adds xhprof_frame_begin [20:03:35] <^d> yay [20:03:37] <^d> ! [20:04:06] he merged by rewriting and taking my name off of the commits, but ... whatever [20:08:30] Krinkle: is there a link to failing test output? [20:08:41] AaronS: They are caught. [20:09:21] AaronS: But I'm working towards making builds fail if error logs are non-empty [20:09:49] so that e.g. php notices/warnings and other such recoverable warnings are not tolerated unless intentionally surpressed. [20:10:05] This shouldn't happen during normal operation. [20:10:21] But we can defer that to after we switch to MySQL if it's not worth supporting sqlite. [20:10:56] I do worry about how this would work in reality for a third party. Would they never have message cache? Or does it try again and then succeed? [20:15:22] I don't know if this is the proper place to ask so naturally, I'll ask here: is there not a magic word for the database name? I'm not seeing anything appropriate https://www.mediawiki.org/wiki/Help:Magic_words [20:15:44] {{FOO}} makes afwikitionary [20:25:43] Krinkle: it think it will just keep retrying [20:26:04] it still generates the expected blob, it will just be slower until cached [20:26:40] AaronS: Right [20:26:59] AaronS: It's no longer causing test failures (such as was for objectcache). [20:27:39] medaiwiki-extensions is failing, but for code reasons with the actual test. So can't tell at the moment whether it woudl pass or not. But yeah, the other two (l10n cache and message cache) are low priority for now. [20:27:55] It'd be nice to fold it back in, but no time/priority for it I guess. [20:29:52] this all happens in load.php right? [20:30:01] * AaronS would be curious to see the exact logs [20:30:32] Yeah msg_resource is triggered by the load.php path [20:31:33] AaronS: What kind of logs? I can enable more log groups etc if you like [20:32:46] dberror, dbperformance, the sames ones in the output that had bagostuff errors [20:33:14] AaronS: Are those in debug by default? [20:33:16] or opt-in? [20:33:23] I wonder if load.php disabled DBO_TRX, insertMessageBlob used start/endAtomic, and explicit trxs uses IMMEDIATE, how much that would help [20:33:31] I see [DBPerformance] in https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-karma/858/artifact/log/mw-debug-www.log/*view*/ [20:34:24] On Special:BlankPage [20:34:30] apparrantly [20:41:30] anyway, it's probably fine to ignore those errors...maybe the logging could be split up a bit [20:44:09] <^d> AaronS: Invalid host name (server: wikipedia, request: /). [20:46:58] <^d> REQUEST_URI wasn't all that interesting [20:47:56] 6MediaWiki-Core-Team, 7Epic: General authentication improvements for MediaWiki - https://phabricator.wikimedia.org/T90925#1071237 (10bd808) 3NEW [20:48:32] 6MediaWiki-Core-Team, 7Epic: General authentication improvements for MediaWiki - https://phabricator.wikimedia.org/T90925#1071252 (10bd808) The work in progress for {T89459} should make all of these tasks easier to achieve. [20:49:54] <^d> AaronS: Shouldn't the server at least be 'wikipedia.org'? [20:50:00] <^d> bare 'wikipedia' seems odd [20:50:33] maybe HOST is used oddly [20:51:14] <^d> I'm going to find out where that request is coming from [20:53:03] 6MediaWiki-Core-Team, 10MediaWiki-extensions-OAuth, 7Epic: Support a nice sso experience with MediaWiki's OAuth - https://phabricator.wikimedia.org/T86869#1071297 (10bd808) [20:53:07] 6MediaWiki-Core-Team, 7Epic: General authentication improvements for MediaWiki - https://phabricator.wikimedia.org/T90925#1071295 (10bd808) [20:54:59] <^d> AaronS: Happens like every 10 minutes. I wonder if this is something internal that's malformed [20:55:05] <^d> Let's find out where this is coming from [20:55:14] like a health check? [20:58:29] <^d> That's my thought [20:58:52] <^d> I added the remote_addr and xff to the debug statement [21:10:57] Krinkle: https://gerrit.wikimedia.org/r/#/c/193248/ not related [21:13:34] eww composer :/ [21:13:53] 6MediaWiki-Core-Team, 7Epic: General authentication improvements for MediaWiki - https://phabricator.wikimedia.org/T90925#1071399 (10bd808) Another possible sub-project would be {T76103} which requires some work on the account creation flow. This isn't strictly authentication related but it does fall under a s... [21:16:59] <^d> AaronS: ignore the 10mins. No real pattern [21:17:27] Anyone have experience with SMW's sparql parsing? [21:58:08] 6MediaWiki-Core-Team, 10Continuous-Integration, 10Librarization, 10MediaWiki-Codesniffer, 5Patch-For-Review: Publish MediaWiki codesniffer config on Packagist - https://phabricator.wikimedia.org/T85631#1071650 (10Legoktm) [22:31:46] 6MediaWiki-Core-Team: SpoofUser deadlocks from rename jobs - https://phabricator.wikimedia.org/T90967#1071844 (10aaron) 3NEW a:3aaron [22:46:47] Krinkle: have you tried setting https://www.mediawiki.org/wiki/Manual:$wgDBservers and having flags => 0, does that remove the rest of the log warnings? [22:47:53] AaronS: What would flags=0 do? [22:48:05] turns off DBO_TRX [22:49:42] AaronS: I see [22:50:20] AaronS: Hm.. LocalSettings.php is generated and already specifies the db config [22:50:27] Would this make sense as a new default? [22:50:41] I guess that's what Daniel was pushing for in 2012 [22:50:53] or perhaps for sqlite only [22:50:58] it makes sense for load.php request for sqlite [22:51:10] which should only write to cache tables [22:51:27] AaronS: Is it something we can set per query? [22:51:36] E.g. mark the messagecache queries with that [22:51:42] I'm not sure how much they benefit from REPEATABLE-READ anyway [22:51:58] and other cache related things from load.php [22:52:09] Krinkle: the problem is that once any caller started an implicit trx you can just kill the trx [22:52:22] and you can't make a second connection in sqlite as that would deadlock [22:52:45] AaronS: Right [22:52:58] AaronS: Do we also implicit transact start for select queries? [22:53:23] If not, the only write queries from load.php should be cache, so if we mark those as non-transact, it would work? [22:53:39] DBO_TRX applies to all queries [22:59:14] anyway if the flag was unset early in load.php, it could work [23:06:03] legoktm: are we going to have an email to announce the RFC re-write today? Or will it bleed into tomorrow? [23:12:05] bd808: I could send it today if anomie and csteipp are good with the RfC [23:12:25] I'm good with it [23:17:50] "[10:17] < anomie> bd808: I like your smushed words" [23:17:56] so say we all [23:46:35] legoktm: https://gerrit.wikimedia.org/r/#/c/193287/1 [23:54:18] /tmp/hudson8349947187478010154.sh: line 2: /srv/deployment/integration/slave-scripts/bin/mw-teardown.sh: Permission denied [23:54:20] hrm