[00:18:20] because URL fragmentation hurts search-engine referrals, which hurts traffic [00:48:20] per : "Consolidating link signals for the duplicate or similar content. It helps search engines to be able to consolidate the information they have for the individual URLs (such as links to them) on a single, preferred URL." [05:49:44] TimStarling: proposed solution: https://gerrit.wikimedia.org/r/#/c/219782/ [16:19:19] anomie: could I ask you to review ? [16:19:55] ori: Brad is out on vacation this week [17:16:54] bd808: ah, thanks for letting me know [17:43:44] legoktm: csteipp: Do we no longer capture the "centralauth" logs? [17:43:54] hoo: it's now "CentralAuth.log" [17:44:02] Oh ok [17:44:09] Got a bit of a heart attack already [17:44:20] * bd808 made a lot of log names change a few months ago [17:44:35] the log files match the $channel now [17:45:22] Taht makes sense [17:45:24] :) [17:45:54] legoktm: Some logs in as foo, but is actually foo~barwiki ... but still we will automatically create the local account "foo" [17:45:59] That's a bug, right? [17:46:04] * Someone [17:46:27] that sounds like a bug yeah [17:46:28] yeah that seems not right but also somehow familiar [17:46:37] like we thought of it and couldn't fix it [17:47:29] Mh... we probably alter the user name in the log in process... but whatever creates the account, if it doesn't exist still has the original name [17:48:21] * bd808 looks at comments on https://gerrit.wikimedia.org/r/#/c/147020/ to see if there is documentation [17:50:24] hoo: I think Chris saw this flaw in his comments on PS9 there [17:50:44] "when it does exist, we're potentially doing an autocreate on a wiki with the pre-rename name, which seems evil." [17:51:02] "But a refactor of the entire Authentication process would be the right way to handle this-- i.e., probably implement the AuthStack RFC." [17:52:52] Ok, mh [17:55:26] The autocreate happens deep in the guts of SpecialUserlogin::authenticateUserData [17:57:12] We could maybe abuse the LoginUserMigrated hook... but it would be yucky I think [17:57:36] hmm nope [17:57:46] it doesn't pass $u by ref [17:58:41] so the only way to fix it is a new hook :/ [17:58:59] Probably not worth it [18:25:37] bd808: does anything use the exception-json log group? [18:27:16] legoktm: logstash does. I don't know that anything else reads the files [18:27:57] When we get https://gerrit.wikimedia.org/r/#/c/213348/ merged then I can switch logstash off of it [19:27:22] bd808: i threw in python-netifaces too for getting the interface ip [19:27:49] I saw that. I'm updating the patch now [19:28:40] are there any other libraries to add while we're at it? I think most of the controversy around scap died down, so it'd be OK now. fabric might still be off-limits, I don't know. [19:29:59] the only other one we inlined was https://github.com/dw/python-pure-cdb [19:31:00] not packaged, sadly [19:31:21] barely maintained I think. welcome to the world of things that use cdb [19:32:27] it was in squeeze [19:33:39] When Joe gets a pybal control lib for us we actually may not need either of these new libs [19:33:53] which he sounded hopeful for in a week [19:34:20] i think that's delusional [19:34:36] :) I have no reference point to judge [19:34:38] maybe SOMETHING in a week, but not something stable enough to run deployments [19:46:14] anomie, do you think we can merge https://gerrit.wikimedia.org/r/#/c/203369/ ? [19:53:25] MaxSem: he's on vacation this week [19:53:37] heh [19:53:59] anyone else wants to pull trigger on that? I [19:54:07] 'll rebase it then [19:58:25] bd808: what about the ability to target a particular dsh group? [19:58:37] it's in a follow up patch [19:58:45] cooool [19:58:53] https://gerrit.wikimedia.org/r/#/c/219752/ [19:59:26] with that you can say scap -Ddsh_targets=foo [20:00:19] MaxSem: I think we should wait until after the continuation change stuff [20:50:22] bd808: So extensions and composer. What's the latest recommendation? It seems rather impractical to have to create a composer.local.json file and maintain which to include. I tested locally by running 'composer update' in the extension dir and include extensions/X/vendor/autoload.php from X.php / (or XHOoks::register callback) [20:50:39] that seems to work fine. [20:52:19] it's hard to make a blanket recommendation. There are some extensions (eg SMW and wikidata) that are not quite like the others [20:54:13] It is possible to setup your composer.local.json to include extension/*/composer.json if you don't need to exclude any "special" extensions [20:54:48] or you can use mediawiki-vagrant which has a `vagrant git-update` command to help out [20:55:12] flow is a special one too as it has composer.json and composer.lock [20:55:51] there's actually an open patch set for handling that properly in mw-vagrant that I need to review [20:56:09] bd808: testing that now [20:56:16] marxarelli: awesome [20:56:46] * bd808 needs to remember to collaborate with marxarelli more [20:57:19] bd808: :) [20:59:38] bd808: This is for syntaxhighlight_geshi [20:59:47] which is about to land a patch that introduces composer.json [20:59:51] with a package it needs [21:00:01] *nod* [21:11:20] bd808: So.. does the merge plugin give any sort of error? [21:11:30] I pasted an inclusion for it in composer.local.json but it's not fetching it [21:12:24] you can try running composer with the --verbose flag to see more output from the plugin [21:15:38] Generating optimized autoload files [21:15:38] [merge] Loading composer.local.json... [21:15:43] those are the last two lines of the output [21:15:44] nothing after [21:15:58] maybe it doesn't support ../extensions ? [21:17:46] legoktm, I'm lost - what lib do we currently use for emailing? [21:17:52] we == prod [21:18:02] MaxSem: PHP's mail() [21:18:12] MaxSem: but iirc something uses the pear mail thing [21:18:42] couldn't find anything like that in mw-config [21:19:21] It's installed as a debian php package isn't it? [21:19:44] php-net-smtp [21:19:55] and php-mail [21:20:14] MaxSem: includes/mail/UserMailer.php#264: is_array( $wgSMTP ) ? PEAR : mail() [21:21:01] but mw-config doesn't touch $wgSMTP [21:21:10] so it's mail()? [21:21:12] yeah [21:21:29] unless we're sending HTML mail [21:21:32] which Echo does [21:21:45] line 228-ish [21:21:58] the mail stuff is a mess [21:22:09] ostriches has a patch somewhere to use composer to get the pear library [21:22:18] yup [21:22:24] would be grat [21:22:38] https://gerrit.wikimedia.org/r/#/c/207455/ [21:23:26] bd808: ok, so there's a new dsh group called 'scap-test', which contains mw1170-mw1175 (eqiad) and mw1270-mw1275 (codfw), all of which have been depooled from pybal. [21:23:33] then ditch all pear-related packages mercilessly [21:25:31] ori: k. So in theory we can update scap on the cluster and try a targeted scap to see what happens. Maybe that should wait until after swat today? [21:25:46] I have a call in an hour [21:26:03] and then swat is at 23:00Z [21:26:08] sure [21:26:17] now works for me too, if you want [21:26:27] if there is breakage, we can revert before swat [21:27:05] I guess it should only take ~30 minutes [21:27:26] less, since there are fewer app servers to deploy to [21:27:46] true. l10nupdate will be the bounding time [21:28:42] ugh [21:29:05] the pybal interface isn't on mw1170 [21:29:13] is that because you de-pooled it? [21:30:00] are you sure? [21:30:03] it's showing up in ifconfig [21:31:42] ah interesting. It doesn't show in `ip addr` [21:32:05] but yeah it does show in ifconfig -a [21:32:05] does the python code pick it up? [21:32:43] I'll check [21:33:42] yeah netifaces sees it fine [21:34:27] ok, lets update scap in beta first and make sure it doesn't melt there [21:42:10] ori: found a problem in beta. the netifaces lib isn't installed on deployment-videoscaler01 [21:42:22] going to check and see if I can figure out why [21:42:44] bd808: puppet hasn't run? [21:43:05] looks like that host is broken with puppet yeah [21:43:10] I can't even log in [21:43:30] which would suggest that puppet hasn't run there since the nfs outage [21:45:33] ori: that was the only broken host so I think we are safe to move on to updating prod [21:59:54] bd808: +1 [22:01:08] bd808: looks like Roan is deploying a security patch, so let's wait for him to finish that [22:01:17] prod updated. I'm double checking that I know the right cli magic to change the target hosts before the runtime test [22:27:06] ori: composer-update only in the mediawiki directory or in the extension directory as well? [22:27:08] I'm not able to get it to work [22:27:18] using the documented approach [22:28:40] Krinkle: in the mw directory [22:29:51] I've created the composer.local.json with the contents but composer is not fetching the kzykhys/pygments [22:29:57] ran several times [22:30:52] legoktm: ^ [22:32:20] Krinkle: it worked for me...let me re-try [22:35:06] Just in case ../ doesn't work I created an empty SytaxHighlight_GeSHi directory in mediawiki-core/extensions with just the composer.json in it from that patch [22:35:36] Krinkle: http://fpaste.org/235570/50125291/raw/ [22:36:16] you have to run "composer update" twice because the first one installs the plugin, and the second actually runs it, but there's a patch to fix that [22:36:16] Ah! [22:36:27] ori's patch has a typo. "SytaxHighlight_GeSHi/comper.json" [22:36:50] There's no error reporting from the merge plugin [22:36:55] Got it now [22:37:08] Ran a diff against's yours. Obvious now. [22:37:35] Thanks legoktm [22:37:51] :) [22:38:14] I think that is intentional so people can optionally use composer.local.json, but if it doesn't exist it won't complain [22:40:17] https://github.com/wikimedia/composer-merge-plugin/issues/37 [22:40:23] Yeah, for composer.local.json itself [22:41:15] Perhaps a non-fatal warning when a pattern matches nothing [22:41:24] with exception for local itself [22:41:28] e.g. the top level [23:59:26] TimStarling: you have a few minutes? [23:59:39] yes