[00:03:37] rpcbind.sock is root:root 666 which is an interesting scheme [00:04:05] basically equivalent to a TCP localhost socket [00:05:08] on my desktop I have another couple of sockets in /var/run that use the same permissions [00:05:11] 666 root:root [00:06:38] btw I just noticed that /var/run/hhvm is owned by apache, which I'm pretty sure is wrong and insecure [00:06:57] since it means apache could delete and recreate the PID file [00:08:23] re: sockets, connections could still be restricted to particular users/groups via getsockopt SO_PEERCRED, which i think is what rpcbind does [00:10:39] sorry to be dumb, but why is that insecure? because it's a way of potentially tricking root into manipulating a different process with start/stop/restart hhvm calls? [00:13:45] yes [00:14:21] to the second point [00:14:50] on the first: yes, connections could be restricted, but we are just talking about a proxy for a remote network service that anyone can directly access anyway [00:15:12] right [00:15:13] and that anyone can access via localhost:11212 [00:17:25] unfortunately we no longer provision the init script for nutcracker, it's part of the package, so it'd be annoying to change. but the script does source /etc/default/nutcracker, which is puppet-managed, so we could set a different umask there [00:17:38] however, any umask we set there will also apply to nutcracker's pidfile [00:18:14] umask? [00:18:26] 3MediaWiki-Core-Team, ContentTranslation-Deployments, MediaWiki-extensions-ContentTranslation: Content Translation Beta Feature security review - https://phabricator.wikimedia.org/T85686#964846 (10Jsahleen) @csteipp @Nikerabbit I have modified the communication between cxserver and the front end for MT. Respons... [00:20:28] setting nutcracker's umask is not the right way to do it [00:21:12] no, ideally the init script or nutcracker itself would set the permissions to a configured value [00:21:26] interesting, "Connecting to the socket object requires read/write permission. This behavior differs from many BSD-derived systems which ignore permissions for UNIX domain sockets. Portable programs should not rely on this feature for security. " [00:21:46] yeah [00:21:50] it was a pain to work out with keyholder [00:22:00] that's the ssh agent proxy socket thing i wrote [00:23:54] so I guess the normal thing to do is to create the socket then chmod it? [00:24:27] yeah [00:28:45] well, a hack would be to have the init script wait until it starts up, then chmod the socket [00:30:09] yes, that's not even a hack, but it does require repackaging nutcracker or clobbering the packaged file [00:34:38] well, for the time being, on mw1231, i'm going the chmod the file permissions on the socket manually [00:34:55] I think doing it properly would be something like [00:34:58] http://paste.tstarling.com/p/QWYxUV.html [00:36:06] yeah. submit a PR? [00:36:17] I would have to test it [00:36:32] but yes, upstream may possibly accept that [00:37:09] my new laptop just arrived [00:37:59] fun, what kind did you get? [00:38:30] thinkpad x240 [00:39:28] nice. i assume you're taking off to play with it. (if not, you should :P) ttyl [02:31:55] ori: in the future you can link to a page like meta.wikimedia.org/wiki/Special:MyPage/global.js and it'll redirect people to the right place [02:32:11] oooh, did not know that. thanks! [03:14:47] 3MediaWiki-Core-Team, ContentTranslation-Deployments, MediaWiki-extensions-ContentTranslation: Content Translation Beta Feature security review - https://phabricator.wikimedia.org/T85686#965149 (10Jsahleen) Patches updated following comments from Krinkle. Code to encode as html entities and decode on the client... [04:22:46] 3MediaWiki-Installer, MediaWiki-Core-Team, Librarization: composer.json should be useable by WMF, core and extensions - https://phabricator.wikimedia.org/T67188#965204 (10Parent5446) As a matter of tracking, here is the pull request to allow plugins to add custom commands to composer: https://github.com/compose... [04:37:25] huh [04:37:41] TimStarling: either I messed something up or scalable and lru are not measureably different on our workload [04:37:56] I don't know what percentage of time we're even spending in the pcre cache anyway [04:38:39] it's possible [04:39:01] I think the tight loop tests were only separated by 10% or so [04:50:04] 3MediaWiki-Core-Team, MediaWiki-extensions-Echo: Allow "article-linked" notifications for pages in a user defined list - https://phabricator.wikimedia.org/T66090#965227 (10Mattflaschen) [04:50:38] I mean, they were separated by 10% in the single-threaded case [04:51:22] and if your benchmark is not hitting the PCRE cache very hard, the results will be approximately the same as single-threaded since lru won't give a significant amount of lock contention [05:19:44] TimStarling: why don't we just set the server address in mc.php to localhost and leave it at that? IMO, a proper patch to nutcracker ought to make the file permission configurable, and not call chmod at all unless permissions were explicitly specified. right now we're looking at testing the patch, landing a PR upstream, rebuilding the package, deploying the package, testing the change, and then finally doing a tricky migration, vs. ju [05:19:44] st s/127.0.0.1/localhost/ in mc.php that can be deployed right away and will effectively make this problem disappear. [05:20:48] ok, give it a go [05:33:42] ok, i set it to localhost for mw123*, we can do the rest later if those ones look okay [05:34:17] 3MediaWiki-Core-Team, ContentTranslation-Deployments, MediaWiki-extensions-ContentTranslation: Content Translation Beta Feature security review - https://phabricator.wikimedia.org/T85686#965268 (10csteipp) >>! In T85686#965149, @Jsahleen wrote: > Patches updated following comments from Krinkle. Code to encode as... [06:00:00] 3MediaWiki-Core-Team, ContentTranslation-Deployments, MediaWiki-extensions-ContentTranslation: Content Translation Beta Feature security review - https://phabricator.wikimedia.org/T85686#965277 (10KartikMistry) @csteipp You can test using our instance on Beta Cluster at: http://en.wikipedia.beta.wmflabs.org/wiki... [07:25:06] hoo: https://en.wikipedia.org/w/index.php?title=User_talk:Hoary&diff=prev&oldid=641694003 :| [07:25:49] legoktm: Yikes :( [07:27:32] * legoktm has no idea and is going to sleep [07:27:37] Absolutely nothing useful in the logs [07:27:58] I'll poke _joe_ when he's up :( [07:28:02] good night [07:42:01] <_joe_> hey hoo [07:42:18] hi _joe_ [07:42:39] We're still seeing these issue with Special:MergeAccount [07:42:48] <_joe_> I imagined that [07:42:56] but not there's nothing in the logs despite the spurious apache log entries [07:43:26] <_joe_> the spurious apache log entries just tell you that the request was responded with a non-200 answer [07:43:36] <_joe_> it's a bug in apache mod_proxy_fcgi [07:43:53] Ok [07:44:00] <_joe_> one thing we could do is try to replay one mergeaccount action directly on a backend [07:44:04] <_joe_> and see the response [07:44:06] <_joe_> there [07:44:44] Ok... any idea why we don't have any log entry from HHVM running out of memory or whatever? [07:44:57] <_joe_> because it does not, apparently [07:45:17] <_joe_> also, it would be the script exceeeding its memory limit, it's not hhvm crashing, right? [07:45:28] yeah, sure [07:46:11] _joe_: If you catch requests to replay them, how do you do that, btw? [07:47:41] <_joe_> hoo: I'd need to do that with a specially-set up account, and we can do it with curl directly from tin, for instance [07:48:45] So there's probably no way around creating a dummy account whith which I can reproduce that? Sounds super fun [08:01:21] <_joe_> hoo: not that /I/ see [08:01:50] <_joe_> someone who knows something more about Special:Mergeaccount inner workings could have better ideas [09:12:53] argh not again [09:16:29] requesting review: https://gerrit.wikimedia.org/r/183798 [09:18:11] done [09:18:46] AaronS: thanks! I thought you were sleeping [09:19:02] I *should* be, I am not [09:24:51] Nikerabbit: I just override jenkins [09:24:56] you are having bad luck today ;) [09:27:13] AaronS: yeah it seems [14:08:13] 3MediaWiki-Internationalization, MediaWiki-Core-Team, MediaWiki-extensions-Scribunto: Scribunto unit tests failing due to MediaWiki core changes - https://phabricator.wikimedia.org/T85854#965654 (10Anomie) 5stalled>3Resolved [15:10:34] legoktm: I debugged the MergeAccount stuff more and was able to come up with a real improvement now [15:10:58] the other patch had no effect on how far the process went until it ran OOM :P [15:14:04] hoo: Who'd be a good person to talk to about Wikibase and API stuff like https://gerrit.wikimedia.org/r/#/c/182858/? [15:14:54] anomie: Daniel or aude, probably... [15:15:17] Maybe I as well, but I'm kind of low on time right now [15:18:23] that's a big patch [15:20:11] probably daniel is best to talk to but doubt he has much time either [17:47:13] bd808: ori at the developers summit, i think it would be a good idea to have a session / discussion about performance and profiling (xhprof etc) [17:47:45] would you be interested in organizing one? [17:47:53] if we could learn more about the profiling tools etc., would be helpful [17:48:07] yeah that could be useful. AaronS knows a ton about using the profiling tools [17:48:13] i'm probably not best to lead, but could organize / schedule [17:48:38] i can show what i've done with xdebug but it doesn't always match production by any means [17:49:45] monday afternoon looks open (4:15pm) [17:50:51] my sense is that most backend development falls into two categories: [17:51:19] (a) generic extensions, which for the most part call APIs in core that have already been tuned extensively over the years [17:51:25] (b) wikidata [17:51:47] * aude nods [17:51:57] wikidata is doing interesting new things, so profiling data is useful [17:52:28] but I can't recall too many instances of PHP profiling data being terribly useful or interesting to anyone but a handful of people in core in the past year or two [17:52:36] (apart from wikidata, I mean) [17:52:36] even in core, though, things like figuring out bcrypt passwords were taking so much time in phpunit [17:52:51] bcrypt is slow on purpose :) [17:53:00] not as much attention is given there to tests [17:53:06] I mean, don't get me wrong -- it certainly doesn't hurt [17:53:08] bd808: yes [17:53:18] there's nothing bad about disseminating PHP profiling tips [17:53:29] profiling and cleaning up tests would be so awesome [17:53:29] but where we fall desperately short is understanding the performance of frontend code [17:53:33] * aude also interested in js [17:53:43] knows even less how to profile [17:53:48] even though i'm sure it's easy [17:53:54] not really, sadly [17:54:02] :/ [17:54:09] "don't touch the DOM" [17:54:10] Yeah, front-end development is not easier than backend development. [17:54:32] way more variables [17:54:33] it's way harder [17:54:53] and the landscape is constantly shifting [17:55:15] and also Internet Explorer. [17:55:30] Don't touch the DOM is one that is both true and very common. It's interesting how often still touch the DOM when they don't have to. [17:56:06] speaking of shifting landscapes... should we start worrying about a jessie version of MW-Vagrant? [17:56:08] Also don't use the DOM to store data or reflect application state :) E.g. isUserLoggedIn = $("..").hasClass('logged-in'); Put htis way it's obuviously wrong, but there's plenty of cases that esssentially do this. [17:56:20] Jessie? [17:56:23] * aude nods [17:56:26] <^d> Krinkle: Debian [17:56:31] Right [17:56:52] bd808: what's the '#12' thing for fatalmonitor? [17:57:33] the hhvm output has embedded newlines and they get translated to #012 by rsyslog [17:57:40] ohhh [17:57:47] could you add that as a comment above the sed line? [17:57:50] every line from hhvm starts with one for some random reason [17:58:07] that == "the hhvm output has embedded newlines and they get translated to #012 by rsyslog" [17:58:15] *nod* [18:04:11] ori: When wanting to gather some eventlogging data points (more than just counts, e.g. including page/wiki/moduels), I'm tapping in the feed from tin. Some feedback on the script? https://gist.github.com/Krinkle/aeef8f7fb78a4105902b [18:04:22] I've mostly glued it together from other scripts, not really sure waht I'm doing. [18:08:00] Krinkle: there's a much nicer API now: https://gist.github.com/Krinkle/aeef8f7fb78a4105902b#comment-1369390 [18:08:37] ori: that module is available systemwide on tin? [18:09:25] > ImportError: No module named jsonschema [18:09:26] 3MediaWiki-General-or-Unknown, MediaWiki-Core-Team: Drop PHP 5.3 support - https://phabricator.wikimedia.org/T75901#966225 (10chasemp) [18:10:02] Seems the module is available but not all its dependencies. [18:10:14] Krinkle: try again? [18:12:11] Krinkle: did it work? [18:12:20] standup, biab [18:13:32] ori: the parserout/secondary updates code is pretty awful [18:13:37] ori: import working, now failing at AttributeError: 'module' object has no attribute 'connect' [18:13:42] * AaronSchulz tries to figure out how to detangle that mess [18:14:08] Krinkle: oh, heh. EL needs to be updated on tin. i'll update it [18:14:24] ori: cool, no worries [18:14:37] it's gonna be a source of random things to investigate when I idle [18:14:55] Krinkle: try now [18:15:13] ori: perfect [18:15:46] ori: btw, noticed these lines in our deamon: https://github.com/wikimedia/operations-puppet/blob/production/modules/webperf/files/deprecate.py#L5-L6 – is that needed? [18:16:05] or is prod configured centrally to use utf-8 (I'd hope) [18:18:41] [tin:~] $ python -c 'import sys; print sys.getdefaultencoding()' [18:18:41] ascii [18:18:55] right [18:19:28] ascii is the default for python 2.x. at some point everyone recognized that utf-8 would have been the better choice by a wide margin, but then there was uncertainty in the community about whether that would break things that expect the default encoding to be ascii. [18:19:32] and from __future__ import unicode_literals [18:19:43] a fairamount of bootstrapping alltogether [18:20:26] it is a little bit, but with those two things you can more or less ignore encoding issues everywhere else [18:20:34] yeah [18:32:04] bd808: (shamelessly exploitative ask) if you feel like it, composerifing would be cool [18:32:16] i threw it up on github in a slapdash way because people were asking about it after the hhvm blog post [18:33:47] tsk tsk Apache 2.0 without a NOTICE file [18:34:02] :) [18:34:27] is that a thing? I had no idea. [18:34:50] I used GitHub's 'auto-initialize this repo with a license' feature, so if it's wrong, so is GitHub, probably. [18:35:31] yeah apache 2 is weird. The copyright holder line goes in a notices files so that the license file is identical for all projects [18:36:09] it's in github only so if you do end up doing anything with it feel free to push [18:36:19] as opposed to GPL/MIT/everthing else? where you add the copyright holder at the top of the license file [18:36:36] but yeah I can take a look [18:36:41] \o/ [18:38:42] I'm not sure its really Composer package sort of stuff but I'll think about how it might be easier to install [18:39:05] your only php bit is really an auto-prepend file [18:40:08] php + python + bash :) Truly a polyglot project [18:40:58] don't forget perl [18:41:26] oh yeah FlameGraph is perl [18:41:32] *shudder* [18:42:10] I have the reaction to the word "perl" that the hipster kids have to "php" -- I've seen very bad things. very bad [18:43:12] I know that perl can be used to make beautiful tools and apps but the horror of maintaining things built by self taught tinkerers sticks with me stronger [19:00:35] <^d> AaronSchulz: Um, https://gerrit.wikimedia.org/r/#/c/167655/ didn't merge lol [19:15:29] is scopedProfileIn the replacement for sub-function level profiling? Is it awful if I create a wrapper function (in Translate) which checks whether it is available? [19:19:49] <^d> Yes. I don't think so. [19:23:32] bd808|LUNCH: No worries, but reminder: Be careful fixing notices. Sometimes the missing key is actually an error. Making the current behaviour explicit (e.g. add isset and fallback to empty array) would then hide the problem and dirty the code. [19:23:47] These are fine though :) [19:24:32] ^d: I think zuul was restarted a minute before that. [19:24:54] <^d> Ah ok :) [19:24:59] Yesterday [19:25:23] Yeah https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:25:25] Sorry :) [19:26:13] hoo: review the tests I added https://gerrit.wikimedia.org/r/#/c/182467/ ? [19:26:26] Sure [19:26:32] That class is horrible [19:28:50] legoktm: Look good to me :) [19:30:10] legoktm: If Special:MergeAccount works fine (at some point...) can we also reenable automerge? [19:30:19] Same issue, I presume [19:31:10] https://people.wikimedia.org/~hoo/tmp/LocalAccs.txt [19:31:37] There are only a few names for which we have more than 50 local accounts (and no global account) [19:32:44] hoo: yeah, but we need to get your fix backported first right? [19:33:07] yeah. I'll submit it for SWAT on Monday, if I can be around [19:43:52] <^d> ori: I think I narrowed down my forking problem a bit. It's yelling when the child is done and we're trying to notify the parent I think. [20:01:38] "JavaScript's RegExp compiles the from pattern. Therefore no optimization is needed beforehand" [20:01:39] * Reedy sighs [20:03:28] Nikerabbit: also consider just breaking up the method [20:03:42] scopedProfile should not be a crutch to work around that [20:04:03] (I haven't looked at this particular code though, just speaking generally) [20:06:07] <^d> +1 [20:06:09] AaronSchulz: I have some http calls and then this monster: https://github.com/wikimedia/mediawiki-extensions-Translate/blob/master/MessageGroups.php#L77 (and also https://github.com/wikimedia/mediawiki-extensions-Translate/blob/master/api/ApiQueryMessageGroups.php#L112 ) [20:18:49] <^d> AaronSchulz: Trivial: https://gerrit.wikimedia.org/r/#/c/183900/ [20:20:56] <^d> And https://gerrit.wikimedia.org/r/183902 [20:24:39] 3MediaWiki-extensions-TitleBlacklist, MediaWiki-Core-Team: Title blacklist intermittently failing, allowing users to edit things they shouldn't be able to - https://phabricator.wikimedia.org/T85428#966676 (10Steel1943) Looks like the software' ability to enforce the title blacklist on enwiki is down again. Are t... [20:31:07] legoktm, i can haz an example extension conversion? [20:32:08] MaxSem: https://gerrit.wikimedia.org/r/#/c/166704/ but that might not be using the latest schema [20:32:44] hmm, it's also incomplete [20:32:56] what's it missing? [20:33:04] I created that back in december so... [20:33:27] a plug in extension php file? [20:38:03] you mean something for back-compat? I'm not sure if we're adding those yet. [20:39:04] so... just nuke the php file? [20:40:19] does this new method support specifying a required mw version? [20:41:12] yeah nuke it or have it die with an explanation. probably will need some updates to jenkins for that... [20:41:54] No, but it should be extremely trivial to add. I wanted to keep the initial implementation just matching feature parity with nothing new. [20:43:41] https://gerrit.wikimedia.org/r/183906 :P [20:44:57] damn [20:45:03] looks like a few errors in the script though [20:45:10] "SpecialPages": null, [20:45:20] and "localBasePath": "/Users/msemenik/dev/vagrant/mediawiki/extensions/MobileFrontend", [20:46:18] legoktm, https://gist.github.com/MaxSem/7a70d1289d43246090d3 [20:46:52] very likely https://phabricator.wikimedia.org/T86311 [20:47:47] also, how extensions are supposed to augment core variables? [20:49:46] like which ones? if it makes sense for extensions to append to, it should be added to the schema, otherwise you can do it via the "callback" feature [21:00:03] * ^d shall convert Cirrus [21:01:31] well, don't merge anything yet because we need a plan to switch over prod still [21:22:13] 3MediaWiki-Core-Team, MediaWiki-extensions-ContentTranslation, ContentTranslation-Deployments: Content Translation Beta Feature security review - https://phabricator.wikimedia.org/T85686#966963 (10csteipp) The Drafts feature allows for stored xss. As one user, I add javascript to a draft with something like, c... [22:06:39] <^d> legoktm: The extension.json can exist in masters without problem, right? [22:06:53] <^d> I suppose I'd have to make it coexist with the old way for now [22:08:24] ori: Adding your js event error display script has made using mw.o very sad. Missing property "action" warnings on every link I click. [22:24:41] ^d: yup, you'll just have to remember to maintain both [22:24:52] <^d> I wait [22:26:54] xD [22:33:09] ori: I gave you and arc lamp a shout out in the December librarization report -- https://www.mediawiki.org/wiki/Library_infrastructure_for_MediaWiki/status#2014-12-monthly [22:33:47] ^d, legoktm, AaronSchulz: Take a look and see if I missed any big ticket items plz -- https://www.mediawiki.org/wiki/Library_infrastructure_for_MediaWiki/status#2014-12-monthly [22:35:40] <^d> Sounds about right [22:36:02] bd808: do you want to mention the issue csteipp found in monolog was patched? [22:36:31] legoktm: Sure that sounds like a good thing to promote. [22:37:09] it was pretty low risk right? not outing anything spooky? [22:38:46] should be imo [22:41:20] legoktm: https://www.mediawiki.org/w/index.php?title=Library_infrastructure_for_MediaWiki/status&diff=1349062&oldid=1348970 [22:57:47] bd808: mw.o? [22:58:03] mediawiki.org [22:58:22] oh, there must be rampant schema violations of some kind. [22:58:29] what is the full error? [22:58:43] let me trigger it again... [22:58:47] that's the snippet doing what it should, which is making schema violations hard to ignore. [22:59:24] [undefined] Missing property "action" [22:59:25] It'd be optimal if there was a way to ignore any future schema violations for that particular schema/revision but I haven't gotten around to doing that yet. [22:59:35] [undefined] Value null is the wrong type for property "duration" (integer expected) [22:59:59] The easiest way to trigger is to click on a TOC link [23:00:26] didn't happen for me [23:00:33] on https://www.mediawiki.org/wiki/Manual:System_administration [23:00:45] stack track looks like its related to popups extension [23:01:02] ah, let me try and enable that beta feature [23:01:24] yep, that's it. [23:01:28] thanks for flagging that. [23:01:34] \o/ [23:01:39] bd808: seems fine [23:02:37] AaronSchulz: thanks for looking. and thanks for all the profiler work you did too! [23:04:51] bd808: I'm thinking about leaving MWException mostly and just changing things to throw/catch Exception (or make a trivial subclass if needed) [23:05:18] there are several GUI exception classes that extend it, which aren't really reusable anyway [23:05:28] *nod* [23:06:19] That reminds me I need to work on a patch to move the logger factory out of MWLogger and kill that class before folks really start using it [23:32:07] Can you define a global function inside of a class's static function? [23:32:19] I think I should be able to, but my test is still failing. [23:34:15] http://3v4l.org/f3QuZ [23:34:18] * legoktm very confused [23:38:55] 3MediaWiki-Core-Team, MediaWiki-extensions-ContentTranslation, ContentTranslation-Deployments: Content Translation Beta Feature security review - https://phabricator.wikimedia.org/T85686#967526 (10csteipp) There doesn't seem to be the ability to delete/suppress drafts, even though they are publicly available to... [23:53:47] "Can you define a global function inside of a class's static function?" dear god why? [23:54:08] legoktm: are you trying to invent java's inner classes in php? [23:54:44] bd808: I was trying to stub a global function for a test https://gerrit.wikimedia.org/r/184003 [23:55:08] my issue ended up being that the phpunit function is called setUpBeforeClass, not setUpClass >.< [23:56:30] The idea that that syntax works to produce a globally scoped function makes me queasy. [23:57:37] but it works :P