[00:09:59] ^demon|lunch: ew, this bot kinda sucks [00:10:09] [16:08:47] b' \x02SwiftOnSecurity\x02 "I don\'t want autoupdates. I want to decide." THEN TURN THE UPDATES OFF, NERD. YOU KNOW HOW.' [00:10:46] <^demon|lunch> Ugly output is ugly. [00:11:25] I bet it'll work in python2. [00:11:42] * legoktm grumbles [00:11:43] <^demon|lunch> legoktm: Reply is funny. [00:11:44] <^demon|lunch> "I HAVE A RIGHT TO BE VULNERABLE, OK?" [00:14:31] heh, it won't even start in python2. [00:16:07] I'll fix this later, blegh [00:20:28] The phrase 24/7/365 has always bugged me. It should be 24/7/52 to match the progression [00:25:57] 3MediaWiki-Core-Team, Wikimedia-General-or-Unknown: Certain users are unable to log into their account (HTTP 503 upon login attempt) - https://phabricator.wikimedia.org/T75462#849667 (10csteipp) I tried to reproduce this by creating a new user, deleting the global account, adding another local account with a dif... [00:27:25] <^demon|lunch> bd808: {{sofixit}} [00:39:38] ^d: I could fix this usage in the topic but I don't think I can fix the common idiom :) [00:39:53] <^d> Hey, do what you can :) [00:40:02] <^d> Nobody's asking you to solve all the world's problems! [00:42:53] AaronS: how much suffering would i incur in the afterlife if i were to try and do something like this? . this would only run on metawiki for the schema NS, which has 20 or so log items where action = 'move'. [00:46:16] looking [00:46:40] the result would be cached for a long time [00:48:51] <^d> It'll filesort. [00:49:09] <^d> | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | [00:49:09] <^d> +------+-------------+---------+------+-----------------+-----------+---------+-------+------+----------------------------------------------------+ [00:49:10] <^d> | 1 | SIMPLE | logging | ref | page_time,times | page_time | 4 | const | 1 | Using index condition; Using where; Using filesort | [00:49:55] where there deletes and restores? [00:50:12] potentially [00:50:26] this fails to account for that, right? hm [00:52:49] ori: when I first wrote FR I started with this approach and then abandoned it [00:53:04] e.g. "what text was at this template at time X?" [00:53:16] i'll just disallow moves in the schema ns and call it a day, i think [01:19:50] heh [01:34:56] Krinkle: sorry, was on a plane most of the day today, I'll look at that issue shortly [01:37:30] bd808: you're online? regarding that monolog/irc feed issue (https://phabricator.wikimedia.org/T78599), things will be "ok" if we don't push more config changes (right?), so I shouldn't worry about tomorrow too much with it? [01:38:18] greg-g: Yeah. It's a config only thing and shouldn't block rolling to group1/group2 at all [01:38:39] I can switch back to the old logger on mw.o if its causing any real grief too [01:39:02] * bd808 is glad he decided to gate this with a feature flag [01:48:50] bd808|BUFFER: doesn't seem like it's a huge deal now, so status quo is fine [01:49:04] bd808|BUFFER: I could be convinced otherwise, but that's my read. [01:58:36] 3MediaWiki-Core-Team: Logs flooded with "MessageCache::parse called by ..." - https://phabricator.wikimedia.org/T78517#849768 (10greg) (Adding @hashar and @mmodell in case this throws off beta cluster at all) [03:52:52] 3MediaWiki-Core-Team, MediaWiki-API, MediaWiki-Parser: Carriage return (\r) line endings not being normalized before being passed to the preprocessor - https://phabricator.wikimedia.org/T78488#849793 (10Jackmcbarn) @Anomie: Rather than modify both the PST and Scribunto (and probably other places in the future),... [07:24:53] manybubbles: ogm, visualvm is awesome [07:25:03] *omg [09:11:32] _joe_: I think the leak also exists in 3.3 but not nearly as severely [09:11:50] this is the diff: https://github.com/facebook/hhvm/commit/12f44e0498189ab75dbdbdc64b76fbe4183af168 [09:12:34] and it's only with certain boost versions. I think ubuntu trusty has one of the bad ones [09:54:29] <_joe_> swtaarrs: thanks [09:54:38] <_joe_> swtaarrs: I *may* have a mega-bug incoming [09:54:49] uh oh [09:54:57] <_joe_> but I strongly suspect it's in apache and not in hhvm [09:55:17] <_joe_> I still need to do more digging, it would be in the fastcgi code anyways [09:55:59] <_joe_> it's typically something in the mediawiki-apache-mod-fastcgi-hhvm mix, I'm checking right now [09:56:02] what's going wrong? [09:56:07] heh [09:56:14] there's a lot of software in that mix :) [09:56:42] <_joe_> certain very particular requests fail if they contain an If-modified-since header that would make the software respond with a 304 [09:57:18] <_joe_> but I have a whole day to figure this out :) [09:57:42] huh [09:59:29] <_joe_> right now my guess is - the ajax calls are marked as HTTP/1.0, and mod_proxy_fastcgi refuses response codes not allowed in HTTP/1.0 [09:59:36] <_joe_> but I'm still doing tests [10:06:06] <_joe_> swtaarrs: it's not user-impacting anyway, luckily [10:09:23] cool [10:21:58] 3Wikimedia-General-or-Unknown, MediaWiki-Core-Team: Certain users are unable to log into their account (HTTP 503 upon login attempt) - https://phabricator.wikimedia.org/T75462#850060 (10Aklapper) Another case mentioned at https://en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_%28technical%29&oldid=638... [12:37:27] <_joe_> swtaarrs: it was the craptastic mod_proxy_fastcgi again, no HHVM bug :) https://phabricator.wikimedia.org/P158 if you're curious [13:08:31] _joe_: hah, whew :) [13:08:57] <_joe_> oh still here, I assumed you would be sleeping by now [13:20:03] 3MediaWiki-Core-Team: Logs flooded with "MessageCache::parse called by ..." - https://phabricator.wikimedia.org/T78517#850367 (10hashar) a:3Legoktm Sampled 1:1000 by @Legoktm with https://gerrit.wikimedia.org/r/#/c/179870/2/wmf-config/InitialiseSettings.php [13:20:29] 3MediaWiki-Core-Team: Logs flooded with "MessageCache::parse called by ..." - https://phabricator.wikimedia.org/T78517#850370 (10hashar) 5Open>3Resolved [13:38:01] _joe_: I'm in france again this week [13:38:22] <_joe_> oh, neat [14:47:49] AaronS: yeah - you see why I said that our profiling system was years behind Java's? PHP in general is catching up but the deep inspection that you could do with visualvm (and jconsole before it) year and years and years ago still compares favorably. [14:48:20] <_joe_> yeah, you can inspect things that don't work [14:48:31] <_joe_> it's so nice [14:49:52] _joe_: could you sling some of that troll php's way too? equal oportunity and all that [14:50:22] <_joe_> in that case, you can't inspect things that don't work [14:50:23] <_joe_> :P [14:51:31] * _joe_ joking [14:52:34] _joe_: as much as I defend java I firmly believe that everything is crap [14:52:44] java probably moreso than most things [14:53:01] <_joe_> manybubbles, I will never like the JVM - there are tons of great low-level performance profiling tools that the jvm renders almost useless [14:53:17] <_joe_> so it's not java, it's the jvm I have most issues with [14:53:17] I have hope for rust, but only really because it hasn't had enough time expose issues [14:53:40] <_joe_> and yeah, all software is crap [14:54:29] <_joe_> excluding the ones I wrote, of course [14:54:39] * _joe_ NIH [14:56:11] 3MediaWiki-API, MediaWiki-Parser, MediaWiki-Core-Team: Carriage return (\r) line endings not being normalized before being passed to the preprocessor - https://phabricator.wikimedia.org/T78488#850631 (10Anomie) >>! In T78488#849793, @Jackmcbarn wrote: > @Anomie: Rather than modify both the PST and Scribunto (and... [15:22:55] 3MediaWiki-Core-Team, MediaWiki-Search: Special:Search on MediaWiki wiki is throwing an error - https://phabricator.wikimedia.org/T78553#850651 (10Manybubbles) a:3Manybubbles [15:23:37] 3MediaWiki-Core-Team, MediaWiki-Search: Special:Search on MediaWiki wiki is throwing an error - https://phabricator.wikimedia.org/T78553#848137 (10Manybubbles) I'm going to grab this. I'm not particularly well qualified for it so if a Javascript expert wants it feel free to trump me. I just think it needs look... [15:45:07] 3MediaWiki-Core-Team, MediaWiki-Search: Special:Search on MediaWiki wiki is throwing an error - https://phabricator.wikimedia.org/T78553#850699 (10Manybubbles) a:5Manybubbles>3rmoen [15:46:25] 3MediaWiki-Core-Team, MediaWiki-Search: Special:Search on MediaWiki wiki is throwing an error - https://phabricator.wikimedia.org/T78553#848137 (10Manybubbles) Reassigning to rmoen as its very likely related to Ic6ec29817ab7deadc6bc7125f8b4cd16686baa04. I don't want to just revert his work because it looks like... [15:50:17] 3MediaWiki-Core-Team, MediaWiki-Search: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#850710 (10Aklapper) [15:50:32] 3MediaWiki-Core-Team, MediaWiki-Search: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#848137 (10Aklapper) [16:28:18] <^d> bd808: #til `vagrant reload` can race itself. [16:28:44] <^d> I wasn't done halting the old vm and clearing ports before it tried to forward ports again and yelled for ports being in use. [16:28:56] <^d> `vagrant halt && vagrant up` solved it. [16:32:20] ^d: I've stubbed my toe on that too I think [16:32:46] <^d> Stupid python question for anyone: how would one go about seeing if a particular string is an ip address (v4 is fine, no v6 in use here)? [16:33:01] <^d> I figure I could regex it easily enough but I was wondering if there's an easier import someone knows of [16:33:33] <_joe_> there is a module for ip validation [16:34:35] <_joe_> ^d: I use ipaddr [16:34:40] <_joe_> (https://pypi.python.org/pypi/ipaddr) [16:35:46] <^d> Ah good, it has a deb package. [16:35:53] <^d> Then I can use it for es-tool :) [16:36:17] <^d> (I didn't want some pip-only thing that would be a PITA to install on elastic* boxen) [16:36:34] <^d> thx _joe_ [16:53:33] <_joe_> yw [17:37:30] 29,120 GlobalTitleFail log events in the last 15 minutes. /me thinks sampling doesn't work in his monolog config [17:38:00] That's just from the group0 wikis [17:40:01] <^d> People should be less fail-y. [17:43:11] Message::toString/Message::parseText/MessageCache::parse is most of them but that tells nothing about how to fix because Message passes this->title [17:43:30] I guess that's why legoktm made the patch to up the trace depth [17:43:46] <^d> Yep [17:44:21] mhm [17:47:33] https://logstash.wikimedia.org/#/dashboard/elasticsearch/GlobalTitleFail [17:52:14] ^d: A +2 of https://gerrit.wikimedia.org/r/#/c/170080/ comes with a free hug! [17:53:03] *(or other reasonable sign of affection subject to mutual approval of the involved parties) [17:54:15] <^d> We could possibly reuse that in ProfilerOutputUDP as well. [17:54:15] <^d> It does its own manual socket creation as well [17:55:31] way to dodge the hug offer [17:55:47] <^d> bd808: +2 [17:56:33] ^d: <3 have your people call my people about claiming your reward [17:59:43] Krinkle: This backport should fix the irc event stream -- https://gerrit.wikimedia.org/r/#/c/180214/ [17:59:55] anomie, ApiQueryExtracts::onApiOpenSearchSuggest (149 samples, 3.12%) [18:00:02] :gigi: [18:01:07] <^d> bd808: I have people? [18:01:16] <^d> :) [18:01:42] MaxSem: Is that good or bad? [18:01:59] used to be 10 times less :) [18:03:07] but yeah, since the rfc to rework the extracts has been accepted, I just need some time to implement it, and it'll be muuuuch faster [18:05:47] <^d> Whee, unsubscribed from another 80 tasks. [18:09:59] completed tasks, I hope? :P [18:13:18] <^d> No, crud I'd accumulated over years of being on BZ. [18:13:22] <^d> I still have like 400 or 500 I'm still subscribed to [18:15:41] <^d> That's just open. I think I'm currently CCd on ~4400 or so tasks, open and closed. [18:15:49] <^d> If I have this conduit query right [18:56:39] <^d> bd808: On my work from home days, fatalmonitor on logstash looks beautiful on a 40in tv :p [18:57:51] What resolution does it have? :P [19:09:52] <^d> hoo: 1080p duh :p [19:10:50] <^d> 4k tvs are still too much. Maybe next year. [19:20:34] 3MediaWiki-Search, MediaWiki-Core-Team: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#851446 (10rmoen) >>! In T78553#850699, @Manybubbles wrote: > Reassigning to rmoen as its very likely related to Ic6ec29817ab7deadc6bc7125f8b4cd16686baa... [19:21:24] 3MediaWiki-Search, MediaWiki-Core-Team: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#851447 (10Manybubbles) Do they have to be logged in? Not that it really matters. Is it something you can fix in an hour or two? If so we should just... [19:24:34] 3MediaWiki-Search, MediaWiki-Core-Team: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#851473 (10rmoen) >>! In T78553#851447, @Manybubbles wrote: > Do they have to be logged in? Not that it really matters. Is it something you can fix in... [19:34:39] 3MediaWiki-Search, MediaWiki-Core-Team: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#851482 (10Manybubbles) Awesome! Can you link it here? We can certainly SWAT it tomorrow morning or this afternoon if we can get it reviewed ASAP. [19:36:22] 3MediaWiki-Search, MediaWiki-Core-Team: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#851486 (10rmoen) https://gerrit.wikimedia.org/r/#/c/180237/ [19:39:27] anomie: https://en.wikipedia.org/w/api.php?callback=jQuery111109750149530017929_1418758663872&search=module%3Aw&action=opensearch&_=1418758663879 [19:44:24] <^d> bd808: Do you have any thoughts on arcanist/phab for vagrant? [19:45:08] ^d: Are those patches still open... [19:46:15] <^d> Yeah [19:46:25] I just +2d [19:46:34] <^d> Phab's still more a WIP. [19:46:42] <^d> You still have to do some manual wrangling after provisioning [19:47:18] What are you having trouble automating? [19:47:40] <^d> Not trouble, just haven't finished it. [19:47:52] <^d> I'm trying to spend today finishing some of my open things rather than starting anything new :) [19:47:59] +1 [20:05:09] hoo, MaxSem: It appears TextExtracts throws a nosuchsection error if you try to get extracts for a page that doesn't support sections and that isn't in the parser cache already. e.g. https://en.wikipedia.org/w/api.php?action=query&prop=extracts&exintro=1&titles=Module:HtmlBuilder at the moment. [20:06:06] :s [20:11:57] mmm, makes sense! :P [20:12:31] also, I should just make it return extracts only for mainspace [20:20:43] * ^d stabs puppet. [20:28:35] <^d> bd808: It works! [20:29:02] stabbing puppet?! [20:29:14] <^d> stabbing it made it behave. [20:29:22] <^d> i got the exec{} calls needed to setup the database. [20:29:42] <^d> Browsing to http://phabricator.local.wmftest.net:8080/ puts you at the initial "make the admin account" page :) [20:29:42] ah. excellent [20:30:10] puppet and exec can be a scary combination to manage. [20:30:23] <^d> This was pretty straightforward, amending my patch. [20:36:56] <^d> Do we not set our default timezone in php.ini in vagrant? [20:37:00] <^d> phab complains after install. [20:37:30] we do [20:37:31] I don't think we do. Actually I don't think we do anything to the stock php.ini do we? [20:37:39] well, for hhvm [20:37:41] * bd808 is maybe really wrong [20:37:55] the way we use hiera sucks, i decided [20:37:58] <^d> I don't see date.timezone anywhere in vagrant. [20:38:07] it's in common.yaml [20:38:15] <^d> Ah, there it is. [20:38:17] <^d> For hhvm right. [20:38:25] <^d> I think we can add a Php::ini call [20:38:28] <^d> Harmless enough [20:39:47] <^d> Patch incoming. [20:40:19] the measure of a good puppet manifest is whether it presents all the essential facts about a software setup in a clear and concise manner [20:40:36] now you have to jump around to get the full picture [20:40:54] <^d> This is the right place to do it. [20:41:09] and what's worse, the manifest provides no indication of which parameters have defaults in hiera [20:41:13] <^d> The init.pp for php module has a couple of other default inis it sets too [20:41:14] so you don't even know what you don't know [20:42:07] <^d> Really either php or the distros should set better default timezones, but I digress. [20:51:14] <^d> How can I get the apache port of 8080 from the manifest? [20:51:26] <^d> I don't see it in hiera. [20:57:30] ^d: it's hidden somewhere... let me find it [20:58:50] ^d: $::port_fragment is probably what you want. See mediawiki::server_url in hieradata/common.yaml [20:59:21] <^d> Got it [20:59:40] It will either be an empty string (on labs-vagrant) or something like ":8080" [21:00:26] $::forwarded_port is the raw port number [21:00:34] Set from Vagrantfile [21:01:23] <^d> Got it, thx [21:24:20] 3MediaWiki-Core-Team, MediaWiki-API, MediaWiki-Parser: Carriage return (\r) line endings not being normalized before being passed to the preprocessor - https://phabricator.wikimedia.org/T78488#851714 (10Anomie) 5Open>3Resolved [21:29:17] ^d: SMalyshev doesn't have +2 in core. do we not trust him? [21:29:31] he is suspicious [21:29:34] <^d> I dunno, he's a shifty guy. [21:29:49] just add him [21:30:00] after the hazing of course [21:30:46] <^d> :) [21:31:40] <^d> He's in wmf ldap group now. [21:33:45] SMalyshev: you should totally have +2 in all the great places now [21:33:52] just don't +2 bad stuff [21:33:59] if you do, you quickly undo it [21:33:59] shuld I log out/log in? [21:34:03] I dunno [21:34:36] ^d: what is the best place to articulate constraints for new article titles in the Schema NS? I currently have it in EditFilterMerged, but is there a better place? (Titles have to be 63 characters or fewer, for example) [21:35:07] ok, now i see a lot of new groups [21:36:30] anomie: ^ in case you have an idea [21:36:58] <^d> Hmm [21:38:31] <^d> https://www.mediawiki.org/wiki/Manual:Hooks/PageContentSave maybe? [21:38:48] <^d> That should be fired before page creation and on all edits. [21:41:43] hmm [21:41:56] ori: No good ideas. Keep in mind page moves too, depending on what hook you pick for page creation. [21:45:59] hmm, right. [21:47:56] WTF mysql [21:48:13] whyyyy does it round floats mercilessly? [21:53:09] <^d> I'm struggling with the best way to do onlyif/unless here. [21:57:24] ^d: If there is something in a config file you can grep for, that's often an easy unless guard [21:57:35] <^d> I'm trying that now :) [21:58:47] I think the scholarships and iegreview roles have some onlyif/unless fanciness in them. The composer class does too [22:02:11] 3Wikimedia-General-or-Unknown, MediaWiki-Core-Team: Certain users are unable to log into their account (HTTP 503 upon login attempt) - https://phabricator.wikimedia.org/T75462#851790 (10Edgars2007) User:Pirags (also hasn't global account) also has the problem https://lv.wikipedia.org/wiki/Vikip%C4%93dija:Kopiena... [22:07:09] <^d> Grrr, phab's json escapes the /'s [22:07:23] <^d> So my naive grep won't work [22:07:26] <^d> (although it works for all but those 2 settings :)) [22:10:45] 3Wikimedia-General-or-Unknown, MediaWiki-Core-Team: Certain users are unable to log into their account (HTTP 503 upon login attempt) - https://phabricator.wikimedia.org/T75462#851814 (10Paju) 1. I wonder if something has been done to fix the problem "...the failure is probably in SpecialUserlogin between lines 7... [22:18:37] 3MediaWiki-Search, MediaWiki-Core-Team: Special:Search on mw.org throwing an error - Advanced search checkboxes broken - https://phabricator.wikimedia.org/T78553#851818 (10rmoen) Evening swat deployment scheduled: https://wikitech.wikimedia.org/w/index.php?title=Deployments&oldid=138343 [22:29:31] <^d> bd808: Ok, latest revision is in a pretty good spot, other than the few comments I left for myself. [22:40:08] ^d: Cool. I'll take a peek. [22:40:25] If you have a minute to read -- https://www.mediawiki.org/wiki/Manual:MWLoggerMonologSpi -- and tell me what's missing that would be great [22:40:41] Mostly wondering if it actually explains anything [22:47:14] arghhhhh [22:47:18] why can I login fine??? [22:49:34] legoktm: is that a problem? ;) [22:49:44] yes :< [22:49:49] I can't reproduce the bug [22:57:02] 3Librarization, MediaWiki-Core-Team: Document use of Monolog with MediaWiki - https://phabricator.wikimedia.org/T78596#851899 (10bd808) Lots more work done on https://www.mediawiki.org/wiki/Manual:MWLoggerMonologSpi [23:00:19] <^d> bd808: https://www.mediawiki.org/w/index.php?title=Requests_for_comment%2FBetter_PHP_profiling&diff=1319424&oldid=1315039 [23:00:50] nice! [23:02:26] ^d: I guess we need a small xhprof install tutorial still. [23:02:38] * bd808 has some phab task about that [23:02:41] <^d> It's on Manual:Profiling/Xhprof [23:02:46] <^d> Linked from Manual:Profiling [23:03:04] <^d> Oh, it's lacking any real info lol [23:03:11] pecl install xhprof [23:03:17] <^d> tarting with MediaWiki 1.25, you can use xhprof to profile code. You can install the pecl package or see if your distribution has a package for xhprof. HHVM has xhprof built in. [23:03:18] and yeah some small config bits [23:03:29] *nod* [23:14:30] <^d> Fixed the unless for updating the db. [23:43:44] 3Wikimedia-General-or-Unknown, MediaWiki-Core-Team: Certain users are unable to log into their account (HTTP 503 upon login attempt) - https://phabricator.wikimedia.org/T75462#851945 (10Legoktm) @ori noticed that there were OOM error mesages corresponding with the same timestamp as the centralauth debug log entr... [23:54:30] 3MediaWiki-extensions-CentralAuth, SUL-Finalization, MediaWiki-Core-Team: Re-enable $wgCentralAuthAutoMigrate = true - https://phabricator.wikimedia.org/T78727#851988 (10Legoktm) 3NEW [23:56:01] 3Wikimedia-General-or-Unknown, MediaWiki-Core-Team: Certain users are unable to log into their account (HTTP 503 upon login attempt) - https://phabricator.wikimedia.org/T75462#775446 (10Legoktm) If another user who was experiencing the 503s can confirm they can login now, I'll go ahead and close this. And filed... [23:56:15] MatmaRex: the login issue should be fixed now, see ^^