[00:26:26] woot, looks like the HHVM team has a patch for https://github.com/facebook/hhvm/issues/4108 finally [00:27:18] (much of the last stretch of the hhvm rollout was spent fighting with memory leaks in the libxml extension) [10:42:55] legoktm: Yikes... any idea why the users to rename table isn't on the dbstores? [11:37:51] hoo: fyi https://gerrit.wikimedia.org/r/#/c/197507/ [11:39:22] Nemo_bis: Are they really fully ok? Don't they also contain oversighted user names? [11:40:09] I did a bit of data processing with them, but never published that [11:40:18] (DerHexer published an aggregated version) [11:41:00] hoo: as I said in the comments, I'm not 100 % sure [11:41:18] Let me check [11:41:23] presumably the view can filter as the special page does [11:42:31] It could, but that would be a cross database view... that's not really nice [11:42:46] but it should be ok as the labsdbs have all tables locally (they aren't federated) [11:43:47] hoo: for a small wiki https://gerrit.wikimedia.org/r/#/c/198121/ would be enough [11:44:08] Then very active users of all small wikis can chase any "big" user [11:45:00] Nemo_bis: If you don't explicitly set the limit, couldn't one just do ?limit=5000 [11:45:11] hoo: and why is that a problem [11:45:26] it's a rather small table, compared to e.g. recentchanges [11:45:32] Because it doesn't batch the queries against user [11:45:47] So you get 5000 User object creations + queries [11:46:01] Will probably still work, but might be slow [11:46:29] Queries for the edit_count field? Can't DOS much [11:46:54] * Nemo_bis checks special page code again [11:50:40] oh, it actually does batch the queries [11:50:41] my bad [11:51:08] approved [11:54:51] Thanks [11:55:12] I'm totally swamped with users that need help today... [11:55:38] hoo: do they email directly? [11:56:14] No, a few used my talk page... but many emailed K.eegan and he forwarded [11:56:35] Ah, ok. I saw only a handful messages on your talk ;) [14:06:41] good morning [15:40:44] bd808: when ya wanna do the grants app deploy next week? [15:43:31] Thursday 19:00Z-21:00Z [15:43:38] greg-g: ^ [15:49:09] greg-g: I just put it on the deployments page [15:49:46] bd808: so I was just told " [15:49:46] Someone else has changed this page since you started editing it. The upper text area contains the page text as it currently exists. Your changes are shown in the lower text area. You will have to merge your changes into the existing text. Only the text in the upper text area will be saved when you press "Save page". [15:49:51] " [15:49:54] ;) [15:50:04] heh [15:50:15] I wish each day was it's own section [15:50:35] you included a link to the phab ticket, so you win (otherwise the same, well, I put your real name in that ircnick template) [15:50:53] s/real/legal/ [15:50:55] ;) [15:51:23] heh. Too many br[iy][ao]n's [16:00:55] <^d> We should have a page on mw.org [16:01:08] <^d> Like we do for Wiki-p-mediawiki [16:03:56] <^d> https://www.mediawiki.org/w/index.php?title=Differences_between_Brion,_Bryan_and_Brian&action=edit&redlink=1 [16:04:30] https://office.wikimedia.org/wiki/Ambiguous_names [16:37:26] <_joe_> bd808: I'd like your feedback on https://gerrit.wikimedia.org/r/#/c/197885/ (no rush, I am off for the weekend now) [16:37:49] cool. I saw that in my email [17:10:47] hoo: want to sanity check my data recovery scripts? [17:11:11] I can have a look [17:13:12] hoo: ok, in my home on terbium, updateUsersToRename.log is the log output from the deletion script, users_to_rename_removals.log is the binlogs. I ran data_recovery.py to create recovery.txt. Then I'll run (haven't yet) dataRecovery.php to re-insert it into the CA database [17:18:08] legoktm: dataRecovery looks sane to me [17:18:36] ok :D [17:19:38] Did Sean give you the binlogs? [17:20:08] hoo: akosiaris did [17:20:49] ah, so it deleted each user individually from the table? [17:20:59] That's producing nice binlogs [17:21:12] I thought you knocked them by some other criteria :P [17:21:24] no, it batch deleted :P [17:21:50] the binlogs are from the notification script which marked each user as notified individually [17:22:16] and we just need the utr_status value, since I have utr_name/utr_wiki from the script output [17:23:27] Ah, I see in "data_recovery.py" ... that make sense to me [17:23:52] 6MediaWiki-Core-Team, 10Wikidata-Query-Service: Wikidata Query: Experiment with RDF exports - https://phabricator.wikimedia.org/T91691#1135793 (10Smalyshev) [17:28:14] hoo: ok, re-inserted :D [17:28:29] Nice :) [18:38:04] hoo|away: if you're still around could you look at https://gerrit.wikimedia.org/r/#/c/198269/ ? I triple checked and tested it this time :/ [18:51:36] looking [18:52:03] Looking as well [18:54:07] aspiecat: why an inner/left join? [18:54:32] easier to read when it's explicit IMO [18:54:58] it would be INNER in this case, I'm speaking generally [18:55:16] Works as is, but having it explicit could be nice [18:56:08] ok, updated [18:57:13] 6MediaWiki-Core-Team, 10MediaWiki-General-or-Unknown, 6Release-Engineering, 5MW-1.23-release, 15User-Bd808-Test: Create a minimal backport of PSR-3 logging to MediaWiki 1.23 LTS - https://phabricator.wikimedia.org/T91653#1136246 (10greg) [18:57:38] bd808: ^ not so secret anymore :P [18:58:27] hoo: should I use utr_id or is the wait for slaves sufficient? [18:59:09] wait for slaves should be sufficent [18:59:15] * sufficient [18:59:21] yeah, barring weird edge cases [19:01:52] 6MediaWiki-Core-Team, 10Datasets-General-or-Unknown, 6Services, 10Wikimedia-Hackathon-2015, 7Epic: Improve Wikimedia dumping infrastructure - https://phabricator.wikimedia.org/T88728#1136264 (10GWicke) For HTML dumps we are currently considering the best distribution format to use. One option on the tabl... [19:03:17] legoktm: Oh noes! More work I hope to get to someday :/ [19:04:24] 6MediaWiki-Core-Team, 10MediaWiki-Debug-Logging, 10MediaWiki-General-or-Unknown, 6Release-Engineering, and 2 others: Create a minimal backport of PSR-3 logging to MediaWiki 1.23 LTS - https://phabricator.wikimedia.org/T91653#1136279 (10bd808) [19:05:31] Did we only start logging user registrations in late 2005? [19:06:12] legoktm: ^ [19:06:13] yup [19:06:35] brion added it for autoconfirmed / semi protection [19:06:50] and AFAIK Tim only ran the back-fill script on enwiki [19:07:22] https://phabricator.wikimedia.org/T20638 is on my list of things to look into [19:08:49] If you do the math right, I guess you can get pretty good estimates based on logs/ revision for at least the big wikis [19:09:45] https://test.wikipedia.org/wiki/File:Example_of_user_first_actions_for_en.wp_1-750000_%28by_thousand%29.gif <-- Splarka proved that it should be possible [19:15:00] :) [19:15:06] legoktm: https://meta.wikimedia.org/wiki/Special:CentralAuth/Frau Any idea what happened hereß [19:15:10] *? [19:15:16] Doesn't seem to be a global rename [19:16:05] hmm [19:16:42] CentralAuth account migration for: Andrea Frau [19:16:42] INFO: Incomplete migration for 'Andrea Frau' [19:16:50] wait no that's a different user [19:17:07] hoo: oh, it's the stupid randomness bug [19:17:35] hoo: basically, I made a mistake in that CentralAuthUser::chooseHomeWiki uses randomness, so you won't always get the same response [19:18:28] so, we first created the list of users to rename, which implicitly picked a winner (the person who isn't being renamed) [19:19:07] and then I ran a modified version of migrateAccount to globalize the winners picked via chooseHomeWiki, except if the "winner' was already marked as being renamed, it just skipped it [19:19:52] and there are about 9.5k usernames affected. [19:20:03] :P But probably not a huge problem [19:20:05] such ties [19:20:42] so I just need to make the script smarter to be able to figure out that if everyone else is being renamed, the last account left must be the winner [19:31:15] hoo, aspiecat: could I get a +2 on https://gerrit.wikimedia.org/r/#/c/198269/ ? [19:31:41] Done [19:32:34] thanks [19:45:00] legoktm: What if have Foo on barwiki and that to rename them to Foo~barwiki, but that already exists... can we handle that, yet? [19:45:29] (I guess that's actually a realistic case... we have a barwiki and there's probably a user Foo :P) [19:45:49] hoo: the script will just try to rename them to Foo~barwiki1, then Foo~barwiki2, etc. Except that doesn't work with the login using old username code last I checked [19:46:23] ok [19:52:11] 6MediaWiki-Core-Team, 10MediaWiki-Debug-Logging, 10MediaWiki-General-or-Unknown, 6Release-Engineering, and 2 others: Create a minimal backport of PSR-3 logging to MediaWiki 1.23 LTS - https://phabricator.wikimedia.org/T91653#1136523 (10bd808) [19:57:23] Yeah, that wouldn't work with the login-with-old-username setup [19:57:49] and I'd really not want to try to make it work unless it's a really really common problem [20:00:58] 6MediaWiki-Core-Team, 10MediaWiki-Debug-Logging: Move MWLogger* classes to PHP namespaces rather than using faux namespaces in the class names - https://phabricator.wikimedia.org/T93406#1136538 (10bd808) [20:24:14] 6MediaWiki-Core-Team: Quick perf review of Gather - https://phabricator.wikimedia.org/T93417#1136638 (10aaron) 3NEW a:3aaron [20:29:13] greg-g: https://phabricator.wikimedia.org/T93418 Adding an extension to beta... does this need a +1 by you in some way? If so, can you please? [20:39:10] hoo: when is it planned to go out to prod? how soon? [20:39:51] greg-g: I don't really have a plan for that... it's not urgent, but having it out before Wikimania would be awesome (and realistic, I think) [20:42:20] * greg-g opens task now :) [20:46:08] hoo: "License No license specified" on https://www.mediawiki.org/wiki/Extension:Capiunto :P [20:48:19] fixed [20:49:53] 6MediaWiki-Core-Team: Quick perf review of Gather - https://phabricator.wikimedia.org/T93417#1136756 (10aaron) [20:55:19] csteipp: Any idea what a "measurable result" for making OAuth better would be? [20:55:40] * bd808 is trying to fill in some Q4 goal template junk [21:01:07] greg-g: Awesome picture :'D [21:02:44] bd808: More users authorizing? More authorizations? (we don't track those, but it would be an easy db query) [21:03:24] bd808: My basic kpi's for health are number of apps using oauth, and number of authorizations.... I just don't actually track them. [21:05:06] csteipp: sounds reasonable. A bit of a community outreach goal implicit in there [21:05:14] but that's ok I think [21:13:15] Are other people getting phabricator emails today? I haven't received any since yesterday, and realized I've missed a couple things... [21:13:52] Works flawless for me [21:14:09] Last thing I got was Greg's reply to my bug and that came within seconds of the change [21:14:35] Oh, gmail is sending them all to spam. Awesome. [21:18:28] <^d> csteipp: It knew what you were really thinking :p [21:22:50] csteipp: which database has the oath tables in prod? mediawikiwiki? [21:22:58] *oauth [21:24:24] yeah, mediawikiwiki [21:43:15] we have 34 apps that have more than 1 user that have been used this quarter [21:43:38] of 96 approved apps [21:44:23] number goes up to 47 when I include single user apps [22:10:02] Okay, so Special:GlobalRenameRequest, when a request is completed, is notifying all accounts with that name that the request was fulfilled [22:10:13] So a bunch of users think they're being renamed :/ [22:10:28] ack! [22:10:36] I'm filing the bug [22:12:03] $oldUser->sendMail( $subject, $body ); [22:12:14] that should only be the renamed user... [22:12:58] Or is there some other notification besides the email? [22:13:46] Just the mail, AFAIK [22:13:51] It's not logged on wiki [22:15:43] https://phabricator.wikimedia.org/T93444 [22:17:36] stewards are doing all approvals from meta? [22:18:33] * bd808 has a hunch what's wrong with this code [22:20:14] legoktm: I think we missed a big problem in SpecialGlobalRenameQueue with the email notification :( [22:20:46] User::newFromName() is going to be the locally attached user on whatever wiki is processing the request [22:21:07] that means we are going to email whichever user is attached to meta [22:21:12] :/ [22:22:59] wait... this code doesn't even work if the user isn't already global [22:23:34] I thought we had talked about this at one point... [22:24:11] This should probably be a job [22:24:17] oh there's a branch for unattached users [22:24:22] that does the rename correctly [22:24:32] but not the notification email [22:25:12] centralauthrename.log should have a list of all users who were incorrectly notified [22:25:26] though it appears to be emailing people without emails set? [22:26:38] It's emailing whatever account is attached to meta with the name that is being renamed [22:27:36] And assuming that the user has an email because the request form requires that [22:27:41] that's... yikes [22:27:59] yeah it's not right at all [22:28:03] so I guess we need to do a cross-wiki db lookup to get the right email? [22:28:27] Make it a job, that's probably the easiest [22:29:12] cross wiki db or a job or move the email logic into LocalRenameUserJob when it is used [22:29:33] job is probably easiest like hoo says [22:29:50] ok, job then [22:29:56] sounds good to me! [22:30:14] mh... will we run into timing issues with jobs (where the state of the rename matters)? [22:30:20] * legoktm is moving to different internet, brb in 10ish [22:30:34] If so, then we would probably need to get the email address per hand :/ [22:31:29] yeah. that's what led to this mess actually [22:32:27] https://github.com/wikimedia/mediawiki-extensions-CentralAuth/blob/master/includes/specials/SpecialGlobalRenameQueue.php#L374-L379 [22:33:01] Oh, I remember approving that patch [22:33:52] heh. all your fault! [22:34:41] * hoo looks at UserMailer... ewk [22:35:19] If we want to do it properly, I guess we need both: A lookup for the email address pre-rename and then a job in local wiki context to send the mail (via UserMailer) [22:35:49] Yeah, that would work I think. [22:36:27] if ( $request->userIsGlobal() ) { send email } else { assume the rename job did it } [22:36:58] This will all work brilliantly once all the users are global ;) [22:39:08] Like so many things... [22:40:44] so how in the hell do I fix this? [22:41:50] UserMailer doesn't need a user name, so just using the mail address as obtained from the user table should be fine [22:42:12] if the account is global, use the global email address, if not, read the user table of the user's wiki [22:42:21] I guess that suffices [22:42:35] I've never done a cross wiki lookup. Is there an easy to understand example somewhere? [22:42:58] Probably, but it's not hard at all [22:44:22] $lb = wfGetLB( $wiki ); [22:44:22] $localDB = $lb->getConnection( DB_SLAVE, array(), $wiki ); [22:45:13] $lb->reuseConnection( $localDB ); [22:45:20] don't forget that, after you're done [22:45:30] $localDB is a DatabaseBase [22:46:42] <^d> bd808: "Core" is silly and redundant :) [22:47:17] ^d: k. I started to respond and got sidetracked [22:51:22] I like Gergo's suggestion of $IP/src too [22:51:32] I wasn't bold enough to suggest that in the first place [22:53:27] uhhh please no [22:53:43] I wanna have 1 directory to grep [22:53:55] not includes and src [22:54:05] <^d> No need, we'll just git mv includes src [22:54:07] <^d> ;-) [22:54:19] APOCALYPSE [22:54:39] yeah that would appease no one actually [22:55:15] I'd rather have incdues/somedamnthing and then rename that to src when all the bits are inside it [22:55:30] also, can we put whole mw into a namespace and proclaim 2.0? [22:55:36] MaxSem: you already have to grep language too :P [22:55:54] yes but it will take more than a week or two [22:55:54] nothing useful there usually [22:56:35] Imagine a word where you can know what file to open just by knowing the class name. So crazy [22:57:45] wtf is that for, anyway? completely useless feature even with crappy ides [22:59:06] I have never worked in a codebase that didn't have that "completely useless" feature. At least I've never left a codebase in the state of multiple classes per file and random directory structure [22:59:29] and all ides are crappy [23:01:48] <^d> Namespaces completely breaks back-compat heh [23:02:20] the bandaid has to come off at some point. PHP 5.5+ will do the same thing really [23:06:28] I guess 5.5+ breaks in a different way though [23:06:53] namespacing User would break the whole damn world [23:07:13] namespacing new things seems not horrible though [23:07:43] <^d> I mean it's an immediate and irreversible break. [23:07:57] <^d> We'd have to update every extension in deployment on day 1 :) [23:08:10] I'd even be ok with PEAR faux namespaces. I just can't stand the everything wondering if there is a collision when a name is picked mess [23:08:19] <^d> Assuming you namespace something like User or Title [23:09:21] <^d> Do we have an RfC on widespread namespace use in core? [23:15:12] 6MediaWiki-Core-Team, 10MediaWiki-API, 5Patch-For-Review: let list=logevents use the new LogFormatter to format its output - https://phabricator.wikimedia.org/T35235#1137269 (10ksmith) [23:15:13] 6MediaWiki-Core-Team, 10MediaWiki-API, 5Patch-For-Review: API: Add jsconfigvars to action=parse - https://phabricator.wikimedia.org/T67015#1137268 (10ksmith) [23:15:14] 6MediaWiki-Core-Team, 10MediaWiki-API, 5Patch-For-Review: Clean up core API data formats for new formatversion - https://phabricator.wikimedia.org/T87053#1137271 (10ksmith) [23:15:16] 6MediaWiki-Core-Team, 10MediaWiki-API, 5Patch-For-Review: list=logevents tries to return both the creating user's and the created user's ids as "userid" - https://phabricator.wikimedia.org/T73020#1137270 (10ksmith) [23:28:49] Is there really no way redefine a class under the old name after it's been moved into a namespace? Seems like there should be a way, even if it's horribly ugly. [23:29:31] Add a stub class [23:29:50] there is the class_alias thing too but it really sux [23:29:56] hhvm doesn't like it [23:30:06] Oh yeah, can't you just have User extends \MediaWiki\Core\User {}? [23:30:28] No [23:30:41] then \MediaWiki\Core\User is not instanceof User [23:30:56] You would need them to inherit each other, which doesn't work [23:31:06] did hhvm yet fix there class_alias? [23:32:15] 6MediaWiki-Core-Team, 10Analytics, 10VisualEditor, 10VisualEditor-Performance, and 3 others: Apply Schema:Edit instrumentation to WikiEditor - https://phabricator.wikimedia.org/T88027#1137389 (10Jdforrester-WMF) [23:32:28] I think it *sort of* works [23:32:49] but it causes problems with repoauthoratative and their JIT I think [23:33:17] uhh. can we just rewrite MW in rails, gonna be much less painful [23:34:07] You can do your fork in whatever you want MaxSem [23:34:24] deal [23:34:33] * MaxSem turns his fork into a psoon [23:34:35] * bd808 picks python [23:34:42] *spoon [23:36:42] bd808, we will probably need a RVM (Rap Virtual Machine) for Python, but I'm all for it! [23:57:44] 6MediaWiki-Core-Team: Quick perf review of Gather - https://phabricator.wikimedia.org/T93417#1137508 (10aaron) ApiQueryLists is a bit hard to read but I see some queries that could be batched, like the isInWatchlist() method, some User::newFromId() calls. I wonder if the max limits for that API should be lowered... [23:58:22] 6MediaWiki-Core-Team: Quick perf review of Gather - https://phabricator.wikimedia.org/T93417#1137509 (10aaron) 5Open>3Resolved I don't see any huge blockers here though.