[00:00:04] Keegan: Our existing dashboarding frameworks are designed for EventLogging afaik. [00:00:29] Keegan: A dump into a table would probably be easier. [00:00:31] ...that still doesn't mean I know how to do all that majik :D [00:00:46] Keegan: So that's 639 accounts out of how many? [00:02:12] I'm not sure the exact number legoktm stopped at today, he's doing it in batches. [00:02:31] What was the overall exact number again, legoktm? IIRC it was somewhere around 87k [00:02:43] * legoktm checks [00:02:45] But I'm mentally tired from today :) [00:03:53] 81k total, and we sent out...7.6k so far (started slow today) [00:05:39] Not bad [00:07:31] Dinner. Deskana, I'll touch base with you sometime tomorrow if that's cool with you [00:13:45] Keegan|Away: Sure thing [00:26:08] legoktm: What was the text of the email that was sent out? [00:26:11] legoktm: Chatting to Erik. [00:26:17] legoktm: He was curious. [00:30:26] Deskana: https://translatewiki.net/wiki/MediaWiki:Centralauth-finishglobaliseemail_body/en [00:30:48] s/translatewiki.net/{{SITENAME}} [00:30:50] / [00:38:39] legoktm: Cheers. :-) [00:40:01] Hey ori, where is the best place to report hhvm bugs? github? [00:44:45] csteipp: yeah [00:45:08] ori: potential security bugs too? [00:45:09] csteipp: and then pinging swtaarrs if you want to have it looked at more quickly [00:45:18] * swtaarrs looks up [00:45:34] ah [00:45:35] swtaarrs: do potential HHVM security bugs go on github? [00:45:51] csteipp: you can send those to me directly [00:45:56] Deskana, Keegan|Away: https://www.mediawiki.org/wiki/User:Legoktm/graph is how many account merges we've had per day, including the auto-merge stuff on login and maintence scripts (which is the large bump in October). [00:46:51] Sweeeet [01:38:50] bd808: I'm not sure why you're jumping in on T88813 [01:39:22] Because Kevin put it high on his list of thing he wanted help from mw-core with [01:39:31] oh, makes sense [01:39:40] as an immediate priority actually [01:39:53] the code changes have to do with VCL [01:39:53] but reasonable to check my sanity [01:40:00] what does he need us for? [01:40:04] presumably not legal review [01:40:34] probably really nothing [01:40:56] is the mw-core thing visible anywhere? [01:40:59] except maybe some handholding on getting a vcl change done and perfomance [01:41:04] the fact that kevin put it high on a list [01:48:14] bd808: um, how do we delete jobs that got queued on the wrong wiki and don't have a class set for them? GlobalUserPage accidentally did it... [01:48:37] careful redis surgery? [01:48:42] AaronS: ^^ [01:49:29] legoktm@fluorine:/a/mw-log$ grep LocalGlobal exception.log | wc -l [01:49:30] 7570862 [01:49:30] :/ [01:49:46] tres mal [01:50:40] didn't we clean up something like this last month? [01:51:07] stuck busted job that was puking all over the logs I mean [01:52:22] yeah [01:52:53] did we take notes somewhere? [01:53:56] https://phabricator.wikimedia.org/T87398 [01:54:07] I guess not :( [01:54:44] i think bd808 updated wikitech [01:56:47] hmmm... not seeing anything obvious in my wikitech or mw.o contributions lists [01:58:02] I'd like to try $wgJobClasses['LocalGlobalUserPageCacheUpdateJob'] = 'NullJob'; and see if that gets rid of all the jobs? [01:58:54] that should work I think [01:59:00] 3Datasets-General-or-Unknown, MediaWiki-Core-Team, Services: Improve Wikimedia dumping infrustructure - https://phabricator.wikimedia.org/T88728#1035923 (10RobLa-WMF) I would like this to be considered as something that the #MediaWiki-Core-Team does in concert with the #Services team. This seems like the kind o... [02:00:13] grmbl, and fxing today's woes would have been SO much easier with logstash [02:06:46] bd808: hmm, doesn't seem to have worked :/ [02:07:30] bd808: shouldn't /rpc/RunJobs.php reload the config on every request? [02:08:01] legoktm: ummm yes? [02:08:26] it's just a http request to php so it's stateless [02:08:34] I think [02:08:38] it's still throwing exceptions :/ [02:08:51] i'll restart it [02:09:54] MaxSem: coming soon! I hope. There's an RT ticket for hardware quotes now [02:10:13] and I'm ready to test syslog datagrams as transport in beta [02:10:23] legoktm: better? [02:10:24] could've just ditched hadoop in meanwhile ;] [02:10:48] bd808: syslog-formatted datagrams or UDP->rsyslog->logstash? [02:11:05] ori: nope :( [02:11:33] what did you restart? [02:11:43] ok, time to reverse-engineer the job runner again [02:12:00] ori: syslog formatted udp that can go either directly to logstash or be relayed via rsyslog. Planning on direct path first and ops can bring rsyslog into the mix if they decided they want it [02:12:09] direct seems more sensible [02:12:14] bd808: i'm *sure* we wrote it somewhere [02:12:17] the job queue thing [02:12:19] but where? [02:12:36] a phab task? [02:12:47] searching phab is a nightmare right now [02:15:24] 3Datasets-General-or-Unknown, MediaWiki-Core-Team, Services: Improve Wikimedia dumping infrustructure - https://phabricator.wikimedia.org/T88728#1035954 (10GWicke) RESTBase can indeed help, especially with [HTML dumps](https://github.com/gwicke/htmldumper). We might try to do one manual pass of HTML dumps after... [02:15:30] what was that problem last time ... [02:15:49] oh exception trying to create the job class itself right? [02:16:16] https://phabricator.wikimedia.org/T87360 [02:16:48] redis-cli -h rdb1001.eqiad.wmnet -a $PASSWORD hdel jobqueue:aggregator:h-ready-queues:v2 LocalRenameUserJob/vewikimedia [02:18:10] yeah, already figured it out again :P [02:18:14] heh [02:19:35] legoktm: better? [02:20:44] now it's DBConnectionErrors? :/ [02:21:09] 2015-02-13 02:20:39 mw1016 labswiki: [ba17b29b] /rpc/RunJobs.php?wiki=labswiki&type=LocalGlobalUserPageCacheUpdateJob&maxtime=60&maxmem=300M JobQueueConnectionError from line 770 of /srv/mediawiki/php-1.25wmf16/includes/jobqueue/JobQueueDB.php: DBConnectionError:DB connection error: Access denied for user 'wikiuser'@'10.64.0.46' (using password: YES) (208.80.154.136) [02:21:14] only labswiki though [02:21:46] nothing in the last minute, though [02:22:38] I see one from 02:22:25 [02:23:16] in which log file? [02:23:31] /a/mw-log/exception.log [02:24:30] who is in charge of migrating wikitech to the cluster? [02:26:18] i'm just going to put the whole jobqueue-eqiad.php in an if $wgDBname !== labswiki block [02:27:47] legoktm: is there a task for this? [02:27:59] no, do you want me to file one? [02:28:14] please [02:30:36] Andrew Bogott, Sam and I mostly did the migration when it was on virt1000. I haven't really helped with the move to silver except debugging to php5 config issues they had there [02:31:42] We could have missed random stuff on virt1000 because it was firewalled off from the prod cluster resources and Sam and I didn't have access to logs [02:32:11] it was just "oops still broke", ask andrew to look for things and then make more config patches [02:32:15] yeah no problem, just wondering whom to ping [02:33:26] i can hack the jobrunner [02:33:53] Andrew was at our mercy for the mulit-config stuff. Today I guess the same folks you'd poke for any other config cleanup [02:35:27] well, i think we just need to remove other job types for labswiki from the ready queue [02:35:31] and hopefully they won't be re-inserted [02:36:37] 3GlobalUserPage, MediaWiki-Core-Team: GlobalUserPage jobs stuck on wikis where extension is not deployed yet - https://phabricator.wikimedia.org/T89427#1035973 (10Legoktm) 3NEW [02:36:44] done [02:37:22] 3SUL-Finalization, MediaWiki-JobQueue, MediaWiki-JobRunner, MediaWiki-Core-Team: Bad LocalRenameUserJob stuck in jobrunner for vewikimedia - https://phabricator.wikimedia.org/T87360#1035982 (10bd808) Trying to make the answers here easier to find the next time I'm looking for them by adding the job queue and run... [02:38:09] 3GlobalUserPage, MediaWiki-Core-Team: GlobalUserPage jobs stuck on wikis where extension is not deployed yet - https://phabricator.wikimedia.org/T89427#1035987 (10ori) I ran: ``` redis-cli -a $REDIS_PASSWORD hdel jobqueue:aggregator:h-ready-queues:v2 LocalGlobalUserPageCacheUpdateJob/labswiki ``` [02:38:50] legoktm: I think it stopped, no? [02:39:21] yup, I don't see any more errors [02:39:24] thanks :) [02:39:41] OK, I hope that does it. [02:39:50] I'm off, ttyl. [02:40:01] o/ bye [02:40:20] night all [02:40:25] in other exciting news, vagrant is now in the Fedora 21 stable repo! [02:42:21] 3GlobalUserPage, MediaWiki-Core-Team: GlobalUserPage jobs stuck on wikis where extension is not deployed yet - https://phabricator.wikimedia.org/T89427#1035988 (10Legoktm) 5Open>3Resolved a:3Legoktm This is fixed now. [02:54:27] > Fatal error: plus on $GLOBALS is not implemented yet. :| [02:54:49] heh, array_merge doesn't work either >.> [04:49:43] 3MediaWiki-Core-Team: Try to avoid doCascadeProtectionUpdates() call on page view - https://phabricator.wikimedia.org/T89389#1035191 (10aaron) [04:49:44] 3MediaWiki-Core-Team: MediaWiki multi-datacenter investigation and work - https://phabricator.wikimedia.org/T88445#1036055 (10aaron) [05:24:36] 3Parsoid, MediaWiki-Core-Team, Services: Replace Tidy in MW parser with HTML 5 parse/reserialize - https://phabricator.wikimedia.org/T89331#1036066 (10GWicke) [08:09:38] 3MediaWiki-API, MediaWiki-Core-Team, Librarization: Allow API to retrieve information about libraries as shown on "Special:Version" - https://phabricator.wikimedia.org/T89385#1035089 (10Ricordisamoa) [08:15:07] TimStarling: I uploaded https://gerrit.wikimedia.org/r/190419, and ended up having to special case $wgGroupPermissions as well [09:21:41] 3MediaWiki-API, MediaWiki-Core-Team, Librarization: Allow API to retrieve information about libraries as shown on "Special:Version" - https://phabricator.wikimedia.org/T89385#1035089 (10Kghbln) You rock! [14:00:26] 3Datasets-General-or-Unknown, MediaWiki-Core-Team, Services: Improve Wikimedia dumping infrustructure - https://phabricator.wikimedia.org/T88728#1036612 (10ArielGlenn) They attempt to serve a number of audiences and so probably satisfy no one either in format or in speed or in contents. They take a long time to... [14:49:24] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1036699 (10Manybubbles) Hey! Does BigData support GeoSpatial queries? I see https://github.com/varunshaji/bigdata-geosparql but I'm not sure how well supported it is. [15:08:26] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1036715 (10Beebs.systap) >>! In T88717#1036699, @Manybubbles wrote: > Hey! Does BigData support GeoSpatial queries? I see https://github.com/varunshaji/bigdata-geosparql but I'm not sure how wel... [15:16:58] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1036720 (10Manybubbles) >>! In T88717#1036715, @Beebs.systap wrote: >>>! In T88717#1036699, @Manybubbles wrote: >> Hey! Does BigData support GeoSpatial queries? I see https://github.com/varunsha... [15:19:42] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1036728 (10Beebs.systap) Actually, the github was a first heard for us! That was for another project. [15:57:41] 3MediaWiki-Core-Team: Devise stashing strategy for multi-DC mediawiki - https://phabricator.wikimedia.org/T88493#1036757 (10bd808) [15:57:48] 3MediaWiki-Core-Team: Remove cache anti-dependencies - https://phabricator.wikimedia.org/T89184#1036758 (10bd808) [15:58:05] 3MediaWiki-Core-Team: Make a ReplicatedBagOStuff class for multi-DC use - https://phabricator.wikimedia.org/T88634#1036764 (10bd808) [16:09:43] _joe_: All jobrunners are running on trusty and hhvm now right? [16:41:28] bd808: Given we don't get any more fatal logs from them either: yes [16:41:42] Only misc. hosts are on Zend still [16:41:59] snapshot*, tin, terbium, probably tmh*, maybe some more [16:48:24] hoo: that's what I thought. And I haven't forgotten about your fatal.log woes. I have an idea but haven't had time to work on it [16:48:36] I should at least add notes on the phab task [16:51:20] <^d> All I see in fatal.log these days is crap from terbium [16:51:28] <^d> I guess snapshot* too? [16:51:56] Yeah, snapshot should as well [16:52:05] but I think logging from them was broken at some point [16:53:10] <^d> In the case of tin/terbium, do we really need hhvm? [16:53:42] No, just a newer zend version would be fine [16:58:33] Ok, logging from snapshot works :P [17:10:40] bd808, silver is trusty but runs jobs under php5 cli [17:11:03] Krenair: *nod* wikitech is a crazy edge case though [17:11:06] in a lot of ways [17:11:23] getting more normal all the time though [17:11:32] <^d> Getting on the train was a big deal [17:11:34] <^d> I'm so happy [17:11:38] I was moaning to Andrew about that being broken and sending errors to fluorine earlier [17:12:00] I was mostly asking because I found an old phab task to do the conversion and wanted to cehck before I closed it [17:12:30] (Scribunto relies on a PHP extension that is symlinked in for apache2 but not cli) [17:12:50] is that still not fixed? [17:13:16] 2015-02-13 17:03:02 silver labswiki: [5ed5deb4] [no req] Scribunto_LuaInterpreterNotFoundError from line 233 of /srv/mediawiki/php-1.25wmf16/extensions/Scribunto/engines/LuaSandbox/Engine.php: The luasandbox extension is not present, this engine cannot be used. [17:13:21] not as of approx. 10 minutes ago [17:13:34] * bd808 grumbles [17:13:43] needs root to fix... [18:00:47] <_joe_> bd808: yes [18:00:54] <_joe_> sorry, not feeling well today [18:03:06] _joe_: thanks for the answer. Hope you feel better and enjoy a weekend at some point [18:04:19] <_joe_> heh, me too :) [18:11:10] getting sick just in time for the long weekend (hate when that happens) [18:13:10] manybubbles: are you going to be in the team practices interview today? /me hopes so [18:13:18] oh yeah [18:13:34] reading resume now [18:13:42] excellent [18:13:53] I'm reading the homework assignment [18:20:30] <^d> bd808: I was just about to send the etherpad link again but saw you on it :) [18:20:40] :) [18:21:35] I think we should clear off the block from the last interview. Seem reasonble? [18:22:15] just to avoid head to head comparison (HR does not like that at all) [18:22:46] <^d> True. And we've already done the summaries from that too [18:23:27] I saved it just in case somebody wants it at some point. [18:31:55] <^d> bd808, manybubbles: He's already here meeting with kristen [18:32:20] ^d: neat. I'm in the weekly deployment meeting now [18:32:39] * ^d was supposed to get an invite to that he thought [18:32:42] * ^d shrugs [18:33:09] they are in the room right behind your desk [18:33:14] join them :) [18:33:34] <^d> I'm on 6 [18:33:45] crazy pants [18:44:08] <^d> legoktm: Just 3 left :) https://gerrit.wikimedia.org/r/#/q/owner:Chad+status:open+-puppet+-Cirrus,n,z [18:44:21] all of those are failing tests :/ [18:44:39] <^d> Not my fault! [18:50:58] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037229 (10Smalyshev) Is there a way to to traversals and aggregation in SPARQL? I.e. Cypher examples: This is list of all professions. Note the '*' there: ``` MATCH (v:item {wikibaseId: 'Q28640'... [18:54:07] 3MediaWiki-extensions-SecurePoll, MediaWiki-Core-Team: Fix SecurePoll_BallotStatus for Status/StatusValue changes - https://phabricator.wikimedia.org/T89475#1037244 (10Aklapper) Is T89425 a dup / related? [18:55:33] 3MediaWiki-extensions-SecurePoll, MediaWiki-Core-Team: Fix SecurePoll_BallotStatus for Status/StatusValue changes - https://phabricator.wikimedia.org/T89475#1037248 (10Anomie) Seems likely. [19:11:29] hoo: why not remove localnames as well? [19:11:52] legoktm: ehm... no diea [19:12:27] But if you do such manual deletions, be very careful [19:12:41] using a transaction on that table could easily lock it completely: Bad [19:13:31] I'd use the runbatchedquery script [19:16:59] Yeah, but that's not using a transaction [19:17:04] don't forget the where :D [19:23:01] <^demon|busy> csteipp: I thought you'd get a kick out of this one. [19:27:43] Do I wade into that or not... [19:28:16] <^demon|busy> Hehe, I just thought the OP was hilarious [19:56:46] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037466 (10Beebs.systap) >>! In T88717#1037229, @Smalyshev wrote: > Is there a way to to traversals and aggregation in SPARQL? I.e. Cypher examples: > > This is list of all professions. Note the... [20:02:05] ughhhhh https://meta.wikimedia.org/wiki/Special:CentralAuth/Ff02::3 isn't a valid username anymore [20:02:20] csteipp: ^ [20:03:57] legoktm: Yay IPv6 [20:05:53] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037477 (10Manybubbles) Thanks! We found the answer about an hour ago and didn't update the ticket in time. Thanks for the reference pages! [20:07:40] var_dump( IP::isIPAddress( 'Ff02::3' ) ); [20:07:40] bool(true) [20:07:51] So yeah... we should be detecting that [20:09:08] legoktm: So someone had the username, then we enabled ipv6, and it's no longer a valid username, so they can't globalize? [20:09:47] well, my email script can't email them :P [20:09:56] I doubt they'd be able to globalize though [20:10:54] legoktm: here? [20:11:06] yes? [20:11:22] I don't think they would be able to login either [20:11:23] Who are you sending emails to right now? [20:11:36] Because you seem to mail people you really shouldn't [20:11:39] causing confusion [20:12:37] hoo: the list of users is in /home/legoktm/spam8/ on terbium (already sent logs are in ~/spamlogs/)...who's getting emails they shouldn't? [20:13:00] Users with one edit in 2005 [20:13:19] why shouldn't we be emailing them? [20:13:37] Because it confuses them, apparently [20:13:48] Are you sending localized mails? [20:13:59] yes [20:14:05] That's weird [20:14:15] but still... people seem to find that confusing at points [20:14:24] and I'm not sure it's worth to email these for confirms [20:14:36] might be worth to email them like "Hey, we'll rename your account" [20:14:56] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037524 (10Beebs.systap) It's also worth checking out the RDF GAS API that Mike references. You can execute graph analytics within the SPARQL queries. It's bundled with BFS, SSP, Connected Compo... [20:15:10] that's in a few weeks :P [20:15:14] Where's the message you're sending? [20:15:22] Maybe the German translation is confusing [20:15:35] https://translatewiki.net/wiki/MediaWiki:Centralauth-finishglobaliseemail_body/en the links go to Special:MergeAccount [20:17:01] legoktm: I think it might be good to have someone in UX(?) take a look at our messaging for this. who has currently vetted this? [20:18:42] bd808: this may be something for you to get up to speed a little on [20:19:00] legoktm: hah! Found it [20:19:16] To [...] merge in any of your accounts that we could not do automatically [20:19:24] robla: SUL emails? Or??? [20:19:44] We got the text from Deskana this summer. several rounds of refinement [20:20:07] But people seem to assume from them that they will get other accounts [20:20:21] robla: I'm not sure, all of this stuff (the mass email script) was set up before I started working on the project. [20:20:28] So someone was writing to us: I'm not that guy on $otherwiki please let him have his account [20:20:29] s [20:20:54] and they had one edit in 2005 on a wikibooks [20:22:32] hoo, legoktm, robla: If we need to refine any of the wording or process it should go through Keegan [20:23:22] How many did we yet send? Still worth it? [20:23:31] If so, please amend that to be less confusing [20:23:51] we've sent 16k out of ~80k [20:24:04] oh, this is 80k messages total? [20:24:30] * robla somehow got the impression this was a much larger mailing [20:24:48] I thought this was about 2M accounts [20:25:33] csteipp: Can you maybe have a look at Capiunto today? It wont take you long, but we're waiting for probably 10 months now) [20:25:53] Less than 100 loc PHP, I think [20:26:02] no JS, mostly Lua [20:26:23] we removed accounts that had 0 edits and only included those where there were 2+ unattached accounts with the same unconfirmed email [20:26:49] hoo: Yep, it's next on my list! (Sorry, last set of sec bugs has been a little crazy..) [20:26:51] legoktm: Will there be more mailings with that text? [20:27:21] after this set of 80k? I don't think so. [20:27:34] Keegan can confirm though ^ [20:27:52] In that case it might be bearable to continue... might earn some OTRS spam and some tickets, though [20:28:09] OTRS spam? [20:28:13] is the reply-to set to otrs? [20:28:28] no, but people don't know who to message, I assume [20:28:33] Not sure's whos the reploy to [20:28:54] it should be the same as password reset emails? [20:29:03] password reset / email confirmation [20:30:23] From: wiki@wikimedia.org [20:30:36] That was a mail from 2010 though [20:30:56] Can't find any newer ones... although people request me a new password every once in a while [20:33:43] I don'see a reply-to header, just From: MediaWiki Mail [20:34:00] (and that was from 2013) [20:34:01] Yeah... so I guess people just search or something to reply [20:34:07] * for [20:35:57] hoo: there will not [20:40:11] so I guess 80k emails is not worth *too* much handwringing [20:40:32] should I start sending them out again? [20:40:51] Keegan: you up to speed on the backlog? [20:40:52] +1 [20:41:05] yes [20:41:08] actually if hoo is +1, I'm +1 as well [20:41:44] But for larger mailings, I guess it would be nice to be more careful here. Maybe also add some hint about who to contact. [20:42:06] Oh yes, of course [20:42:19] From what I understand, this email was intended to be very technical in nature [20:42:48] the "big" email will contain a brief explanation, link to meta, contact link [20:43:20] Keegan: +1 to what hoo just said. I think he's right that if we're sending out a mail to a significant chunk of long-inactive users, we should also make sure that our messaging doesn't sound like a bunch of inside baseball [20:43:56] I agree as well [20:44:45] ok, restarted [20:45:15] we also need to figure out what to do about User:'Ff02::3 ... [20:45:27] er, User:Ff02::3 [20:45:55] rename them? [20:47:22] I wonder how they weren't caught before we enabled ipv6 users [20:48:28] * Keegan sighs [20:52:18] 3SUL-Finalization, MediaWiki-Core-Team: Check for IPv6 usernames - https://phabricator.wikimedia.org/T89495#1037629 (10Legoktm) 3NEW [21:00:06] csteipp: https://github.com/facebook/hhvm/wiki/INI-Settings , several options match /(time)|(max)/, but no obvious candidates for max_execution_time alternatives [21:23:32] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037712 (10Jdouglas) Is there a standard way to load a huge amount of RDF data into Bigdata? I tried the following (with a 3GB GZ file), but it very quickly blew the heap: ``` Repository repo =... [21:29:05] legoktm: I just saw your new metrics [21:29:14] Why do we have *more* unattached local accounts now [21:29:28] I don't know :/ [21:29:46] That's a change by 500k [21:29:53] so definitely significant [21:30:38] Having more local accounts that clash kind of makes sense, if people (auto-)merge their accounts [21:30:49] I haven't had the chance to actually look at the data, I just ran the script and posted the output [21:33:08] 3SUL-Finalization, MediaWiki-Core-Team: Check for invalid usernames - https://phabricator.wikimedia.org/T89495#1037714 (10Legoktm) a:3Legoktm [21:34:25] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037721 (10Manybubbles) Looks like we're going to have trouble with some dates too. [[http://www.w3.org/TR/xmlschema-2/#dateTime-lexical-representation | xsd:dateTime]] supports [[https://www.wik... [21:44:47] 3MediaWiki-extensions-CentralAuth, MediaWiki-Core-Team: Undefined index warnings in ApiQueryGlobalUserInfo.php - https://phabricator.wikimedia.org/T89295#1037817 (10greg) a:3Legoktm [21:52:11] 3SUL-Finalization, MediaWiki-Core-Team: Check for invalid usernames - https://phabricator.wikimedia.org/T89495#1037828 (10Legoktm) Running a script across all wikis to look for invalid usernames... [22:07:54] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037879 (10Beebs.systap) >>! In T88717#1037712, @Jdouglas wrote: > Is there a standard way to load a huge amount of RDF data into Bigdata? I tried the following (with a 3GB gzipped .nt file), but... [22:10:10] 3SUL-Finalization, MediaWiki-Core-Team: Check for invalid usernames - https://phabricator.wikimedia.org/T89495#1037883 (10Nemo_bis) [22:14:32] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037902 (10Manybubbles) @Beebs.systap, is this still true (from [[http://blog.blazegraph.com/?p=281 | old blog post]]): For example, this can happen if your query has a large result set and uses O... [22:15:07] <^demon|busy> manybubbles: Have you had a look at my \t theory yet? [22:15:16] \t ? [22:15:29] <^demon|busy> https://gerrit.wikimedia.org/r/#/c/190223/ [22:16:09] sounds good to me [22:17:16] <^demon|busy> Search backend error during full_text search for '　' after 44. Parse error on '　': Encountered "" at line 1, column 1. [Called from CirrusSearch\ElasticsearchIntermediary::failure in /srv/mediawiki/php-1.25wmf16/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php at line 97] in /srv/mediawiki/php-1.25wmf16/includes/debug/MWDebug.php on line 300 [22:17:20] <^demon|busy> That's the failure we see ^ [22:24:04] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037914 (10Manybubbles) @Beebs.systap, I can't log in to your wiki with Google. It says "OpenID auth request contains an unregistered domain: http://wiki.blazegraph.com/wiki". I imagine that has... [22:25:13] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037916 (10Beebs.systap) >>! In T88717#1037914, @Manybubbles wrote: > @Beebs.systap, I can't log in to your wiki with Google. It says "OpenID auth request contains an unregistered domain: http://... [22:26:22] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037925 (10Beebs.systap) >>! In T88717#1037721, @Manybubbles wrote: > Looks like we're going to have trouble with some dates too. [[http://www.w3.org/TR/xmlschema-2/#dateTime-lexical-representati... [22:28:31] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037944 (10Jdouglas) @Beebs.systap thanks! It's working with: ``` LOAD ``` [22:34:17] 3MediaWiki-extensions-CentralAuth, MediaWiki-Core-Team: Undefined index warnings in ApiQueryGlobalUserInfo.php - https://phabricator.wikimedia.org/T89295#1037964 (10greg) p:5Triage>3High [22:38:38] hoo|away, csteipp: can you look at https://gerrit.wikimedia.org/r/190579 ? I'm not sure how that ever worked...and it means that people's emails might not be getting confirmed? [22:39:37] 3MediaWiki-extensions-SecurePoll, MediaWiki-Core-Team: Fix SecurePoll_BallotStatus for Status/StatusValue changes - https://phabricator.wikimedia.org/T89475#1037976 (10bd808) [22:41:04] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037978 (10Manybubbles) >>! In T88717#1037925, @Beebs.systap wrote: > > From Bryan Thompson. > > The native xsd:dateTime support is based on an int64 value. It does support negative int64 values... [22:41:58] This is why I never finish hoo's review... [22:42:19] sorry :/ [22:42:37] Just ironic that every time I start it, something comes up :) [22:43:29] someone came into #wikimedia saying they got a duplicate email after already confirming, and the script checks that your email is not confirmed before sending it to you...except when I looked in the db their email was never marked as confirmed [22:46:07] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1037986 (10Beebs.systap) >>! In T88717#1037902, @Manybubbles wrote: > @Beebs.systap, is this still true (from [[http://blog.blazegraph.com/?p=281 | old blog post]]): > For example, this can happen... [22:52:39] legoktm: Why would account that's attached to a CA account get an email anyway? [22:53:33] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1038020 (10Beebs.systap) Update from SYSTAP/Metaphacts. We are reloading the wikidata into the demo application shown to include provenance data. This should be ready in a day or so. [22:53:38] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1038021 (10Manybubbles) >>! In T88717#1037986, @Beebs.systap wrote: >>>! In T88717#1037902, @Manybubbles wrote: >> @Beebs.systap, is this still true (from [[http://blog.blazegraph.com/?p=281 | old... [22:53:54] legoktm: Looks fine, I'll see if hoo disagrees, otherwise I can +2 [22:54:40] csteipp: hmm, I don't actually see any place in the script that's checking for it... [22:54:58] ok, I'd like to get that deployed asap [22:55:34] What the CA patch I just got an eamil about? [22:55:45] hoo: https://gerrit.wikimedia.org/r/#/c/190579/1 [22:56:57] How did that get broken? [22:57:31] Or well... how did it ever work? [22:57:45] yeah, I couldn't figure that out either [22:58:02] That's what makes me wonder about that patch [22:58:13] it's just what the hell... :P [22:58:36] the UserInvalidateEmailComplete hook also calls setEmailAuthenticationTimestamp, but it calls saveSettings right afterwards [22:59:58] onUserSaveSettings [23:00:02] so that one saved it? [23:00:04] Just saw that [23:00:07] weird [23:00:23] relying on implementation details... [23:00:50] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1038030 (10Jdouglas) Speaking of dates, I see this from the server: ``` [java] ERROR: LexiconConfiguration.java:618: "1895-02-29" is not a valid representation of an XML Gregorian Calendar v... [23:01:24] Oh, yeah, so that's how it was working. [23:01:30] So it still should work, no? [23:01:46] I mean, it's nice to save closer to where we do the set... but no need to [23:02:00] MergeAccount calls: $user->confirmEmail();\n$user->saveSettings(); [23:02:30] Yeah, so does SpecialConfirmEmail [23:15:16] 3MediaWiki-Core-Team: Investigate DB connection spam from RL global modules - https://phabricator.wikimedia.org/T89507#1038063 (10aaron) 3NEW a:3aaron [23:16:13] 3GlobalCssJs, MediaWiki-Core-Team: Investigate DB connection spam from RL global modules - https://phabricator.wikimedia.org/T89507#1038075 (10Legoktm) [23:25:14] 3wikidata-query-service, MediaWiki-Core-Team: Investigate BigData for WDQ - https://phabricator.wikimedia.org/T88717#1038086 (10Manybubbles) Also, thanks for documenting your code so well. Its really a nice read. [23:51:14] AaronS: https://wikitech.wikimedia.org/wiki/Generating_CAPTCHAs Given you've written that: What does it take (you?) to re-generate them?