[04:00:16] 6MediaWiki-Core-Team, 10SUL-Finalization, 10Wikimedia-General-or-Unknown: Invalid usernames on Wikimedia web sites - https://phabricator.wikimedia.org/T5507#1058118 (10Keegan) >>! In T5507#1056396, @TTO wrote: >>>! In T5507#1047903, @Keegan wrote: >> There are 218 local badusername accounts > > F2314 (link... [04:02:27] TimStarling: six segfaults in the last 14 hrs, all in HPHP::GlobStreamWrapper::opendir. Does that ring a bell? [04:55:54] 6MediaWiki-Core-Team, 10SUL-Finalization, 10Wikimedia-General-or-Unknown: Invalid usernames on Wikimedia web sites - https://phabricator.wikimedia.org/T5507#1058150 (10TTO) >>! In T5507#1058118, @Keegan wrote: >>>! In T5507#1056396, @TTO wrote: >> Do any of these invalid users have significant contributions?... [05:03:45] ori: no [08:27:18] bd808|BUFFER: what's this about a weekend in france? I'm already in france! [08:27:20] well, for now [08:27:25] (catching up on irc after a long weekend) [08:44:55] 6MediaWiki-Core-Team, 10Datasets-General-or-Unknown, 6Services, 7Epic: Improve Wikimedia dumping infrastructure - https://phabricator.wikimedia.org/T88728#1058337 (10ezachte) Some comments on process. Before we start to actually work on this Big Hairy Project aka Grand Redesign. This initiative could wel... [10:19:02] 6MediaWiki-Core-Team, 10Datasets-General-or-Unknown, 6Services, 7Epic: Improve Wikimedia dumping infrastructure - https://phabricator.wikimedia.org/T88728#1058461 (10Nemo_bis) I agree with the "keep the lights on" sentiment and I feel responsible for a possibly-excessive enlargement of the wiki page's scop... [12:55:40] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Zookeeper - https://phabricator.wikimedia.org/T90109#1058712 (10Joe) First of all, sorry If I did not get back to you earlier. I don't like the idea of having a complex tool like zookeeper running just to ensure HA. This is actually pret... [13:29:00] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Zookeeper - https://phabricator.wikimedia.org/T90109#1058742 (10Beebs.systap) >>! In T90109#1058712, @Joe wrote: > First of all, sorry If I did not get back to you earlier. > > I don't like the idea of having a complex tool like zookeepe... [14:54:51] 6MediaWiki-Core-Team, 6Multimedia, 7database: Pages with invalid titles (ns=0, namespace in page_title) created around Jan 27 on commons - https://phabricator.wikimedia.org/T90442#1058882 (10Bawolff) 3NEW [14:57:08] 6MediaWiki-Core-Team, 6Multimedia, 7database: Pages with invalid titles (ns=0, namespace in page_title) created around Jan 27 on commons - https://phabricator.wikimedia.org/T90442#1058904 (10Steinsplitter) [14:58:33] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Geo - https://phabricator.wikimedia.org/T90130#1058913 (10Beebs.systap) [14:58:57] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Value representation - https://phabricator.wikimedia.org/T90123#1058915 (10Beebs.systap) [14:59:41] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Scale out plans - https://phabricator.wikimedia.org/T90117#1058918 (10Beebs.systap) [15:00:06] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1058920 (10Beebs.systap) [15:00:52] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Prefix vs Suffix - https://phabricator.wikimedia.org/T90121#1058923 (10Beebs.systap) [15:01:13] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Pluggable inline values - https://phabricator.wikimedia.org/T90131#1058925 (10Beebs.systap) [15:05:52] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Pluggable inline values - https://phabricator.wikimedia.org/T90131#1058938 (10Thompsonbry.systap) Inline values are not necessary. They represent a tradeoff between dictionary encoding values and their direct representation as inline val... [15:08:11] 6MediaWiki-Core-Team, 6Multimedia, 7database: Pages with invalid titles (ns=0, namespace in page_title) created around Jan 27 on commons - https://phabricator.wikimedia.org/T90442#1058954 (10Anomie) [15:08:12] 6MediaWiki-Core-Team, 10Wikimedia-General-or-Unknown, 5Patch-For-Review: Existed pages without ability to reach and obviously wrong namespace - https://phabricator.wikimedia.org/T87645#1058955 (10Anomie) [15:14:45] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Scale out plans - https://phabricator.wikimedia.org/T90117#1058973 (10Thompsonbry.systap) This depends on how you model the reified RDF data. However, the inlined statements about statements are not in the same part of the statement indi... [15:16:46] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Pluggable inline values - https://phabricator.wikimedia.org/T90131#1058977 (10Manybubbles) >>! In T90131#1058938, @Thompsonbry.systap wrote: > Inline values are not necessary. They represent a tradeoff between dictionary encoding values... [15:17:16] 6MediaWiki-Core-Team, 10Security-Reviews, 10wikidata-query-service: BlazeGraph Finalization: Security Review - https://phabricator.wikimedia.org/T90115#1058978 (10Manybubbles) We should also look into Apache River from a security perspective. [15:17:33] 6MediaWiki-Core-Team, 10wikidata-query-service: Investigate BlazeGraph aka BigData for WDQ - https://phabricator.wikimedia.org/T88717#1058980 (10Thompsonbry.systap) I've received the signed CLA from corporate. Once I get Nik's SF account I will set him up as a developer. [15:17:49] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Zookeeper - https://phabricator.wikimedia.org/T90109#1058981 (10Joe) @Beebs.systap to be more explicit, it's highly probable we won't use ZK as our distributed, consistent KV store of choice internally, so maintaining a separated ZK clust... [15:20:40] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Pluggable inline values - https://phabricator.wikimedia.org/T90131#1058989 (10Thompsonbry.systap) I agree that this is related to how you choose to represent, index, and query values with additional annotations (error bounds, uncertainty,... [15:21:20] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Plan for no downtime upgrade - https://phabricator.wikimedia.org/T90445#1058994 (10Manybubbles) 3NEW [15:26:28] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Zookeeper - https://phabricator.wikimedia.org/T90109#1059011 (10Beebs.systap) >>! In T90109#1058981, @Joe wrote: > @Beebs.systap to be more explicit, it's highly probable we won't use ZK as our distributed, consistent KV store of choice i... [15:33:26] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Geo - https://phabricator.wikimedia.org/T90130#1059021 (10Thompsonbry.systap) We would be interested in both the elastic search and GeoSPARQL integrations. Full GeoSPARQL support is complex. It involves not only the spatial index, but... [15:42:11] swtaarrs: Wikimedia EU hackathon 2015-05-22 to 2015-05-24 in Lyon France. [15:42:37] swtaarrs: WMF staff are supposed to find "buddies" from outside the Foundation to pair with on demoable projects ideally [15:42:46] ooh neat [15:43:19] heh [15:43:29] That's about a week after I get back from another trip to France [15:43:36] (back to CA) [15:43:52] So many frequent flyer miles! [15:47:33] heh, yep :) [15:47:44] all I have to show so far is a free second checked bag [15:51:01] miles are a racket. I was trying to figure out how to use my United miles to get to Fiji. "all" I have to do is take a series of flights from LAX to HON to AUK to NAN that takes 30+ hours each way and they will let me spend miles on the LAX to HON leg. :( [15:51:36] * bd808 is taking a direct LAX to NAN flight instead [16:08:52] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Update performance - https://phabricator.wikimedia.org/T90114#1059076 (10Thompsonbry.systap) The data storage on the disk is identical for the single machine and HA replication cluster (HAJournalServer) modes. In fact, you can take a com... [17:18:10] manybubbles: why is it that elastic search + java take so much resources on my macbook? [17:18:19] i don't think this has always been the case... [17:18:27] * aude gonna kill it again [17:18:27] <^d> cpu, ram? [17:18:31] yeah [17:18:35] 6MediaWiki-Core-Team, 10Deployment-Systems, 10Librarization, 5Patch-For-Review: Have a check for reported security issues in dependencies - https://phabricator.wikimedia.org/T74193#1059370 (10Aklapper) [17:18:35] especially cpu [17:18:38] aude: dunno - can you post me the command line? [17:18:52] i am using 1.3.0 (should upgrade) [17:18:52] aude: if memory fills up then it takes up tons of cpu gcing - horrible way to die [17:19:07] <^d> that ^ [17:19:07] hm [17:19:10] <^d> gc is expensive [17:19:28] <^d> If you've got the ram, a larger heap would help [17:19:37] bin/elasticsearch -d start [17:19:41] probably can/do [17:20:02] would like to keep elastic enable for development etc [17:20:06] 6MediaWiki-Core-Team, 10Datasets-General-or-Unknown, 6Services, 7Epic: Improve Wikimedia dumping infrastructure - https://phabricator.wikimedia.org/T88728#1059380 (10ArielGlenn) I will cetainly be maintaining/fixingup the current dumps while any investigation/scheming goes on towards this project. My imme... [17:23:35] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Geo - https://phabricator.wikimedia.org/T90130#1059396 (10Smalyshev) @Thompsonbry.systap I don't think we need full GeoSPARQL at least at the beginning stage, but support for something like distance filtering would already cover the most... [17:27:49] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Pluggable inline values - https://phabricator.wikimedia.org/T90131#1059403 (10Smalyshev) We may need custom dateTime to represent that dreaded https://www.wikidata.org/wiki/Q1#P580 [17:35:33] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Scale out plans - https://phabricator.wikimedia.org/T90117#1059439 (10Smalyshev) @Thompsonbry.systap currently our reification model is pretty close to one described here: http://korrekt.org/papers/Wikidata-RDF-export-2014.pdf except that... [17:37:03] ^d: manybubbles think i made it better, at least for now [17:37:23] aude: cool [17:37:27] thanks :) [17:43:15] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Geo - https://phabricator.wikimedia.org/T90130#1059471 (10Thompsonbry.systap) It is very easy to write and register custom functions for things like distance filtering. See [1]. This could be combined with the MGRS approach and prefix sc... [17:44:53] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Scale out plans - https://phabricator.wikimedia.org/T90117#1059473 (10Thompsonbry.systap) BlazeGraph supports arbitrary nesting of statements on statements, so, yes, that would be fine. [17:57:46] aude: With the latest greatest mediawiki-vagrant you can set `vagrant config vagrant_ram auto` to give your VM 1/4 of your total system ram. On my MBP that makes many things better for working in the VM. [17:58:25] `vagrant config vagrant_cores auto` is also nice. That shares all of your cores with the VM rather than the default of 2. [17:58:27] bd808: cool [17:58:35] although i'm mostly not using vagrant (but should) [17:58:48] * bd808 stares at aude [17:58:51] :) [17:59:06] this is why the wikidata role doesn't work as well as it should ;) [17:59:13] true true [18:00:11] not at all high priority but if you would look at https://gerrit.wikimedia.org/r/#/c/192320/, would be nice :) [18:00:27] any idea what i'm doing wrong or if it is sane [18:26:11] legoktm: i don't 100% understand your comment on https://gerrit.wikimedia.org/r/#/c/192320/ [18:26:31] should we include ext-* dependencies in composer.json (we do for suggest) [18:26:45] and the checker skip them? [18:27:05] seems i have to run composer install before i run install.php [18:27:18] aude: right now in update.php we have a script that compares what's required in composer.json and what is installed in composer.lock. I'm saying that that script should ignore "ext-*" and "php" and let the installer make sure required extensions are installed [18:27:36] hm [18:27:52] AFAIS ext-* isn't even in composer.lock [18:28:14] that's the issue i think [18:28:35] * aude doesn't want to spend that much time on this [18:30:00] 6MediaWiki-Core-Team, 10wikidata-query-service: Investigate BlazeGraph aka BigData for WDQ - https://phabricator.wikimedia.org/T88717#1059687 (10Manybubbles) 5Open>3Resolved >>! In T88717#1058980, @Thompsonbry.systap wrote: > I've received the signed CLA from corporate. Once I get Nik's SF account I will... [18:30:57] * legoktm works on a quick patch [18:36:29] aude: https://gerrit.wikimedia.org/r/192366 [18:39:13] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Geo - https://phabricator.wikimedia.org/T90130#1059716 (10Manybubbles) 5Open>3Resolved a:3Manybubbles I'm resolving this issue. The resolution is a the following conclusions: 1. There is no builtin support for geo things. 2. Build... [18:39:14] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Verify features work - https://phabricator.wikimedia.org/T90124#1059719 (10Manybubbles) [18:39:31] legoktm: looks good to me [18:41:59] +2? ;) [18:43:35] there [18:44:24] * anomie writes lots of words [18:50:57] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Range Queries - https://phabricator.wikimedia.org/T90129#1059780 (10Manybubbles) >>! In T90129#1055704, @Jdouglas wrote: > Let's find everyone born within a month of that date: > > ``` > prefix wdq: > pr... [18:51:47] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Investigate inference rules and truth maintenance - https://phabricator.wikimedia.org/T89849#1059786 (10Manybubbles) I believe this is basically validated given your comments on T90129. Correct? Can close? [18:52:27] legoktm: bah. that composer.lock check script is annoying. [18:52:56] I think having it is better than not having it :P [18:53:02] It uses the md5/sha1 of the composer.json so any change at all makes it mad [18:53:46] like changing a text description or adding the thing that aude is in 192320 that has nothing to do with the lock contents [18:53:47] no, it compares the actual dependencies [18:53:53] really? [18:54:12] https://github.com/wikimedia/mediawiki/blob/master/maintenance/checkComposerLockUpToDate.php#L53 [18:55:03] So it is failing based on the "ext-iconv: not installed, * required." bit then [18:55:24] seems to be, which confuses me [18:55:29] yeah, because ext-* stuff isn't in the lock file [18:55:37] or http://stackoverflow.com/questions/25122037/composer-extension-iconv-is-missing [18:55:42] saw that [18:56:05] iconv is definitely there and one shouldn't have to touch any php.ini for this [18:56:37] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Range Queries - https://phabricator.wikimedia.org/T90129#1059803 (10Jdouglas) > Once you have a more bigger data set can you post the plan both with and without the runtime query optimizer option? Sure thing. > Why? Maybe not, if we're... [18:58:31] jenkins approves now [18:58:47] * aude off for food.... [19:00:03] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Investigate inference rules and truth maintenance - https://phabricator.wikimedia.org/T89849#1059825 (10Manybubbles) I see what you mean now about vocabularies. Oooooh - what are axioms? [19:02:48] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Investigate inference rules and truth maintenance - https://phabricator.wikimedia.org/T89849#1059833 (10Manybubbles) Ahhh - looks like Axioms are just extra triples claimed on knowledge base initialization. [19:07:16] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Range Queries - https://phabricator.wikimedia.org/T90129#1059860 (10Manybubbles) >>! In T90129#1059803, @Jdouglas wrote: > Maybe not, if we're able to capture it all in a SPARQL query as above. I was thinking about how to support a singl... [19:08:32] <^d> bd808: Heh. https://gerrit.wikimedia.org/r/#/c/192371/ [19:10:43] ^d: not too surprising really but dramatic [19:11:35] That read test is what cdb is for [19:11:44] legoktm: https://gerrit.wikimedia.org/r/#/c/192268/ [19:12:14] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Validate AST rewrite - https://phabricator.wikimedia.org/T90128#1059876 (10Manybubbles) a:3Manybubbles [19:33:31] bd808: We're joining... just a little slow [19:33:41] *nod* [19:37:05] TimStarling: AaronSchulz: So regarding the sqlite issues, I feel I'm in over my head. I need mediawiki-core sqlite to work properly. [19:38:07] It's escalating in the jobs being affected. Necessitating rechecks quite often. Presumably some async thing going on elsewhere. We need to clear the house and get the foundation in order before I can move on. Basically everything is blocked on it. [19:40:14] The only alternative option I consider feasible for me to implement, is to reduce global concurrency of apache host on slaves to 1. That'll reduce concurrency in jobs but makes it reliable. It'll reduce it too much (1 request per slave instead of per mw install), but I'm not sure what other short-term options I can do at the moment. [19:40:22] 6MediaWiki-Core-Team, 10SUL-Finalization, 10Wikimedia-Site-requests: Run sendConfirmAndMigrateEmail.php for all unconfirmed emails on all wikis - https://phabricator.wikimedia.org/T73241#1059974 (10tomasz) [19:53:13] 6MediaWiki-Core-Team, 10SUL-Finalization, 10Wikimedia-General-or-Unknown: Invalid usernames on Wikimedia web sites - https://phabricator.wikimedia.org/T5507#1060052 (10tomasz) [19:56:39] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Geo - https://phabricator.wikimedia.org/T90130#1060075 (10Smalyshev) That has impact on export format. Right now we export as `"Point(12.56 34.78)"^^geo:wktLiteral`. Please tell if you think this would be incompatible with whatever we may... [19:59:46] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Geo - https://phabricator.wikimedia.org/T90130#1060110 (10Thompsonbry.systap) I have no reason to believe that this would be a problem. The code to translate that into MGRS information could be pushed down into the server when the data a... [20:18:08] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Range Queries - https://phabricator.wikimedia.org/T90129#1060189 (10Jdouglas) > Once you have a more bigger data set can you post the plan both with and without the runtime query optimizer option? It doesn't seem to make much difference.... [20:21:55] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Investigate inference rules and truth maintenance - https://phabricator.wikimedia.org/T89849#1060193 (10Jdouglas) >>! In T89849#1059833, @Manybubbles wrote: > Ahhh - looks like Axioms are just extra triples claimed on knowledge base initi... [20:22:15] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Investigate inference rules and truth maintenance - https://phabricator.wikimedia.org/T89849#1060194 (10Jdouglas) >>! In T89849#1059786, @Manybubbles wrote: > I believe this is basically validated given your comments on T90129. Correct?... [20:22:27] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Verify features work - https://phabricator.wikimedia.org/T90124#1060196 (10Jdouglas) [20:22:28] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Investigate inference rules and truth maintenance - https://phabricator.wikimedia.org/T89849#1060195 (10Jdouglas) 5Open>3Resolved [20:58:47] legoktm: https://gerrit.wikimedia.org/r/#/c/192006/ [21:11:44] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: What about non-wikidata ontologies - https://phabricator.wikimedia.org/T90122#1060348 (10Jdouglas) What would be the advantages of doing this? It seems generally "good" to create a Rosetta Stone between W3C standards and [[ https://www.w... [21:14:03] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Range Queries - https://phabricator.wikimedia.org/T90129#1060360 (10Manybubbles) >>! In T90129#1060189, @Jdouglas wrote: >>>! In T90129#1059780, @Manybubbles wrote: >>>>! In T90129#1055704, @Jdouglas wrote: >>> Let's find everyone born wi... [21:18:14] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Range Queries - https://phabricator.wikimedia.org/T90129#1060371 (10Jdouglas) Explain output (without query optimizer): ``` Query SPARQL prefix wdq: prefix wdo: prefi... [21:18:55] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: What about non-wikidata ontologies - https://phabricator.wikimedia.org/T90122#1060386 (10Smalyshev) I think we should take this slow and maybe even look at a way to defer this to the community (e.g. by designating specific claims on prope... [21:19:16] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Investigate inference rules and truth maintenance - https://phabricator.wikimedia.org/T89849#1060389 (10Manybubbles) >>! In T89849#1060193, @Jdouglas wrote: >>>! In T89849#1059833, @Manybubbles wrote: >> Ahhh - looks like Axioms are just... [21:21:56] <^demon|lunch> anomie: We're getting OOMs from the api now [21:22:04] <^demon|lunch> 223 error: request has exceeded memory limit in /srv/mediawiki/php-1.25wmf17/includes/api/ApiModuleManager.php on line 145 [21:22:04] <^demon|lunch> 155 error: request has exceeded memory limit in /srv/mediawiki/php-1.25wmf17/includes/api/ApiBase.php on line 975 [21:22:38] ^demon|lunch: Starting when? [21:23:23] bd808: https://gerrit.wikimedia.org/r/192083 Can you +1 if you're ok with re-enabling? [21:23:23] <^demon|lunch> At least Feb 23 08:41:08 mw1129: [21:23:26] I really need these logs [21:23:28] 1.25wmf17 ApiModuleManager.php on line 145 is the closing-brace of a function... [21:23:35] At least for now [21:24:24] <^demon|lunch> 0202? [21:24:32] <^demon|lunch> anomie: Yeah, line numbers are useless here. [21:24:34] <^demon|lunch> archive/hhvm.log-20150202.gz:Feb 1 15:11:44 mw1240: #012Fatal error: request has exceeded memory limit in /srv/mediawiki/php-1.25wmf14 [21:24:50] <^demon|lunch> earliest I see [21:24:59] hoo: {{done}} [21:25:37] bd808: Thanks. Could you maybe post to ops or wikitech or so before disabling logs again? [21:25:46] I know we ahve a lot of logs which people probably don't use [21:25:47] ^demon|lunch: Do we have any idea what the problem requests were? [21:25:57] <^demon|lunch> Not yet, but we might be able to do some matching [21:25:57] but there are some logs which eg. I need from time to time [21:26:20] hoo: Or we could use logs better. Needing a record of the start time of every job ever run seems a bit extreme [21:26:24] And it's a little disruptive to just see them gone [21:26:43] True that... for my use case right now Wikidata only should be enough [21:26:56] But when I have to debug client, it might not be enough [21:27:00] ^demon|lunch: Doesn't seem to be anything new: Sep 12 20:51:38 mw1053: #012Fatal error: request has exceeded memory limit in /srv/mediawiki/php-1.24wmf20/includes/api/ApiModuleManager.php on line 145 [21:27:21] <^demon|lunch> ugh [21:27:28] <^demon|lunch> ok, just seemed to be bubbling up in fatalmonitor [21:28:38] <^demon|lunch> I can't find any of the mw* servers from the hhvm.log in the api.log [21:28:44] <^demon|lunch> Maybe it's a non-API request? [21:30:08] Could be... What pool are those servers in? [21:31:18] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: What about non-wikidata ontologies - https://phabricator.wikimedia.org/T90122#1060475 (10Jdouglas) +1 [21:33:23] <^demon|lunch> Got it, I think [21:33:56] <^demon|lunch> 2015-02-23 10:23:13 mw1123 enwiktionary: API POST T=71ms action=expandtemplates format=json title=%EC%82%AC%EB%82%B4 text=%7B%7Bko-etym-Sino%7C%E7%A4%BE%7C%7C%E5%85%A7%7Cinterior%7D%7D revid=28203279 prop=wikitext%7Ccategories%7Cproperties [21:34:01] <^demon|lunch> Feb 23 10:23:37 mw1123: #012Fatal error: request has exceeded memory limit in /srv/mediawiki/php-1.25wmf17/includes/api/ApiModuleManager.php on line 145 [21:34:45] <^demon|lunch> Or [21:34:54] <^demon|lunch> 2015-02-23 10:23:19 mw1123 frwiki: API GET T=17ms action=opensearch format=json search=R%C3%A9sistante%20fran%C3%A7a limit=10 namespace=0 suggest= [21:35:10] <^demon|lunch> Or [21:35:16] <^demon|lunch> 2015-02-23 10:23:31 mw1123 enwiki: API GET T=31ms action=query format=json titles=File:Flag_of_Virginia.svg prop=imageinfo iiprop=mediatype%7Csize%7Curl iiurlwidth=25 [21:36:12] <^demon|lunch> Those are the only mw1123 api.log around 10:23:37 [21:36:40] <^demon|lunch> Or 2015-02-23 10:23:34 mw1123 enwiki: API GET T=77ms action=query format=json generator=prefixsearch redirects= prop=pageimages list=prefixsearch gpssearch=lord%20and%20lady%20baden%20po gpsnamespace=0 gpslimit=15 piprop=thumbnail pithumbsize=80 pilimit=15 pssearch=lord%20and%20lady%20baden%20po pslimit=15 [21:39:33] ori: Is your -2 on https://gerrit.wikimedia.org/r/#/c/191259/ going to stand forever? Do I have to finish rewriting all usage of the logging layer to move forward again? Inquiring minds want to know. [21:44:59] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Verify features work - https://phabricator.wikimedia.org/T90124#1060536 (10Manybubbles) [21:45:00] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Range Queries - https://phabricator.wikimedia.org/T90129#1060534 (10Manybubbles) 5Open>3Resolved OK - I did some more code review and have a decent understanding of some of the path from BTree up. I haven't fully traced it up to spar... [21:45:45] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1060542 (10Manybubbles) [21:45:45] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: What about non-wikidata ontologies - https://phabricator.wikimedia.org/T90122#1060539 (10Manybubbles) 5Open>3Resolved a:3Manybubbles Resolving as "not right now". [21:46:41] ^demon|lunch: Looking at Feb logs, I see 596 lines coming from hosts not listed in https://config-master.wikimedia.org/pybal/eqiad/api, and only 36 from hosts in that file. Not sure if that helps. [21:46:52] anomie: question on https://gerrit.wikimedia.org/r/183596 [21:47:10] <^demon|lunch> anomie: Doubt it, but thanks :) [21:51:47] legoktm: Good catch [21:56:11] bd808: I think you should rewrite $wgDebugLogGroups. I get suspicious when I see a shift to the passive voice when talking about plans ("I expect the complexity of the Monolog config to be reduced dramatically as we full migrate MediaWiki") because it means there's no concrete plan to do it [21:56:38] but I can reply on the patch or chat with you about it [21:57:07] *shrug* it's not going to happen any time soon. And today it would break the fuck out of everything [21:57:27] why? make the monolog config canonical and generate $wgDebugLogGroups from that [21:57:27] since we are currently relying on $wgDebugLogGroups for all the wikis [21:59:03] that's better? [21:59:33] going to the meeting room, bbiab [23:10:08] bd808: our messages are in WikimediaMessages [23:10:33] would the right think be to backport our change to the deployment branch [23:10:39] and updates would come with scap [23:11:00] https://gerrit.wikimedia.org/r/#/c/192462/ [23:11:49] The changes from master should be applied even without backporting. The hangup right now is that the updates are only going out via the manual runs of scap instead of during the nightly update process. [23:12:16] hmmm, ok [23:12:45] somehow i think i did this wrong last time and we were not geting updated translations [23:12:48] even after scap [23:12:51] So those messages should be mixed into the l10n cache files on the next run and then just need a scap to get them from tin to the mw servers. [23:13:00] ok [23:13:35] * aude should have made this patch a week ago... [23:13:44] but forgot and was on holiday [23:13:47] One trick is that the master messages only end up in the branches when l10nupdate does it's magic [23:13:56] ok [23:14:01] they don't get mixed in by a normal scap [23:14:19] right [23:14:30] it's just that l10nupdate isn't syncing them [23:14:33] but the rebuild that scap does won't erase them once they are in the l10n cache files on tin [23:14:42] ok [23:14:45] that's part of what is so expensive about the l10n cache builds [23:14:57] we check the date/time of each and every message [23:14:57] * aude nods [23:15:35] we can manually run the steps that are failing for l10nudpate too if needed [23:15:41] ok [23:16:11] thanks :) [23:16:13] when do you need the new messages live? [23:17:18] tomorrow, probably after the train [23:17:38] as translations come in, so we do scap and then there is scap again on wednesday [23:17:51] *nod* [23:18:09] not that critical but people will wonder why their translations don't ever appear [23:18:37] a day or so is okish to wait [23:19:17] it would be possible to selectively sync the files and use dsh to rebuild the caches, but just running a full scap is probably the easiest and safest thing to do manually [23:19:48] The part that is failing in the nightly l10nupdate job is the sync-dir [23:21:30] ok [23:59:21] 6MediaWiki-Core-Team, 10GlobalCssJs, 5Patch-For-Review: Investigate DB connection spam from RL global modules - https://phabricator.wikimedia.org/T89507#1060990 (10bd808) 5Open>3Resolved [23:59:41] 6MediaWiki-Core-Team, 5Patch-For-Review: Try to avoid doCascadeProtectionUpdates() call on page view - https://phabricator.wikimedia.org/T89389#1060992 (10bd808) 5Open>3Resolved [23:59:42] 6MediaWiki-Core-Team, 7Epic, 5Patch-For-Review: MediaWiki multi-datacenter investigation and work - https://phabricator.wikimedia.org/T88445#1060994 (10bd808) [23:59:55] 6MediaWiki-Core-Team, 10MediaWiki-extensions-AbuseFilter, 5Patch-For-Review: Slow AFComputedVariable users query - https://phabricator.wikimedia.org/T90036#1060995 (10bd808) 5Open>3Resolved