[00:09:31] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Machine Sizing/Shaping - https://phabricator.wikimedia.org/T90116#1064445 (10Smalyshev) Running it with 16g of RAM seems to behave fine, though it takes 500% CPU (on 16 CPU machine) even when running single query. Simpler queries behave w... [00:24:41] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Machine Sizing/Shaping - https://phabricator.wikimedia.org/T90116#1064477 (10Beebs.systap) [00:38:16] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Operational issues - https://phabricator.wikimedia.org/T90103#1064523 (10Jdouglas) `labs.wdq_cluster.lock.release()` After much gnashing of teeth, I managed to get a three-server HA journal replication cluster going. Instructions for r... [01:05:32] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Machine Sizing/Shaping - https://phabricator.wikimedia.org/T90116#1064564 (10Thompsonbry.systap) There are two known issues with the property path operator that Nik and I discussed today. These are [1] and [2]. The first of these issues... [01:12:49] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Machine Sizing/Shaping - https://phabricator.wikimedia.org/T90116#1064574 (10Thompsonbry.systap) By label lookup I assume that you mean materializing (projecting out through a SELECT expression) the actual RDF Values for URIs or Literals... [01:17:29] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Machine Sizing/Shaping - https://phabricator.wikimedia.org/T90116#1064577 (10Thompsonbry.systap) Concerning thread. See [1] for a general background on query optimization for blazegraph. The QueryEngine class in blazegraph supports pa... [04:23:19] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Machine Sizing/Shaping - https://phabricator.wikimedia.org/T90116#1064763 (10Smalyshev) @Thompsonbry.systap I was thinking about maybe separating the statement data and the label/description data in different bigdata instances, since for... [04:54:58] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Machine Sizing/Shaping - https://phabricator.wikimedia.org/T90116#1064768 (10Smalyshev) > Out of all the property hierarchies, only the geo-political one involves transitive traversal up more than a single predicate We probably have more... [05:08:58] 6MediaWiki-Core-Team, 10MediaWiki-extensions-OAuth, 7Epic: Support a nice sso experience with MediaWiki's OAuth - https://phabricator.wikimedia.org/T86869#1064797 (10Eloquence) Just experimenting a bit with a general "roadmap" type workboard, feel free to move things around. Erik [13:52:14] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1065608 (10Manybubbles) One of the things that @Thompsonbry.systap and I talked about yesterday was representing statements kind of like this: ``` wd:Q23 wdt:P39 wd:Q11696 . <------ Op... [13:58:32] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1065623 (10Thompsonbry.systap) The RDR inlining of reified statement models is handled by the StatementBuffer class. It is important to have a limited lexical scope in the dump for t... [13:59:02] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: Scale out plans - https://phabricator.wikimedia.org/T90117#1052885 (10Manybubbles) This isn't really a scale out thing but i'm putting it here any way: One option for installation is keep all the BlazeGraph servers independently up to da... [14:05:54] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1065635 (10Manybubbles) >>! In T90119#1065623, @Thompsonbry.systap wrote: > The RDR inlining of reified statement models is handled by the StatementBuffer class. It is important to h... [14:06:43] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1065639 (10Manybubbles) >>! In T90119#1065635, @Manybubbles wrote: >>>! In T90119#1065623, @Thompsonbry.systap wrote: >> The RDR inlining of reified statement models is handled by the... [14:10:16] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1065645 (10Thompsonbry.systap) You can use URIs instead of blank nodes. Most of the time when people use blank nodes they SHOULD be using URIs. Blank nodes are existential variables.... [14:22:46] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1065667 (10Manybubbles) >>! In T90119#1065645, @Thompsonbry.systap wrote: > You can use URIs instead of blank nodes. Most of the time when people use > blank nodes they SHOULD be usin... [14:31:04] 6MediaWiki-Core-Team, 10wikidata-query-service: BlazeGraph Finalization: RDF Issues - https://phabricator.wikimedia.org/T90119#1065676 (10Thompsonbry.systap) There is support for inline UUIDs for blank nodes. See UUIDBNodeIV. You could also define a fully inline URI with a well-known prefix and a UUID. Bryan... [15:59:57] hmmm... newest version of znc deprecated the plugin that changes nick on detach. Not sure I love that [16:07:04] Why not use /away [16:07:27] That's their argument too. :) [16:07:49] and they have an auto_away plugin to set an away reason on detach [16:07:57] so I'll try that for a while [16:09:00] the source for the old plugin is available but they are not shipping it any more and strongly discouraging its use [16:19:40] bd808: nick changes for afk status are uncooth in some hardcore circles [16:20:15] yeah yeah [16:20:51] :) [16:21:19] I was in a channel that would kick you if you did it [16:21:36] * bd808 looks at |Away, |detached, |zzz, _zzz, _afk suffixes in the channels he's in [16:21:52] yeah, we're obviously not followers of that doctrine :) [16:30:27] 6MediaWiki-Core-Team, 10CirrusSearch, 10Wikimedia-Blog: Finish up blog post for Cirrus - https://phabricator.wikimedia.org/T85176#1066086 (10Manybubbles) 5declined>3Open [16:30:51] 6MediaWiki-Core-Team, 10CirrusSearch, 10Wikimedia-Blog: Finish up blog post for Cirrus - https://phabricator.wikimedia.org/T85176#940725 (10Manybubbles) Reopening as per conversation with @robla. [17:35:13] 6MediaWiki-Core-Team, 10MediaWiki-Vagrant: Fix "Error: Exec[add page devwiki/Main_Page]: mwscript edit.php returned 1 instead of one of [0]" - https://phabricator.wikimedia.org/T90584#1062531 (10bd808) [17:35:48] 6MediaWiki-Core-Team, 10Incident-20150205-SiteOutage, 10Wikimedia-Logstash, 6operations, 5Patch-For-Review: Decouple logging infrastructure failures from MediaWiki logging - https://phabricator.wikimedia.org/T88732#1066220 (10bd808) [17:35:49] 6MediaWiki-Core-Team, 10Incident-20150205-SiteOutage, 10Wikimedia-Logstash, 6operations, 5Patch-For-Review: Prototype Monolog and rsyslog configuration to ship log events from MediaWiki to Logstash - https://phabricator.wikimedia.org/T88870#1066219 (10bd808) 5Open>3Resolved [17:37:14] 6MediaWiki-Core-Team, 10MediaWiki-extensions-SecurePoll: Set up mini wikifarm in Labs which has SecurePoll on it - https://phabricator.wikimedia.org/T88725#1066222 (10bd808) 5Open>3stalled [17:40:45] Krinkle: I left a comment on https://gerrit.wikimedia.org/r/#/c/192821/ [17:40:51] I think that's the last bit [17:40:55] 6MediaWiki-Core-Team, 10Wikimedia-Wikimania-Scholarships: Operational support for Wikimania Scholarships 2015 cycle - https://phabricator.wikimedia.org/T90739#1066261 (10bd808) 3NEW a:3bd808 [17:41:00] AaronSchulz: Yup [17:41:01] already deploying [17:41:10] AaronSchulz: quick check? https://gerrit.wikimedia.org/r/#/c/192829/1/bin/mw-install-sqlite.sh [17:41:58] that works too [17:42:10] Because the basename is different [17:42:32] I guess I could even do /* entirely. [17:42:35] Not sure. [17:42:49] * doesn't do something crazy like match .. right? [17:44:50] also it seems like the wal/shm files aren't created until usage [17:44:58] so the chmod doesn't find them [17:45:11] maybe they can start off as empty [17:45:12] * AaronSchulz tests [17:45:31] AaronSchulz: Aye, and the cli run would find them first of course. Good point [17:45:38] AaronSchulz: Perhaps a sticky bit would work? [17:45:48] or some other mechanism that makes it do stuff.. [17:46:03] I guess sticky is only for group, not world [17:47:16] Hm.. I guess that only works based on environment, but based on directory. Something like umask [17:48:25] AaronSchulz: He, we could run a quick eval.php script that triggers creation within the install.sh [17:48:35] (if touching empty files doesn't work) [18:29:11] Krinkle|detached: have you tried https://gerrit.wikimedia.org/r/#/c/192829/1 ? [18:29:31] the shm file will get the right perms and the wal one won't exist, but it would be created by www-data, so that should be fine right? [18:31:03] bd808: your handling SoS? [18:31:12] *you're [18:31:17] AaronSchulz: sure [18:31:40] Wander in if you'd like but I can cover it [18:34:18] <^d> manybubbles: T90652 is hitting vagrant right now with Cirrus. I've isolated the cause but I'm not sure what's the right fix yet [18:34:33] <^d> (tldr: cache warmers at index creation fail due to no dynamic scripting) [18:46:36] TimStarling, AaronSchulz: Nikerabbit would like help figuring out how to make Special:SupportedLanguage query the DB in a performant way -- https://phabricator.wikimedia.org/T54728 [18:51:35] yes, I made some changes, but I need review. If those are not enough, I need pointers how to improve further ;) [18:51:48] bd808: thanks [18:56:05] Nikerabbit: yw. I'll add it to my reminder list too [19:39:35] 6MediaWiki-Core-Team, 10CirrusSearch: CirrusSearch: Ignore ( and ) in prefix search - https://phabricator.wikimedia.org/T89201#1066982 (10Manybubbles) p:5Triage>3Normal [19:44:57] 6MediaWiki-Core-Team, 10CirrusSearch: CirrusSearch: Ignore ( and ) in prefix search - https://phabricator.wikimedia.org/T89201#1067017 (10Manybubbles) >>! In T89201#1056485, @FriedhelmW wrote: > Please don't do this. In de.wikipedia, there a titles such as "(2) Pallas" where the parenthesis is meaningful. Wou... [19:57:23] 6MediaWiki-Core-Team, 10CirrusSearch: CirrusSearch: Ignore ( and ) in prefix search - https://phabricator.wikimedia.org/T89201#1067072 (10FriedhelmW) If the "(" is ignored, "(2" matches "2015" and more. [20:01:52] 6MediaWiki-Core-Team, 10CirrusSearch: CirrusSearch: Ignore ( and ) in prefix search - https://phabricator.wikimedia.org/T89201#1067113 (10Manybubbles) That doesn't sound that bad to me. I suppose it'd be better to require the ( if it is there but make it optional. That's a harder problem, but if its the righ... [20:12:04] 6MediaWiki-Core-Team, 10CirrusSearch: CirrusSearch: Ignore ( and ) in prefix search - https://phabricator.wikimedia.org/T89201#1067270 (10FriedhelmW) @Manybubbles: I don't understand your last comment. [20:13:41] 6MediaWiki-Core-Team, 10Beta-Cluster, 6operations, 7HHVM: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1067273 (10Dzahn) Or we might switch them over to Debian jessie right away? What do other ops think in this case? still trusty or jessie already? [20:15:46] 6MediaWiki-Core-Team, 10Beta-Cluster, 6operations, 7HHVM: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1067281 (10Dzahn) Resolving this should also prevent reverts like https://gerrit.wikimedia.org/r/#/c/192866/ [20:45:34] 6MediaWiki-Core-Team, 10Deployment-Systems, 10Librarization, 6Security, 5Patch-For-Review: Have a check for reported security issues in dependencies - https://phabricator.wikimedia.org/T74193#1067388 (10Legoktm) Deployed it as an experimental job for now: * https://integration.wikimedia.org/ci/job/mediaw... [20:53:08] 6MediaWiki-Core-Team, 10CirrusSearch: inefficient work of CirrusSearch in Russian Wikipedia - https://phabricator.wikimedia.org/T88724#1067410 (10Jdouglas) Here are some more examples that are related to (2) above: **target:** https://en.wikipedia.org/wiki/Functional_programming * intitle:functional intitle:... [21:40:57] Nikerabbit: DELETE FROM `translate_groupstats` WHERE tgs_group = 'page-Wikidata:List of properties/Summary table' AND tgs_lang = 'en' [21:41:11] why are there so many identical deletes at the same time? [21:41:20] like 50x [22:29:19] ori: can you review https://gerrit.wikimedia.org/r/#/c/189049 ? As I recall Brad's current stance on it is that it looks technically correct but he doesn't understand the use case and doesn't want to be the one to +2 [22:29:48] I was hoping since your name had been associated with the idea you would understand the use case [22:37:37] bd808: done [22:37:52] :) thx [22:46:38] AaronS: So setting chmod is fine? Or should I touch to create all three and then chmod? [22:47:26] I think it's fine [22:50:30] AaronS: OK [22:56:40] 6MediaWiki-Core-Team, 10MediaWiki-extensions-OAuth, 7Epic: Support a nice sso experience with MediaWiki's OAuth - https://phabricator.wikimedia.org/T86869#1068011 (10bd808) [22:56:41] 6MediaWiki-Core-Team, 10MediaWiki-extensions-OAuth, 5Patch-For-Review: Add way for OAuth apps to only authenticate (no other valid rights) - https://phabricator.wikimedia.org/T88757#1068012 (10bd808) [23:08:26] chmod 777 '/mnt/home/jenkins-deploy/tmpfs/mediawiki-core-qunit-karma/wikicache.*' No such file or directory [23:08:29] AaronS: ^ [23:10:02] yeah that happens if nothing was found, so I could see that notice showing up without the patch [23:15:38] TimStarling: you get random bug requests by email? [23:16:08] yes [23:16:25] this one was actually sent to my wikiquote user account via Special:Emailuser [23:16:42] how about phone calls? [23:16:45] I hope not [23:17:06] build4589.sqlite [23:17:19] Krinkle: is that how collisions are avoided? [23:17:25] ori: no [23:17:34] that would also be a problem with wikicache if it has no prefix [23:17:58] * AaronS wonders why there isn't a dir prefix instead [23:18:10] before the foundation got going, I distributed my mobile number to about a dozen active wikipedians so that they could call me if the site went down [23:18:24] this was pre-nagios, you understand [23:18:55] "Hey Tim, I'm getting this weird VE bug!" [23:19:01] I once got a call from that group, I think that was the only call from a community member I've received [23:19:16] Press 1 if Wikipedia is down [23:19:27] [hold music plays] [23:19:40] <^d> Your call is very important to us, please remain on the line. [23:19:45] hahaha [23:19:46] AaronS: No, the sqlite file doesn't need to be unique. Itcould just be wikidb.sqlite [23:19:55] But it's made somewhat unique for no good reason [23:20:04] AaronS: You hardcode "wikicache" though, so that should be fine. [23:20:54] there are adjectives that never go with "somewhat" [23:21:15] like "consistent", "reliable", "correct", etc. :) [23:21:21] ah, ok [23:21:32] AaronS: The other two files may not exist, but at least the sqlite file should exist right? [23:21:37] (after install.php) [23:21:41] right [23:21:43] for wikicache [23:21:45] Weird. [23:24:02] it passes on zend right? [23:45:48] chmod: cannot access ‘/mnt/home/jenkins-deploy/tmpfs/mediawiki-core-qunit-karma/*’: No such file or directory [23:45:54] Krinkle: any idea how that is possible? [23:46:05] AaronS: Is it not expanding the wildcard? [23:46:10] It should at least match the db flie [23:46:11] file [23:47:46] hrm, those quotes [23:48:56] Yeah, confirmed [23:49:24] * AaronS really really hates bash [23:49:30] even for simple things [23:51:41] 6MediaWiki-Core-Team, 10CirrusSearch: inefficient work of CirrusSearch in Russian Wikipedia - https://phabricator.wikimedia.org/T88724#1068268 (10Jdouglas) How is `"foo*"` //supposed// to work? The case of asterisks within quotes is not documented on [[ https://www.mediawiki.org/wiki/Help:CirrusSearch | Help:... [23:56:47] Krinkle: << find "$MW_DB_PATH" -type f | xargs chmod 0777 >> is probably cleaner [23:57:03] though I guess the path has no weird characters [23:57:19] AaronS: Oh, right. *that* [23:57:28] Because /* is expanded by bash [23:57:44] does it not turn them into separate arguments regardless of spaces or characters? [23:57:57] I thought foo/* becomes 'foo/1', 'foo/2' [23:58:25] without the comma of course [23:59:03] TimStarling: reminder re: