[00:00:22] (03PS28) 10Smalyshev: Performance tweaks [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 [00:02:02] (03CR) 10Smalyshev: Performance tweaks (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 (owner: 10Smalyshev) [00:03:59] (03CR) 10jenkins-bot: [V: 04-1] Performance tweaks [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 (owner: 10Smalyshev) [00:05:40] (03PS29) 10Smalyshev: Performance tweaks [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 [00:11:36] (03CR) 10Smalyshev: "Implemented state transition table. XML code is a bit ugly due to PHP 5.3 missing proper closure scope and fcall support... It may be fixe" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 (owner: 10Smalyshev) [00:34:24] [13Geo] 15JeroenDeDauw comment on pull request #47 144c31983: > Not sure about the syntax, but since the test succeed it obviously does what I wanted it to do.... 02http://git.io/jc2F [00:40:32] [13WikibaseDataModel] 15JeroenDeDauw 04force-pushed 06StatementByGuidMap from 14b383c72 to 14fd703a2: 02http://git.io/jcVg [00:40:32] 13WikibaseDataModel/06StatementByGuidMap 14fd703a2 15jeroendedauw: Initial work on introducing StatementByGuidMap... [00:41:43] wmde/WikibaseDataModel/StatementByGuidMap/fd703a2 : jeroendedauw The build has errored. http://travis-ci.org/wmde/WikibaseDataModel/builds/56039159 [05:12:31] (03PS3) 10Aude: Fix focus when clicking "add qualifier" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199962 (https://phabricator.wikimedia.org/T89300) [05:12:33] (03PS4) 10Aude: Fix focus on value field on propery select [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199910 (https://phabricator.wikimedia.org/T93864) [07:10:18] (03PS1) 10Smalyshev: T93451: ontology fix - ontology predicates go in lowercase [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200117 [07:11:08] (03PS2) 10Smalyshev: T93451: ontology fix - ontology predicates go in lowercase [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200117 [07:21:25] (03PS1) 10Smalyshev: T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 [07:22:49] (03PS2) 10Smalyshev: T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 [07:27:16] (03CR) 10jenkins-bot: [V: 04-1] T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 (owner: 10Smalyshev) [07:29:40] (03PS3) 10Smalyshev: T93451: ontology fixes - namespace the value predicates. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 [07:35:27] (03CR) 10jenkins-bot: [V: 04-1] T93451: ontology fixes - namespace the value predicates. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 (owner: 10Smalyshev) [08:40:49] 10Wikidata, 10Wikidata-Query-Service, 5Patch-For-Review: Data format updates for RDF export - https://phabricator.wikimedia.org/T93451#1156140 (10mkroetzsch) > Don't see why it would be this many. It'd be like 4 additional rows per property: I was referring to the labels. For some use cases, it could be con... [08:51:36] 10MediaWiki-extensions-WikibaseClient, 10Wikidata: Unstyled "Edit links" when previewing a page - https://phabricator.wikimedia.org/T94143#1156147 (10Schnark) 3NEW [09:02:56] (03PS1) 10Aude: Include style for "edit links" for page preview action [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200125 (https://phabricator.wikimedia.org/T94143) [09:03:23] 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 3§ Wikidata-Sprint-2015-03-24, 5Patch-For-Review: Unstyled "Edit links" when previewing a page - https://phabricator.wikimedia.org/T94143#1156160 (10aude) [09:03:37] 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 3§ Wikidata-Sprint-2015-03-24, 5Patch-For-Review: Unstyled "Edit links" when previewing a page - https://phabricator.wikimedia.org/T94143#1156162 (10aude) a:3aude [09:05:22] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 3§ Wikidata-Sprint-2015-03-24, 5Patch-For-Review: "Add qualifier" button should automatically focus text box - https://phabricator.wikimedia.org/T89300#1156164 (10aude) a:3aude [09:08:47] (03CR) 10jenkins-bot: [V: 04-1] Include style for "edit links" for page preview action [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200125 (https://phabricator.wikimedia.org/T94143) (owner: 10Aude) [09:08:49] (03PS4) 10Thiemo Mättig (WMDE): Split individual tests in UsageAccumulatorContractTester [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199588 [09:09:26] (03PS4) 10Thiemo Mättig (WMDE): Introduce TemplateFactory::getTemplates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/187129 [09:09:34] (03PS4) 10Thiemo Mättig (WMDE): Drop TemplateRegistry::getDefaultInstance [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/187130 [09:09:47] (03PS4) 10Thiemo Mättig (WMDE): Merge TemplateRegistry into TemplateFactory [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/187343 [09:09:55] jenkins failures are github timeouts [09:10:17] (03CR) 10Hoo man: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200125 (https://phabricator.wikimedia.org/T94143) (owner: 10Aude) [09:14:11] 10MediaWiki-extensions-WikibaseClient, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 3§ Wikidata-Sprint-2015-03-24, 5Patch-For-Review: UpdateRepo doesn't always propagate changes - https://phabricator.wikimedia.org/T92789#1156168 (10hoo) Once 9720c6c41079ff49804e1171eaa09121ddeaecc5 is deployed... [09:14:31] (03CR) 10jenkins-bot: [V: 04-1] Split individual tests in UsageAccumulatorContractTester [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199588 (owner: 10Thiemo Mättig (WMDE)) [09:15:49] oh... github is facing a DDoS attack, according to status.github.com [09:16:26] (03CR) 10jenkins-bot: [V: 04-1] Introduce TemplateFactory::getTemplates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/187129 (owner: 10Thiemo Mättig (WMDE)) [09:16:52] (03CR) 10jenkins-bot: [V: 04-1] Drop TemplateRegistry::getDefaultInstance [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/187130 (owner: 10Thiemo Mättig (WMDE)) [09:17:26] (03CR) 10jenkins-bot: [V: 04-1] Merge TemplateRegistry into TemplateFactory [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/187343 (owner: 10Thiemo Mättig (WMDE)) [09:17:31] (03PS2) 10Thiemo Mättig (WMDE): Update data-values/interfaces to 0.1.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199858 (https://phabricator.wikimedia.org/T92280) [09:24:04] [13Geo] 15thiemowmde 04force-pushed 06optionsNull from 144c31983 to 14e2275b4: 02http://git.io/jCFl [09:24:04] 13Geo/06optionsNull 14e2275b4 15Thiemo Mättig: Allow options to be null [09:24:22] (03CR) 10jenkins-bot: [V: 04-1] Update data-values/interfaces to 0.1.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199858 (https://phabricator.wikimedia.org/T92280) (owner: 10Thiemo Mättig (WMDE)) [09:31:58] [13Geo] 15thiemowmde comment on pull request #47 14e2275b4: I did not expected the `~` operator to change it's meaning (from major to minor) depending on the length of the string.... 02http://git.io/jCNm [09:33:25] (03CR) 10Hoo man: [C: 032 V: 032] "Looks sane, manually tested and ran the tests locally." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200125 (https://phabricator.wikimedia.org/T94143) (owner: 10Aude) [09:33:46] 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 3§ Wikidata-Sprint-2015-03-24, 5Patch-For-Review: Unstyled "Edit links" when previewing a page - https://phabricator.wikimedia.org/T94143#1156179 (10hoo) 5Open>3Resolved [09:35:32] Is there an archive with old dumps with history? I want to have a look at the early days of wikidata... [09:36:02] [13Time] 15thiemowmde 04force-pushed 06getFormatterClass from 141ed59dd to 14963af7b: 02http://git.io/pSFc [09:36:02] 13Time/06getFormatterClass 14963af7b 15Thiemo Mättig: Deprecate getFormatterClass in favor of getInstance [09:39:51] [13Number] 15thiemowmde 04force-pushed 06getFormatterClass from 14c85f595 to 14fc1bc18: 02http://git.io/xdPF [09:39:51] 13Number/06getFormatterClass 14fc1bc18 15Thiemo Mättig: Deprecate getFormatterClass in favor of getInstance [09:47:15] wikimedia/mediawiki-extensions-Wikibase/master/5325922 : Hoo man The build has errored. http://travis-ci.org/wikimedia/mediawiki-extensions-Wikibase/builds/56071353 [09:48:11] That's also github failing [09:51:27] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: Refactor ClaimsView and related php View code to work more nicely with statements - https://phabricator.wikimedia.org/T75279#1156204 (10aude) 5Open>3Resolved [09:51:58] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: Refactor ClaimsView and related php View code to work more nicely with statements - https://phabricator.wikimedia.org/T75279#756737 (10aude) ClaimsView no longer exists and many of these issues have been addressed (as well in the javascript too) [09:57:45] (03PS2) 10Hoo man: Mark accessing arbitrary items as expensive [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199969 (https://phabricator.wikimedia.org/T93885) [10:01:25] (03PS1) 10WikidataBuilder: New Wikidata Build - 2015-03-27T10:00:01+0000 [extensions/Wikidata] - 10https://gerrit.wikimedia.org/r/200127 [10:02:34] (03CR) 10jenkins-bot: [V: 04-1] Mark accessing arbitrary items as expensive [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199969 (https://phabricator.wikimedia.org/T93885) (owner: 10Hoo man) [10:05:15] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed in wikibase but has no meaning as xsd:dateTime - https://phabricator.wikimedia.org/T94064#1156221 (10mkroetzsch) Note that all current data representation formats assume that "0000-01-01T00:00:00" is a valid representation: * XML Schema 1.1:... [10:23:20] 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 5Patch-For-Review: Make mw.wikibase.getEntity and mw.wikibase.getEntityObject the same function - https://phabricator.wikimedia.org/T93607#1156232 (10Ricordisamoa) >>! In T93607#1143434, @hoo wrote: >>>! In T93607#1143412, @Ricordisamoa wrote: >> Does this m... [11:00:25] morning ! i was wondering why the revid of entities used isn't available in wbUsedEntities, seeing as their descriptions, labels, type, and id are already there [11:02:25] it would provide a very efficient way of making sure you're not editing a remote page based on outdated information on the current page [11:09:29] is daniel kinzler around? [11:11:21] ping-timeout'ed a little over an hour ago, sorry [11:11:27] thx [11:11:34] yw [11:22:44] 10Wikidata, 10Continuous-Integration: PHP fatal errors are not visible in jenkins output in mwext-Wikibase-client-tests job - https://phabricator.wikimedia.org/T92397#1156368 (10Krinkle) [12:22:09] (03PS1) 10Adrian Lang: Refactor badgeselector [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200144 [12:25:40] (03CR) 10jenkins-bot: [V: 04-1] Refactor badgeselector [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200144 (owner: 10Adrian Lang) [12:28:31] 10MediaWiki-extensions-WikibaseRepository: Put lastrevid of linked entities in mw.config.values.wbUsedEntities - https://phabricator.wikimedia.org/T94159#1156519 (10Alphos) 3NEW [12:28:46] wikibugs : shhhh :p [12:30:41] (03PS2) 10Adrian Lang: Refactor badgeselector [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200144 [12:33:08] 10MediaWiki-extensions-WikibaseRepository: Put lastrevid of linked entities in mw.config.values.wbUsedEntities - https://phabricator.wikimedia.org/T94159#1156531 (10adrianheine) I'm currently working on removing wbUsedEntities, see https://gerrit.wikimedia.org/r/#/c/197290/. You would have to do an API request t... [12:33:25] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: Refactor Wikibase API tests to avoid database access - https://phabricator.wikimedia.org/T70917#1156532 (10Krinkle) [12:45:33] DanielK_WMDE_, ping ))https://phabricator.wikimedia.org/T93897 [12:46:40] DanielK_WMDE_, i am basically proposing to rethink our approach to the api, especially query - to treat it as the business logic [12:46:55] and wrap the formatting for external use on top of it [12:50:28] 10MediaWiki-extensions-WikibaseRepository: Put lastrevid of linked entities in mw.config.values.wbUsedEntities - https://phabricator.wikimedia.org/T94159#1156571 (10Alphos) I did find wgCurRevisionId ;-) Too bad about wbUsedEntities being dropped, it means we're either : - facing race conditions (and potentia... [12:52:18] 10Wikidata, 10Continuous-Integration, 7Browser-Tests, 7Tracking: Delete or fix failed Wikidata browsertests Jenkins job - https://phabricator.wikimedia.org/T94163#1156574 (10zeljkofilipin) 3NEW [13:01:40] 10Wikidata, 10Continuous-Integration, 7Browser-Tests, 7Tracking: Delete or fix failed Wikidata browsertests Jenkins job - https://phabricator.wikimedia.org/T94163#1156595 (10hashar) The Wikidata-WikidataTests one got aborted due to a timeout. It has been raised from 3 to 4 hours with https://gerrit.wikimed... [13:05:47] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 7Performance: Load widgets on demand - https://phabricator.wikimedia.org/T62341#1156607 (10adrianheine) p:5High>3Low Currently we are trying to do less on page load within the current architecture, but we are not planning to completely defer initiali... [13:15:28] (03CR) 10Thiemo Mättig (WMDE): Provide documentation for usage tracking migration (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198223 (owner: 10Daniel Kinzler) [13:15:47] (03CR) 10Thiemo Mättig (WMDE): [C: 031] Provide documentation for usage tracking migration [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198223 (owner: 10Daniel Kinzler) [13:46:10] 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 7Performance: Preload labels and descriptions for Lua and the parser function based on usage tracking data - https://phabricator.wikimedia.org/T76159#1156698 (10Lydia_Pintscher) [13:48:30] (03CR) 10Aude: [C: 032] Provide documentation for usage tracking migration [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198223 (owner: 10Daniel Kinzler) [13:52:07] (03Merged) 10jenkins-bot: Provide documentation for usage tracking migration [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198223 (owner: 10Daniel Kinzler) [13:55:17] (03PS1) 10Aude: Add clarification about subscriptions table deployment [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 [13:58:00] (03PS5) 10Aude: Fix focus on value field on property select [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199910 (https://phabricator.wikimedia.org/T93864) [13:58:26] (03PS4) 10Aude: Fix focus when clicking "add qualifier" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199962 (https://phabricator.wikimedia.org/T89300) [13:58:41] DanielK_WMDE_, i replied - i think you misunderstood my point - i do want to get rid of output format in api - https://phabricator.wikimedia.org/T93897 [13:59:02] Is there an archive with old dumps with history? I want to have a look at the early days of wikidata... [14:00:36] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: Server-side rendering for all property links in the JS UI - https://phabricator.wikimedia.org/T88426#1156747 (10adrianheine) [14:00:36] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 7Performance: Drop referenced entities from JS config - https://phabricator.wikimedia.org/T88428#1156746 (10adrianheine) [14:00:50] wikimedia/mediawiki-extensions-Wikibase/master/5731606 : jenkins-bot The build has errored. http://travis-ci.org/wikimedia/mediawiki-extensions-Wikibase/builds/56095421 [14:01:10] (03PS2) 10Adrian Lang: Remove wbUsedEntities [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197290 (https://phabricator.wikimedia.org/T88428) [14:09:38] 10MediaWiki-extensions-WikibaseRepository: Put lastrevid of linked entities in mw.config.values.wbUsedEntities - https://phabricator.wikimedia.org/T94159#1156774 (10adrianheine) Can you give me a use-case for editing an entity referenced by the entity you're currently on? I think it makes more sense to provide t... [14:26:33] (03CR) 10Adrian Lang: [C: 031] "Seems great" (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199962 (https://phabricator.wikimedia.org/T89300) (owner: 10Aude) [14:39:08] 10Wikidata: Batching and caching sitelink lookup for use from Lua snak formatter - https://phabricator.wikimedia.org/T94168#1156821 (10daniel) [14:42:47] (03PS5) 10Aude: Fix focus when clicking "add qualifier" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199962 (https://phabricator.wikimedia.org/T89300) [14:44:21] 10Wikidata: Implement RDR support in RdfWriter - https://phabricator.wikimedia.org/T94173#1156837 (10daniel) 3NEW [14:46:11] 10Wikidata: Implement RDR support in RdfWriter - https://phabricator.wikimedia.org/T94173#1156849 (10daniel) [14:47:03] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph - https://phabricator.wikimedia.org/T94064#1156851 (10JanZerebecki) [14:47:05] (03CR) 10jenkins-bot: [V: 04-1] Fix focus when clicking "add qualifier" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199962 (https://phabricator.wikimedia.org/T89300) (owner: 10Aude) [14:48:30] 10Wikidata: Implement a validating decorator for RdfWriter - https://phabricator.wikimedia.org/T94174#1156852 (10daniel) 3NEW [14:48:48] 10MediaWiki-extensions-WikibaseRepository: Put lastrevid of linked entities in mw.config.values.wbUsedEntities - https://phabricator.wikimedia.org/T94159#1156858 (10Alphos) I'm working on a gadget that will allow you to insert by hand only one side of a relationship, and click for an automated insertion of the o... [14:50:25] lbenedix: the full dump contains the history [14:51:44] 10Wikidata: Split RdfBuilder - https://phabricator.wikimedia.org/T94175#1156859 (10daniel) 3NEW [14:56:47] 10Wikidata: Use callback table to handle state transitions in RdfWriterBase - https://phabricator.wikimedia.org/T94179#1156894 (10daniel) 3NEW [14:57:31] it does, but crawling through a few 100MB should be faster than through the actual dump with >500GB uncompressed xml [14:59:26] lbenedix: as far as I can tell, the oldest available full dump is from July 2014. I don't think that is sufficiently old for your need then. You'll have to go through the full dump. [15:00:17] basically, you can just go through it once with a script that throws out everything after a given date, and then work on the result. The first script will take a few hours, but after that you're fine [15:00:30] 10Wikidata, 10Continuous-Integration, 7Browser-Tests, 7Tracking: Delete or fix failed Wikidata browsertests Jenkins job - https://phabricator.wikimedia.org/T94163#1156904 (10Tobi_WMDE_SW) Ok, first, please do not remove the Wikidata jobs, even if they are failing. There are several reasons: 1) Due to the h... [15:00:48] that was plan b if no old dump can be found [15:01:13] go for plan b like benedix ;) [15:02:00] I found this: ftp://ftp.muni.cz/pub/wikimedia/wikidatawiki/20121109/ [15:08:47] (03CR) 10Aude: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199962 (https://phabricator.wikimedia.org/T89300) (owner: 10Aude) [15:41:55] 10Wikidata, 10Datasets-General-or-Unknown, 3§ Wikidata-Sprint-2015-03-24: Wikidata JSON dump: file directory location should follow standard patterns - https://phabricator.wikimedia.org/T72385#1157028 (10daniel) For the record: while I was proposing to have the dump "flavor" at the bottom of the hierarchy an... [15:42:17] 10Wikidata: Generate RDF from JSON - https://phabricator.wikimedia.org/T94019#1152708 (10daniel) [15:42:19] 10Wikidata, 10Datasets-General-or-Unknown, 3§ Wikidata-Sprint-2015-03-24: Wikidata JSON dump: file directory location should follow standard patterns - https://phabricator.wikimedia.org/T72385#1157030 (10daniel) [15:53:56] aude: https://www.mediawiki.org/wiki/Architecture_committee/2014-10-08#RFCs_to_triage [15:56:09] aude: https://phabricator.wikimedia.org/T592 [15:56:14] thanks [16:10:22] Can someone reinvite me to -admin? [16:12:17] 10Wikidata, 3§ Wikidata-Sprint-2015-03-24: Move RdfWriter to separate component - https://phabricator.wikimedia.org/T94172#1157107 (10daniel) [16:15:09] done ;p [16:17:51] DanielK_WMDE_: A question that I just stumbled upon: Should we still fallback to attempt to load revision data from master, like Revision used to? [16:31:46] hoo: only when trying to edit [16:33:51] DanielK_WMDE_: mh... right now we do that all the time and I'll also do it in the batched thing, then [16:33:56] but I don't really like that [16:34:06] adds up round trips and potentially master reads [16:34:25] hoo: yes, i agree. we should have a way to only do this if we really need it. [16:34:32] actually, for the batched stuff, we probably never do [16:34:36] since we don't do mass edits [16:34:57] True... maybe do what Aaron is currently doing to core as well [16:35:10] (the new DB_* constant, the thing I showed you yesterda) [16:35:10] y [16:36:52] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph - https://phabricator.wikimedia.org/T94064#1157193 (10JanZerebecki) The lexical representation where the year fraction is 0 has undefined meaning in Wik... [16:37:12] hoo: yes, why not [16:37:54] Ok, will create a bug for that [16:38:13] Once I have more stable internet and don't need to wait an hour for phab [16:38:34] is this a bad time to mess with the Wikibase jenkins job to remove the dependency upon git.wm.o? [16:38:55] * hoo is not about to upload anyhting [16:38:59] How long will the down time last? [16:39:06] well "down time" [16:40:06] github has been down today [16:40:19] which has resulted in jenkins jobs failing [16:40:27] * aude is not uploading stuff [16:40:36] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 5Patch-For-Review, 7Performance, and 2 others: Implement an EntityPrefetcher that would take a list of entity ids and preload revision meta data about them - https://phabricator.wikimedia.org/T87238#1157216 (10hoo) [16:41:12] https://status.github.com/ they're being DDoS'd? o.O [16:41:20] yeah :( [16:41:55] hoo: 5-10 min I guess? I'll modify the job, test if it works, revert if it doesn't. if it does I'll update all the other jobs too [16:41:59] but if github is down, this probably isn't a good time [16:42:15] i think it's back [16:42:24] I think it's good again [16:43:35] heh https://github.com/greatfire/ [16:43:36] ok [16:45:01] I'm testing with the 'mwext-Wikibase-repo-tests' job [16:45:30] (03PS1) 10Legoktm: jenkins job validation, do not submit [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200175 [16:49:06] (03CR) 10jenkins-bot: [V: 04-1] jenkins job validation, do not submit [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200175 (owner: 10Legoktm) [16:49:09] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph - https://phabricator.wikimedia.org/T94064#1157247 (10JanZerebecki) About representation of years BCE: Thank you for starting that mailing list thread.... [16:51:02] (03CR) 10Legoktm: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200175 (owner: 10Legoktm) [16:51:14] SMalyshev: i'm preparing a patch that moves all the RdfWriter related files to a separate component (just a top level directory, not a separate git repo, for now) [16:51:35] SMalyshev: i want to do this asap, before we have more thigns depending on it [16:54:26] 10Wikidata, 10MobileFrontend-Feature-requests: Setup WikidataPageBanner extension repository on Gerrit - https://phabricator.wikimedia.org/T93127#1157264 (10Sumit) Request for new gerrit repository for wikidata pagebanner extension can be tracked at the bottom of the table here: https://www.mediawiki.org/wiki/... [16:55:21] 16:54:47 Time: 0 ms, Memory: 311.75Mb [16:55:21] 16:54:47 [16:55:21] 16:54:47 No tests executed! [16:55:29] so.....that didn't work :P [16:57:51] * legoktm tweaks [16:59:59] :( [17:02:03] argh, again :/ [17:03:11] oh, I think I know why [17:03:50] it's writing to the wrong LocalSettings.php [17:04:24] addshore: heya! can you tell me where i can add a test group for jenkins to run? [17:06:50] DanielK_WMDE_: repo: integration/config, the jjb/wikidata.yaml file [17:07:32] and we're going to clone core again, yippe. [17:07:51] :D [17:08:42] legoktm: thanks, jzerebecki offerd he'd do it [17:10:48] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: Remove obsolete withoutTermWeight Wikibase setting - https://phabricator.wikimedia.org/T94197#1157321 (10aude) 3NEW [17:10:53] jzerebecki: https://phabricator.wikimedia.org/T94172 [17:11:35] 17:10:50 + 'cd /mnt/jenkins-workspace/workspace/mwext-Wikibase-repo-tests/src' [17:11:35] 17:10:50 /tmp/hudson2379460967171661945.sh: line 2: cd /mnt/jenkins-workspace/workspace/mwext-Wikibase-repo-tests/src: No such file or directory [17:11:37] eh [17:12:23] legoktm: hu? [17:12:38] it totally does exist [17:13:20] jzerebecki: I'm testing out https://gerrit.wikimedia.org/r/200122 https://integration.wikimedia.org/ci/job/mwext-Wikibase-repo-tests/13177/consoleFull [17:14:05] Yay! thx. [17:15:44] (03PS1) 10Daniel Kinzler: Moving RdfWriter and friends to a separate component called Purtle. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 [17:17:16] (03PS2) 10Daniel Kinzler: Moving RdfWriter and friends to a separate component called Purtle. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 [17:20:11] (03CR) 10jenkins-bot: [V: 04-1] Moving RdfWriter and friends to a separate component called Purtle. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 (owner: 10Daniel Kinzler) [17:24:56] (03PS1) 10Legoktm: build: jenkins now clones to "src/" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200185 [17:25:37] aude: got it! just needed ^ [17:25:58] \o/ [17:25:59] (03PS4) 10Smalyshev: T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 [17:26:45] (03CR) 10jenkins-bot: [V: 04-1] build: jenkins now clones to "src/" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200185 (owner: 10Legoktm) [17:27:21] aude: if I deploy this, you'll have to backport that to any other active branches otherwise everything will fail, is that ok? [17:27:35] (and it's failing because I didn't update the other jobs yet) [17:28:06] legoktm: ok with me [17:28:14] * aude doing backports on monday [17:28:21] aude : i promised you a new gadget "before this weekend", how long do i still have for that ? :p [17:28:53] (03CR) 10Smalyshev: Moving RdfWriter and friends to a separate component called Purtle. (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 (owner: 10Daniel Kinzler) [17:29:00] (03CR) 10jenkins-bot: [V: 04-1] T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 (owner: 10Smalyshev) [17:30:07] Alphos: midnight local hawaii time? :) [17:30:20] i'll try and make that a deal then :D [17:30:27] ok :) [17:30:52] what's going on with jenkins? doesn't run tests at all [17:30:55] that leaves me an hour to finish 2 measly methods, and about 9 hours to debug those measly methods :D [17:31:05] SMalyshev: sorry, I'm nearly done breaking everything to unbreak stuff :) [17:31:26] need another ~5 min [17:31:32] legoktm: ok, waiting :) [17:31:42] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 5Patch-For-Review, 7Tracking: Implement complete RDF mapping for entities (tracking) - https://phabricator.wikimedia.org/T50143#1157350 (10daniel) [17:31:43] 10Wikidata, 3§ Wikidata-Sprint-2015-03-24, 5Patch-For-Review, 3§ Wikidata-Sprint-2015-03-11: Implement fast sequential RDF output generation - https://phabricator.wikimedia.org/T92523#1157349 (10daniel) 5Open>3Resolved [17:32:24] (03PS3) 10Daniel Kinzler: Moving RdfWriter and friends to a separate component called Purtle. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 (https://phabricator.wikimedia.org/T94172) [17:32:57] (03CR) 10Legoktm: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200185 (owner: 10Legoktm) [17:33:03] DanielK_WMDE_: you don't want to finish with performance, e.g. https://gerrit.wikimedia.org/r/#/c/198115/, before moving things around? [17:34:02] SMalyshev: it's going to be annoying either way, i think. [17:34:25] DanielK_WMDE_: if we finish with perf tweaks fisrt it won't be [17:34:31] *first [17:34:35] if terms are suggested by number of site links... [17:34:37] oh, it's going to be more like 10-15 minutes, I forgot that we have to re-clone mediawiki/core like 5 times again [17:34:38] https://www.wikidata.org/wiki/Q6581097 has 0 [17:34:45] but still was suggested at the top [17:35:27] SMalyshev: there's a ticket for the transition table thing, btw https://phabricator.wikimedia.org/T94179 [17:35:44] DanielK_WMDE_: this would probably be last large change to the writers, since all other tasks relate to RdfBuilder [17:36:15] DanielK_WMDE_: ok, I've implemented the table, you seen it? [17:36:35] (03CR) 10jenkins-bot: [V: 04-1] Moving RdfWriter and friends to a separate component called Purtle. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 (https://phabricator.wikimedia.org/T94172) (owner: 10Daniel Kinzler) [17:37:46] 10Wikidata: Use callback table to handle state transitions in RdfWriterBase - https://phabricator.wikimedia.org/T94179#1157389 (10Smalyshev) See https://gerrit.wikimedia.org/r/#/c/198115/ [17:38:19] SMalyshev: yes, it looks really nice! I especially like that you can put in strings directly, and avoid the call. should speed things up quite a bit, I hope. [17:39:16] DanielK_WMDE_: it speeds up, not drastically (compared to previous one) but compared to what we have now it's about 2x I think [17:39:35] nice! [17:39:48] aude, jzerebecki: +2 https://gerrit.wikimedia.org/r/#/c/200185/ ? [17:39:55] if we do that and fix prefixes, I think we're in good place with performance [17:41:06] SMalyshev: the one thing that i find really problematic is $dataType = $typeId; that's just wrong. and i see no reason to do it that way. [17:41:12] (03CR) 10Aude: [C: 032] build: jenkins now clones to "src/" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200185 (owner: 10Legoktm) [17:41:15] legoktm: ^ [17:41:26] woot [17:41:30] :) [17:41:40] except for that, the performance patch is ready to merge (and I'll fix the "move stuff" patch accordingly) [17:42:44] DanielK_WMDE_: it has been like that since the beginning. This patch doesn't change anything [17:43:12] DanielK_WMDE_: it just handles one specific case like we handled all cases before it. [17:43:25] SMalyshev: yea, it adds a second instance of the breakage. i overlooked this the first time around. [17:43:37] i'll just make a follow up to fix it. [17:43:54] but the exception handling shouldn't be in this patch eiher, really. it's unrelated to performance [17:43:58] DanielK_WMDE_: it's the same instance, it just makes it not throw exceptions. If you want to fix it, fine, but this patch is not the right place [17:44:17] SMalyshev: this patch is not the righ place for the exception handling either :) [17:44:19] DanielK_WMDE_: it's unrelated, true, but without this fix I can't run dumps on test setups [17:44:42] DanielK_WMDE_: I can make separate patch for just exception fix but really is it worth wasting time on that? [17:44:53] it has to be done anyway [17:44:53] that'S not an argument for having it in this patch... in a separate patch, this would have been merged long ago [17:44:58] sure [17:45:19] but separating concerns helps with reviews. because then, disagreements do not block unrelated things [17:45:26] anyway, i [17:45:32] i'll merge the performance patch now [17:45:36] yeah I know, but right now it's already reviewed anyway :) [17:45:40] aude: umm https://integration.wikimedia.org/ci/job/mwext-Wikibase-repo-api-tests/6537/console [17:45:59] 10Wikidata: Use callback table to handle state transitions in RdfWriterBase - https://phabricator.wikimedia.org/T94179#1157437 (10daniel) Related patch https://gerrit.wikimedia.org/r/#/c/198115/29 [17:46:00] I'm making the patches smaller but sometimes things crop up, especially with long-running ones [17:46:01] (03CR) 10jenkins-bot: [V: 04-1] build: jenkins now clones to "src/" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200185 (owner: 10Legoktm) [17:46:22] 17:45:55 Error: 13 database or disk is full [17:46:23] oh come on [17:46:41] /dev/vda2 1.9G 1.8G 160K 100% /var [17:46:41] not again :( [17:46:43] yep [17:46:46] (03CR) 10Daniel Kinzler: [C: 032] "The data type fallback in RdfBuilder is fishy, but i'll fix that in a follow-up." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 (owner: 10Smalyshev) [17:46:55] wohoo! \o/ [17:47:46] (03CR) 10Legoktm: [C: 032] "Try again plz" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200185 (owner: 10Legoktm) [17:48:22] grrr [17:50:15] I, again, forgot that phpunit runs provider functions before setUp :S [17:50:57] (03CR) 10jenkins-bot: [V: 04-1] Performance tweaks [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 (owner: 10Smalyshev) [17:51:29] SMalyshev: Fun fact: the data type "globe-coordinate" uses the value type "globecoordinate" (with no dash). don't ask... [17:51:38] hoo: do you have any idea where in the code would handle suggesting "male" and "female" first for property P21 ? [17:51:57] it has no site links, so if it's simply by term weight, then it would not rank 1st [17:52:08] and i don't see where in the property suggester that it handles this [17:52:09] DanielK_WMDE_: so what you want to do with types? I don't want to call property lookup each time, it's slow [17:52:13] aude: You mean to manipulate the property suggester? [17:52:21] i don't know [17:52:33] DanielK_WMDE_: we go by value types (since that's what matters for understanding the value) [17:52:53] i checked out wmf19 and still don't see a difference in how search is done in the suggester (for values) [17:53:17] (03PS1) 10Legoktm: jenkins job validation, don't submit [extensions/Wikidata] - 10https://gerrit.wikimedia.org/r/200197 [17:53:20] SMalyshev: for string values, you have to do a lookup. For entity-id, you would too, but these are handled as resources. so you only need a static mapping for the rest. [17:53:31] SMalyshev: making a patch now [17:53:34] ok [17:55:08] (03CR) 10Legoktm: "check experimental" [extensions/Wikidata] - 10https://gerrit.wikimedia.org/r/200197 (owner: 10Legoktm) [17:58:10] (03Merged) 10jenkins-bot: build: jenkins now clones to "src/" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200185 (owner: 10Legoktm) [17:58:41] SMalyshev: i now see that $dataType isn't even used anywhere exccept for string values. then it really doesn't make a difference. [17:58:55] i'll just make it null in all other cases. [17:59:02] DanielK_WMDE_: that's true it's needed only for strings [17:59:21] that makes it a lot simpler [17:59:28] (if some more multi-value type pops up, it may change, but until then strings) [17:59:40] so null is fine too [18:00:11] SMalyshev: ok, jenkins should be un-broken now [18:01:09] (03PS1) 10Daniel Kinzler: Fix misleading mixing of data type and value type. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200200 [18:01:20] SMalyshev: --^ +2 please :) [18:02:16] now, let's see how smart git is about moved files [18:02:51] (03CR) 10jenkins-bot: [V: 04-1] jenkins job validation, don't submit [extensions/Wikidata] - 10https://gerrit.wikimedia.org/r/200197 (owner: 10Legoktm) [18:03:04] (03CR) 10Smalyshev: [C: 032] "Looks fine" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200200 (owner: 10Daniel Kinzler) [18:03:26] DanielK_WMDE_: yeah since unknown and null both work for string it's fine [18:03:42] aude: the experimental mwext-Wikidata-* jobs are broken, is that a major issue or can I fix it later? [18:04:07] SMalyshev: yes, could have used "unknown", doesn't matter [18:04:10] or "???" [18:04:23] yeah, doesn't matter same result [18:05:19] legoktm: how do I tell gerrit to rerun the tests on patch? [18:05:31] SMalyshev: comment "recheck" [18:05:42] oh, nice. rebase of my "move everything" patch applied nicely. git is pretty smart about page moves if you don't change the files too much [18:05:49] (03CR) 10Smalyshev: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 (owner: 10Smalyshev) [18:06:24] on $job-1 that was "recheck this please" :) [18:06:52] legoktm: i don't know that we need those jobs now [18:07:15] i don't think we have any experimental features at the moment and haven't been using the flag for a while afaik [18:08:12] oh, git diff -M is nice [18:08:31] ok, I'll just leave them broken for now and file a follow up bug [18:08:36] ok [18:08:43] they can be non-voting or whatever [18:09:39] (03PS4) 10Daniel Kinzler: Moving RdfWriter and friends to a separate component called Purtle. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 (https://phabricator.wikimedia.org/T94172) [18:12:55] SMalyshev: regarding prefixes: write() supports callbacks. You can do write( array( $this, 'getPrefixDeclarations' ) ) on start -> document. [18:13:08] for turtle, i mean [18:13:20] rdf/xml already does this [18:13:21] DanielK_WMDE_: hmm... maybe. [18:13:31] it would keep the interface stable [18:13:33] DanielK_WMDE_: I'll try that [18:13:43] :) [18:14:02] aude: also right now Wikibase phpunit is still using sqlite, might want to switch over to mysql like all other extensions [18:14:02] I just want to kill cleanupPrefixes [18:14:17] SMalyshev: being able to put sub-writers and callbacks into the output buffer is really a powerful tool, it makes a lot of thigns quite simple. [18:14:20] it's time for it to go [18:14:25] yes, of course [18:14:45] 10Wikidata, 10Wikidata-Query-Service, 5Patch-For-Review: Data format updates for RDF export - https://phabricator.wikimedia.org/T93451#1157521 (10Smalyshev) Ah, no, that'd not be good to add so many labels. But :P1234q or q:P1234 are predicates, and RDF browsers should be able to handle predicates without la... [18:15:35] DanielK_WMDE_: about the split, we need to be careful with the tests so that tests that go to purtle/ do not include Wikibase ontologies [18:15:42] n3 allows prefixes to be re-defined mid-document. but we don't really support this anyway, since it would be tricky to do for xml. [18:16:11] not sure if it's the case or not but otherwise it would be hard to tweak the ontology [18:16:11] legoktm: we can do that? [18:16:12] SMalyshev: yes, think i took care that that's the case. [18:16:17] ok, cool [18:16:28] aude: yup :) [18:16:28] * aude would be very happy to switch [18:16:32] :D [18:16:32] just look over it. i'm only moving stuff around. [18:16:42] since when? [18:17:08] DanielK_WMDE_: I don't care about prefixes mid-flight. We declare them upfront anyway. but callback should be fine with that [18:17:10] aude: that effectively means dropping sqlite support [18:17:14] (03Merged) 10jenkins-bot: Performance tweaks [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198115 (owner: 10Smalyshev) [18:17:19] a week or two ago? Krinkle did most of the work [18:17:21] DanielK_WMDE_: sqlite still runs on travis [18:17:28] if we want to support it [18:17:29] hmhm [18:17:43] DanielK_WMDE_: you could also run both mysql and sqlite tests [18:18:10] * aude owes beer for krinkle at the hackathon :) [18:19:42] anyways, whenever you guys figure out what you want, file a bug and cc me :) [18:20:08] 10Wikidata, 5Patch-For-Review: Use callback table to handle state transitions in RdfWriterBase - https://phabricator.wikimedia.org/T94179#1157528 (10daniel) [18:20:22] (03PS5) 10Smalyshev: T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 [18:20:22] 10Wikidata, 10Continuous-Integration: Switch or add additional jenkins jobs to run Wikibase tests with mysql - https://phabricator.wikimedia.org/T94208#1157529 (10aude) 3NEW [18:20:23] 10Wikidata, 5Patch-For-Review: Use callback table to handle state transitions in RdfWriterBase - https://phabricator.wikimedia.org/T94179#1157536 (10daniel) 5Open>3Resolved a:3daniel [18:20:24] ok [18:20:26] made a bug [18:21:08] we'll have to make sure everyone knows about it and decide if we switch or run both [18:21:28] jzerebecki: https://gerrit.wikimedia.org/r/#/c/200200/ [18:22:21] before: real 0m43.681s [18:22:36] after: 0m26.750s [18:22:42] yea! [18:22:44] not bad [18:22:52] down from originally what, 2:40? [18:23:06] yup something like that [18:23:17] sorry to bother you guys again, but where can i find a list of value types and their structures ? wbparsevalue won't suffice, since i'm actually trying to compare two values depending on their type... [18:23:28] (03CR) 10JanZerebecki: [C: 032] "per Smalyshev" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200200 (owner: 10Daniel Kinzler) [18:23:55] Alphos: https://www.mediawiki.org/wiki/Wikibase/DataModel/JSON [18:24:22] thanks ! [18:24:39] there is also https://www.wikidata.org/wiki/Special:ListDatatypes [18:24:58] aude: that page is so strange... [18:25:02] heh [18:25:04] also this I guess: https://www.mediawiki.org/wiki/Wikibase/DataModel [18:25:16] looks reasonably up-to-date to me [18:25:20] aude: it's also wrong :) we should probably ditch it [18:25:20] 10Wikidata, 10Continuous-Integration, 5Patch-For-Review, 7Technical-Debt: Remove dependency on git.wikimedia.org - https://phabricator.wikimedia.org/T74001#1157563 (10Legoktm) Wikibase/Wikidata jobs are now using zuul-cloner. Only usage left of the 'mw-setup-extension' macro is browser tests ('{name}-{ext-... [18:25:41] i'm sure there are wrong things but mostly looks correct [18:25:41] aude: "the year always having 11 digits " i don't think that was ever true, actually [18:25:46] :o [18:25:50] aude: seems up to date [18:26:06] the list of parameters should at least be correct [18:26:16] DanielK_WMDE_: in the data, I think the year does always have lots of digits (never counted them :) [18:26:19] what they do and how to interpret them might not be clear or correct [18:26:28] * Alphos clicks on ALL THE LINKS ! :) [18:26:31] SMalyshev: sixteen, usually. but not always. [18:26:36] in that case, refer to the data model documentaiton [18:32:27] wikimedia/mediawiki-extensions-Wikibase/master/5e67c52 : Kunal Mehta The build has errored. http://travis-ci.org/wikimedia/mediawiki-extensions-Wikibase/builds/56128819 [18:35:18] (03PS1) 10Hoo man: Factor database access out of WikiPageEntityRevisionLookup [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200209 [18:35:37] Oh noes, I missed the 200k today :P [18:35:49] :( [18:36:02] 200K what ? [18:36:13] https://gerrit.wikimedia.org/r/#/c/200000 [18:36:19] ah [18:38:57] * hoo needs to reboot... my cursor is going crazy again :S [18:39:35] (03CR) 10jenkins-bot: [V: 04-1] Factor database access out of WikiPageEntityRevisionLookup [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200209 (owner: 10Hoo man) [18:40:30] (03PS1) 10Daniel Kinzler: Time validator should not accept years with less than four digits [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200210 [18:40:41] (03Merged) 10jenkins-bot: Fix misleading mixing of data type and value type. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200200 (owner: 10Daniel Kinzler) [18:41:11] woaaah, travis-ci totally changed their UI [18:41:30] just to mess with your head [18:41:43] :o [18:42:11] * aude sad https://travis-ci.org/wikimedia/mediawiki/jobs/56131066 [18:42:59] wikimedia/mediawiki-extensions-Wikibase/master/0f45978 : jenkins-bot The build passed. http://travis-ci.org/wikimedia/mediawiki-extensions-Wikibase/builds/56131094 [18:43:53] Is jenkins awry or did I actually break stuff? [18:44:12] hoo: legoktm has been working on our jenkins [18:44:16] to have it not use gitblit [18:44:33] * legoktm looks [18:44:37] you might need to rebase [18:44:45] yeah [18:45:13] but also Catchable fatal error: Argument 2 passed to Wikibase\Lib\Store\WikiPageEntityRevisionLookup [18:45:23] doesn't look like a jenkins issue to me [18:45:26] actually looks like something is broken [18:45:27] yeah [18:46:02] jenkins always does rebase [18:46:06] afaik [18:46:09] Oh, doh, yes [18:46:15] seems I forget to update a caller [18:46:52] * aude puzzled how P21 : male got prioritized in the suggester [18:47:36] (03PS2) 10Hoo man: Factor database access out of WikiPageEntityRevisionLookup [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200209 [18:48:11] aude: That worked in the past? [18:49:10] DanielK_WMDE_: https://www.wikidata.org/wiki/Q6581097 [18:49:14] apparently [18:49:38] * aude could probably implement better suggestions [18:49:41] based on elastic [18:50:36] 10Wikidata, 10Citoid, 6Editing, 10Possible-Tech-Projects, and 2 others: Create a system to store and query links to books - https://phabricator.wikimedia.org/T90852#1157689 (10CeruleanDreamer) I would like to choose this as a possible project for my Outreachy application.I recognize that I am reaching the... [18:58:02] ugh I really hate that 5.3 closures do not have class context... makes code so much uglier [18:58:12] :( [18:58:38] we should really move to 5.4 already :) [18:58:41] bites us often when writing mocks in phpunit tests [18:58:50] +1 [18:58:59] Amir1: I'm writing a pywikibot import script for Wikidata. Is there a best way to create an ItemPage from a label or sitelink? [18:59:27] Nemo_bis: hey, in a meeting [18:59:38] Sure, no hurry :) [18:59:39] catch you in one hour. ok? [18:59:43] Great [19:00:18] (03CR) 10Daniel Kinzler: Factor database access out of WikiPageEntityRevisionLookup (033 comments) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200209 (owner: 10Hoo man) [19:01:24] DanielK_WMDE_: I looked into the prefixes thing and I think we still need writePrefixes() function [19:01:41] DanielK_WMDE_: I need to tell the writer when to write them (or not) [19:01:46] DanielK_WMDE_: Do you think the implementation of our prefetcher should wrap around that new class and act as a cache? [19:02:04] otherwise I can't make dump header generate them but entity omit them [19:02:58] wikimedia/mediawiki-extensions-Wikibase/master/0296665 : jenkins-bot The build passed. http://travis-ci.org/wikimedia/mediawiki-extensions-Wikibase/builds/56134020 [19:03:00] (03PS6) 10Smalyshev: Refactor prefixes to be output only once, eliminate need to clean prefixes on dump [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199543 [19:03:44] That's the most straight forward way that comes to my mind right now [19:03:44] * way to go on from here [19:03:46] (03CR) 10Daniel Kinzler: "That's not quite correct." (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [19:04:43] (03CR) 10jenkins-bot: [V: 04-1] Refactor prefixes to be output only once, eliminate need to clean prefixes on dump [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199543 (owner: 10Smalyshev) [19:05:12] SMalyshev: you are suing the same rdf writer fro all entities, right? [19:05:18] *using [19:05:23] (03CR) 10Hoo man: "re" (033 comments) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200209 (owner: 10Hoo man) [19:05:45] DanielK_WMDE_: not really... dump header uses separate writer [19:05:53] but maybe I can avoid this now? [19:06:02] and make dump header use the same writer [19:06:26] SMalyshev: probably. hm, well. i see the problem. the prefixes get written to the top of the document every time you drain(). [19:06:33] we do that for every entity. [19:06:56] yeah... [19:07:03] SMalyshev: the "real" solution would be to tell the writer that it's not actually a full document writer. give it the SUBWRITER role. [19:07:09] then it shouldn't output prefixes [19:07:50] SMalyshev: we should do this anyway, since the writer *is* only writing a snippet. [19:08:03] well, we could tell the writer that maybe when doing addEntity or something [19:08:07] we'd get broken output for XML otherwise [19:08:46] yeah that's what RDF fix did. however, how we do produce the prefixes then? [19:08:52] no, the writer that goes in simply shouldn't be a full document writer. we'd need something like a "detached sub-writer". [19:08:57] hm. [19:09:02] i.e. making addEntity not dump the prefixes is easy [19:09:22] making something then dump them is harder... something has to tell it to do it [19:09:48] SMalyshev: writers that output a full document should output prefixes at the top. [19:09:56] writers that do not output full documents should not. [19:10:02] 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 5Patch-For-Review, 7Performance, and 2 others: Implement an EntityPrefetcher that would take a list of entity ids and preload revision meta data about them - https://phabricator.wikimedia.org/T87238#1157752 (10hoo) We had a discussion about what to do... [19:11:11] SMalyshev: so it should be based on the writer's role. but we currently don't have a way to construct a sub-writer that isn't "attached" to a top level writer. [19:11:33] DanielK_WMDE_: the problem is buildGraphForEntityRevision doesn't know if it's outputting full doc or not. So I guess we need a param for that [19:12:18] 10Wikidata, 10Citoid, 6Editing, 10Possible-Tech-Projects, and 2 others: Create a system to store and query links to books - https://phabricator.wikimedia.org/T90852#1157757 (10Mvolz) @CeruleanDreamer, this is likely not something that wmf would support as an outreachy project as we already have different p... [19:12:18] SMalyshev: i don't want to make the prefix stuff explicit. it doesn't play well with non-turtle output, and doesn't fit with the rest of the interface. i'd like to come up with a smart solution, but I have to go through my review backlog first. can't promise i'll get around to it today. [19:12:32] and buildGraphForEntityRevision is the part everybody is talking to. Also buildGraphForEntityRevision+dumpHeader kind of assume data is sequential so I have no idea if it even can generate valid XML dump... [19:12:58] no it can't [19:13:00] Another thing that just came to my mind is that we need to make sure that the meta data cache doesn't pile up [19:13:01] that's my point [19:13:11] DanielK_WMDE_: I'll try to see if we can fix that... [19:13:14] we have to fix this in a better way in order to supprot multi-entity xml [19:13:20] If we load 20M entities, we don't want the meta data for all of them in a php array [19:13:24] may require a bit deeper change than I thought [19:14:08] SMalyshev: i feel that the change should actually be much smaller, since the interface can stay the same. it would be a bit more involved in the internals of the writer, though [19:14:10] for XML we produce but i think it's possible simply by setting the writer's role appropriately [19:14:45] SMalyshev: in sub-writer mode, we don't produce that (i hope) [19:15:23] DanielK_WMDE_: it's kind of hacky. Subwriter never calls beginDocument, because it is manually hacked into skipping first state transition [19:15:41] (03CR) 10Daniel Kinzler: [C: 032] T93451: ontology fix - ontology predicates go in lowercase [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200117 (owner: 10Smalyshev) [19:15:42] $writer->state = self::STATE_DOCUMENT; [19:15:58] but that's quite ugly [19:16:01] SMalyshev: if you call sub(), yes. that's wouldn't work [19:16:12] DanielK_WMDE_: exactly. [19:16:18] SMalyshev: but you can just construct a writer with the sub-role directly [19:16:24] the factory could have a flag for that [19:16:59] sub() creates an "attached" subwriter. we need a detached "snippet" writer. but the sub-role should work for that [19:17:18] (03Merged) 10jenkins-bot: T93451: ontology fix - ontology predicates go in lowercase [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200117 (owner: 10Smalyshev) [19:18:09] SMalyshev: RdfWriterFactor::getWriter( $format, $role = RdfWriter::DOCUMENT_ROLE ) [19:18:22] DanielK_WMDE_: so you say EntityData... and dump would construct emitters with different options? [19:18:28] ah, the roles are currently internal. we'd be exposing them, they'd need to move to the interface. [19:18:29] let me try that [19:18:31] but i think that's fine [19:18:42] yea, exactly [19:18:45] we can make it an option, doesn't have to be role directly [19:19:03] RdfWriterFactor::getWriter( $format, $fullDocument = true) [19:19:16] boolean arguments suck for readability. a constant is much nicer. [19:19:32] that means exposing roles, not sure if that'd good [19:19:32] or just have a second method, newSnippetWriter [19:19:53] well, that too. You can always make boolean param into two functions :) [19:19:59] i'm fine with either, but please avoid boolean parameters. [19:20:10] who knows what newWriter( 'turtle', true ) means? [19:20:13] true what?! [19:21:08] yeah right now rdf dump is totally broken... I'll try to unbreak it also [19:21:45] (03CR) 10Daniel Kinzler: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199969 (https://phabricator.wikimedia.org/T93885) (owner: 10Hoo man) [19:22:18] DanielK_WMDE_: the problem though is sharing prefixes. I.e. sub-writer should know about main doc writer to be able to share prefixes if we ever define them inline [19:22:43] 10Wikidata, 10Continuous-Integration: Switch or add additional jenkins jobs to run Wikibase tests with mysql - https://phabricator.wikimedia.org/T94208#1157786 (10JanZerebecki) Both please. [19:22:48] SMalyshev: they do, if you use sub() to create them. that's what i mean by "attached". [19:22:53] DanielK_WMDE_: either that or they have separate prefixes and we rely on the fact that they always are the same in fact [19:23:03] DanielK_WMDE_: right. but snippet one won't be attached [19:23:13] and won't write prefixes [19:23:26] the question is then, how do you generate a full document, without buffering everything... [19:23:32] so if you ever do $snippet->prefix() that'd be broken since snippet doesn't write header [19:23:45] and header writer doesn't know that you did snippet->prefix() [19:23:48] no, it's not broken. it just wouldn't do anything [19:23:56] it would sill use the prefix for n-triples, etc [19:24:12] it's broken in the meaning you can't do that since it would generate broken rdf/ttl output [19:24:28] so we have to ban prefix() on snippets [19:25:04] i don't see the problem there. i see a pretty big problem with having a header writer. [19:25:08] that doesn't work for xml at all. [19:25:27] which we can't do since RdfBuilder does prefix() and it has no idea if it's a snippet... so we have to just rely on the fact that we never do snippet->prefix [19:25:48] DanielK_WMDE_: once again. We have the full doc writer, $fulldoc, and snippet writer, $snippet [19:26:06] $fulldoc will produce nice header with prefixes, $snippet won't [19:26:20] what happens if you do $snippet->prefix('foo', 'bar')? [19:26:21] $fulldoc will also be needed to produce [19:26:35] SMalyshev: you decalre a prefix. nothign else. [19:26:41] this prefix would never be produced by $fulldoc, but $snippet will be using it [19:26:46] result: broken data [19:26:59] only if fulldoc doesn't also produce the prefix [19:27:02] since prefix is used but never defined in the resulting doc [19:27:08] (03CR) 10Hoo man: "Adrian said that this has side effects, so we probably shouldn't backport it, unless there's also a follow-up fix." [extensions/Wikibase] (wmf/1.25wmf22) - 10https://gerrit.wikimedia.org/r/199971 (owner: 10Hoo man) [19:27:17] DanielK_WMDE_: fulldoc has no idea snippet added prefix [19:27:33] since snippet is not attached to fulldoc [19:28:11] SMalyshev: my idea way to just do all the prefix() calls on $fulldoc. And it doesn't matter if you do them again on $snippet or not. [19:28:21] but anyway, all this doesn't dolve the issue for XML at all [19:28:27] DanielK_WMDE_: RdfBuilder only has one emitter [19:28:37] so, what we want is not a compeltely deteched sub-writer. [19:28:46] right... [19:29:01] we want a sub-writer that's not bound to the buffer, and we are responsible for calling drain() and outputting the string [19:29:08] but we want it to be attached to the prefix map. [19:30:11] SMalyshev: could be a param in sub(), or a second method, i.e. have sub() for unattached and inline() for attached, or something. [19:30:21] but all this doesn't address the main problem [19:30:58] I think RdfWriter should have some concept of where the header is... [19:31:06] the main problem is that we need $fulldoc to generate a header and a footer, without buffering any of the output in between, so it can be used with streaming [19:31:13] the current interface doesn't support this at all. [19:31:13] because the only thing that is shared is the writer [19:31:24] right [19:33:13] so maybe we should have start() and finish() along with drain() and use them to generate header/footer [19:33:29] yes, i was thinking about that [19:33:34] instead of calling start() for each subdoc [19:33:43] or drainHeader() to output the header without resetting the state [19:34:24] would could make drain() work at any point in time; it would just drain the buffer, and not mess with the state [19:34:37] if you want to finish the document, call finish() [19:34:46] (that would probably also drain) [19:36:25] hmmm. I wonder where we call start() at all? I don't see any place [19:36:36] except for tests [19:36:56] SMalyshev: ok, i think i got it now. start() should only be needed/allowed for full doc writers. drain() should flush the buffer, but not close the document (would close the current subject, though). finish() would close the document, and drain. [19:37:29] SMalyshev: start() sais "put the header on the buffer". May be a clabback, but it'S "virtually" on the buffer then. [19:37:35] i don't want the constructor to do that [19:37:44] but it should not be needed for sub-writers [19:38:08] so who will be calling start()? The RdfSerializer? [19:38:09] haha, s/clabback/callback/ :) [19:38:35] whatever code controlls the full document. [19:38:46] Could be RdfSerializer, though that class can probably just go away [19:38:54] it doesn't ereally do anythign anymore [19:39:02] that's tricky because that code only knows RdfSerializer but RdfSerializer only has buildGraphForEntityRevision [19:39:04] oh dear, i can't type any more [19:39:51] SMalyshev: start() is called by whatever controlls the loop that goes over all the entities. [19:40:09] which means that whatever does that needs to know the top level writer [19:40:19] DanielK_WMDE_: the problem is that loop only knows about RdfSerializer but not below [19:40:27] i think RdfSerializer would need to be re-engineered for this [19:40:38] so we'd probably need to tweak it... [19:41:10] yes. RdfSerializer could be maintaining the state, and have start() and finish() functions, for example [19:41:13] Maybe RdfSerializer::startDocument, RdfSerializer::buildGraphForEntityRevision and RdfSerializer::endDocument [19:41:18] exactly [19:41:26] ok, I will try with that [19:41:47] see what comes out [19:41:52] SMalyshev: we seem to think along the same lines, when our thought are not completely opposite for once ;) [19:42:00] hehe [19:43:39] (03CR) 10Daniel Kinzler: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199588 (owner: 10Thiemo Mättig (WMDE)) [19:44:31] (03CR) 10Daniel Kinzler: [C: 032] Split individual tests in UsageAccumulatorContractTester [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199588 (owner: 10Thiemo Mättig (WMDE)) [19:48:22] even if there are opposite they can be on the same line... ;) [19:49:04] (03Merged) 10jenkins-bot: Split individual tests in UsageAccumulatorContractTester [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199588 (owner: 10Thiemo Mättig (WMDE)) [19:49:32] DanielK_WMDE_: ok, there's another problem... if we want to make RDF/XML dump, we have to output RDF header first [19:49:33] dennyvrandecic: that would be a head on collision :) [19:49:57] which means if we call prefix() later it's too late - we already wrote the RDF tag [19:50:38] SMalyshev: yes. $writer->start(); $writer->prefix...; print $writer->drain(); ...use sub-writers to generate output for entities; print $writer->finish(); [19:51:20] SMalyshev: for XML, start() just put a callback on the buffer. Look at XmlRdfBuffer::beginDocument() [19:51:20] DanielK_WMDE_: right. but when we call writer->start() etc. it should produce tag immediately. which will contain list of prefixes [19:51:37] DanielK_WMDE_: we can't buffer dumps. Dump is 30G of text [19:51:40] no, why immediately? [19:51:52] nothign needs to *really* be produced until you call drain() [19:51:53] DanielK_WMDE_: see above. We have to write stuff to disk [19:52:11] yes. that's why you call drain() after registering all the prefixes. [19:52:11] DanielK_WMDE_: we have to write to disk once we're done with each entity. [19:52:20] yes. no problem [19:52:32] aha! so that means you can not define prefixes in-flight [19:52:58] SMalyshev: yes. you can still call prefix(), but the ones you define later would not go into the header [19:52:59] and if you can not, then why bother with all this infrastructure that pretends like you can? [19:53:18] because that works nicely with simpler/smaller cases than ours [19:53:33] and it's really not a lot of infrastructure [19:53:39] DanielK_WMDE_: but that'd be just wrong usage. I mean if you call prefix() after entity output started, it's a bug [19:53:41] as i said, for xml, it's already there, and pretty simple [19:53:47] since it generates broken data [19:54:14] only if you call it with a prefix that has not yet been defined [19:54:16] so why we don't just have fixed set of prefixes and output them once and don't have prefix() at all? [19:54:19] we could actually track&check this [19:54:32] but we omit a lot of sanity checking, so why be strict about this? [19:54:39] i.e. you have something like start($prefixList) and that's it [19:55:06] because i want this interface to be nicely usable for other stuff [19:55:15] for our use case, what you propose would be fine, yes [19:55:16] having prefix() just sets us up for failure - if it's there sooner or later somebody would call it in the middle [19:55:46] that's the thing - no other stuff can be valid with calling prefix() mid-flight since header may already have been committed [19:56:03] SMalyshev: sooner or later someone will use an invalid lname or prefix that will kill the syntax, because we never check :) [19:56:06] once you're past startDocument, you can not call prefix() ever again [19:56:37] i'd say, once you called drain() the first time, you can't use prefix again to define a new prefix. [19:56:41] that's not the point. prefix() is the interface we create. so what I say we don't actually need it- we should just pass prefixes to start() [19:56:49] that's a lot more relaxed, but would still prevent the bad cases. [19:57:13] that would require a lot of internal state tracking... I'd rather go with fixed list. [19:57:27] SMalyshev: you are assuming that a) prefixes are always known in advance and b) don't change over the course of the output. both cannot be guaranteed. [19:57:39] well, for writing wikibase dumps, this is the case [19:57:45] DanielK_WMDE_: if that can not be guaranteed we can not generate XML dump [19:57:53] because that's exactly what XML requires [19:58:19] that's what rdf:RDF tag says [19:58:21] yes we can. it works. it's already tehre and tested. [19:58:32] we can't do it without buffering. so it's not possible for BIG files [19:58:34] except that it is completely broken :) [19:58:42] but otherwise, it works quite nicely, and i'd like to keep that option [19:59:12] Nemo_bis: hey [19:59:22] that would require another state dimension - remembering if we ever called drain or not. and of course tracking it through all sub-writers [19:59:40] SMalyshev: i could have implemented this in the time we spent aarguing about it :) except that i have to go through the review backlog first [20:00:01] yes, correct, it woudl require that [20:00:16] so that's actually 2 dimensions since for sub-writer it's ok to call prefix after drain - but only before parent writer called drain, after which it is not ok anymore [20:00:19] the rough&tough version ofthat would be to set $prefixes = false :) [20:00:23] which turns into a hell of a mess [20:00:48] no, no prefix() after drain, for any writer, ever. [20:00:55] DanielK_WMDE_: nope, since with sub-writers it's not that simple - you'd also to have to go through all sub-writer chain and tell them [20:00:58] not even after drain() was called on a sub-writer. [20:01:02] DanielK_WMDE_: but for sub-writers it's fine [20:01:07] no we don't [20:01:15] Amir1: hi [20:01:17] the sub-writers hold a reference to $prefixes [20:01:29] if you change $prefixes, they all know [20:01:41] Nemo_bis: I'm free now [20:01:53] How I can help :) [20:01:56] Amir1: Thanks. All well? [20:01:59] *Can I [20:02:09] Nemo_bis: yeah, pretty much [20:02:14] and you [20:02:30] Amir1: I'm writing a script to merge data from a CSV into Wikidata. But how to find existing items to which some statements can just be added, without creating a new item? [20:02:32] hm... It still has too much state for my liking... [20:02:35] Amir1: fine [20:03:11] I would say go for item._contents [20:03:14] I don't see much point in dynamic prefix() given that since you can't know if drain was ever called on this emitter, in practice you would never use prefix() at all [20:03:19] Looking [20:03:27] or item._content [20:03:32] except in special init routine [20:03:34] let me check [20:04:02] SMalyshev: yea, i understand that. we can settle for only allowing prefix() calls before start() for now. [20:04:31] ok, I'll try with that [20:04:32] that would keep the interface as it is, and I can relax the restrictions again later, without breakign anything [20:06:05] Amir1: from what I see, _content can be checked to see if the ItemPage was just created or fetched data from the repo, but in both cases it's initialised with a Q#? [20:06:19] SMalyshev: are you ok with the other key points? start() only on the main doc, drain() can be called at any time (does a x-> document transition), and finish() closes the top level document (transition ->drain - though that state should then be renamed to END). [20:06:39] I can show you better in examples [20:06:48] SMalyshev: i'llwrite this down in a phab ticket, for reference [20:06:49] Ah there is fromPage [20:06:56] Nemo_bis: what do you have as input [20:07:00] yes... I'm not sure about additional state yet but I'' try and see how it goes [20:07:12] yes, use fromPage and then fetch the data [20:07:13] I need to write some code to see if I understand it or not [20:10:21] Amir1: but I don't know what to do if I only know the name, e.g. the name of the person, and want to add some data to it [20:10:46] hmm [20:10:47] DanielK_WMDE_: would it be OK not to reset prefixes on reset() then? [20:11:01] Nemo_bis: I usually confirm the person through Wikipedia [20:11:04] reset() seems to be still useful to call between entities [20:11:11] I check if the page exists in Wikipedia [20:11:36] (03PS2) 10Aude: Add clarification about subscriptions table deployment [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 [20:11:39] Amir1: these persons were already added to Wikidata but may not exist on Wikipedia, or only in a language I don't know [20:12:45] the issue is afaik there is no way to get Q# of an item based on label [20:13:04] you can't search for English label of someone and get the Q# [20:13:18] it's a flaw in Wikidata's API IMO [20:13:33] (03CR) 10Aude: Add clarification about subscriptions table deployment (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [20:13:40] I see. [20:13:54] (03CR) 10jenkins-bot: [V: 04-1] Add clarification about subscriptions table deployment [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [20:14:03] SMalyshev: depends on the role, i'd say. resetting a sub-writer should not reset the parent's prefixes. calling reset on a top level writer should reset prefixes. [20:14:05] Ok, so that's the state of the art. Good to know! [20:15:22] Nemo_bis: sorry I can't be very helpful [20:15:24] 10Wikidata: Enable header/footer generation with RdfWriter without buffering full output. - https://phabricator.wikimedia.org/T94224#1157901 (10daniel) 3NEW [20:15:35] SMalyshev: https://phabricator.wikimedia.org/T94224 [20:15:57] DanielK_WMDE_: the problem is, after reset you still can't add prefixes so you've got an emitter that has no prefixes. Unless reset resets everything, and then we need another reset than resets only part of it. And then why just not create a new emitter instead of the first reset? [20:16:11] (03CR) 10JanZerebecki: "That failure was me changing that Jenkins job." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [20:16:15] we need soft-reset for between entities [20:16:16] 10Wikidata: Enable header/footer generation with RdfWriter without buffering full output. - https://phabricator.wikimedia.org/T94224#1157925 (10daniel) [20:16:20] (I think) [20:17:00] SMalyshev: the soft reset would be calling reset() on a sub-writer. full reset would be calling reset() on the "master" writer. [20:17:04] wouldn't that work? [20:18:08] DanielK_WMDE_: kind of yeah but I don't like same reset() doing so different things... [20:18:24] but maybe it'll work [20:19:03] SMalyshev: it resets *that* writer, in that writer's context. it's doing the same thing. that thing is just different, depending on context. [20:19:36] I'm not sure then we need headerWriter anymore [20:19:38] Nemo_bis: Amir1: you can search by label [20:19:58] SMalyshev: i'm pretty sure we don't, and i'm happy about it [20:20:37] Nemo_bis: Amir1: the entitysuggester is using an API to search by label [20:20:44] (03CR) 10Daniel Kinzler: [C: 031] "The intention is fine, but i don't know how the "expensive function" stuff works. would need to play with it..." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199969 (https://phabricator.wikimedia.org/T93885) (owner: 10Hoo man) [20:20:47] legoktm: :( i think i broke https://integration.wikimedia.org/ci/job/mwext-Wikibase-repo-api-tests/ when applying https://gerrit.wikimedia.org/r/#/c/200187/ without your last change [20:21:34] Purtle ? :P [20:21:34] but i don't understand [20:21:40] dennyvrandecic: I checked API manual but I couldn't find anything. do you know what is the action in API [20:21:41] ? [20:21:46] addshore: \o/ [20:22:16] addshore: a turtle style rdf generator for php. purtle. the mascot will be a purple turtle, of course [20:22:26] :D [20:22:40] (03CR) 10JanZerebecki: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 (https://phabricator.wikimedia.org/T94172) (owner: 10Daniel Kinzler) [20:22:54] Amir1: wbsearchentities [20:23:07] jzerebecki: umm, that's weird [20:23:08] * addshore is fairly out of the loop :( [20:23:19] dennyvrandecic: thanks [20:23:39] legoktm: very, next recheck works [20:23:44] Nemo_bis: so It should be implemented in pywikibot [20:23:45] jzerebecki: oh it kinda makes sense. the old job config would have grabbed vendor some other way so it's no longer a .git repo or whatever. Just needs a rm -rf on the workspace [20:23:50] jzerebecki: is it on the same slave? [20:24:05] it's not there yet but you can send the api request and use it [20:24:09] legoktm: hey :) [20:24:11] legoktm: ah no. that is the problem. [20:24:17] happy to see you around [20:24:19] it is on a different one [20:24:33] legoktm: failures were on integration-slave1004 [20:32:58] legoktm: would cleanup myself, but lacking permission. can you fix it? [20:33:25] sure [20:33:29] Amir1: hey :D [20:33:52] :) [20:34:51] (03CR) 10Daniel Kinzler: [C: 032] "not happy about the name of "DispatchingValueParser", but hey, who cares about names ;)" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197093 (owner: 10Thiemo Mättig (WMDE)) [20:34:59] (03CR) 10Daniel Kinzler: Split MonthNameUnlocalizer into Base and MediaWiki part (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197093 (owner: 10Thiemo Mättig (WMDE)) [20:35:14] grr, labs is lagging [20:36:47] (03CR) 10Daniel Kinzler: [C: 032] Make SpecialSetLabelDescriptionAliases a 2-step process [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/194105 (https://phabricator.wikimedia.org/T91387) (owner: 10Thiemo Mättig (WMDE)) [20:37:48] (03CR) 10Daniel Kinzler: [C: 04-1] "See https://phabricator.wikimedia.org/T94224 for notes on what is needed to address this issue in a more general way." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199543 (owner: 10Smalyshev) [20:38:08] jzerebecki: rm -rf'd. Do you know if any other hosts or jobs were affected? [20:39:06] legoktm: no others. [20:40:10] jzerebecki: ok. if you already deployed your change, should I +2 it? [20:40:23] legoktm: yes please [20:40:36] (03CR) 10Daniel Kinzler: [C: 032] Split and refactor large SpecialPageTestBase method [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199913 (https://phabricator.wikimedia.org/T92375) (owner: 10Thiemo Mättig (WMDE)) [20:40:46] (03CR) 10JanZerebecki: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/194105 (https://phabricator.wikimedia.org/T91387) (owner: 10Thiemo Mättig (WMDE)) [20:41:06] (03CR) 10JanZerebecki: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [20:41:26] (03CR) 10Daniel Kinzler: [C: 032] Cleanup bits in SpecialSetLabelDescriptionAliasesTest [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199260 (owner: 10Thiemo Mättig (WMDE)) [20:42:01] (03CR) 10jenkins-bot: [V: 04-1] Split MonthNameUnlocalizer into Base and MediaWiki part [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197093 (owner: 10Thiemo Mättig (WMDE)) [20:42:03] (03CR) 10jenkins-bot: [V: 04-1] Make SpecialSetLabelDescriptionAliases a 2-step process [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/194105 (https://phabricator.wikimedia.org/T91387) (owner: 10Thiemo Mättig (WMDE)) [20:42:26] (03CR) 10JanZerebecki: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197093 (owner: 10Thiemo Mättig (WMDE)) [20:43:02] legoktm: it is still failing on that slave https://integration.wikimedia.org/ci/job/mwext-Wikibase-repo-api-tests/6563/console [20:43:27] (03PS7) 10Daniel Kinzler: Drop eu_entity_type from wbc_entity_usage table. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197933 [20:43:40] (03PS4) 10Daniel Kinzler: Update documentation for usage tracking. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197942 [20:43:55] legoktm: ignore please mixed up jobs [20:43:59] ok :) [20:44:23] s/jobs/runs/ [20:44:52] jzerebecki: woudl you have a look at the schema changes for usage tracking? the chain ends with https://gerrit.wikimedia.org/r/#/c/197942/4 [20:45:03] you can see the changes nicely in the diff of the updated docs :) [20:45:27] DanielK_WMDE_: first cleaning up the jenkins jobs i made to fail... [20:45:29] jzerebecki: I'm about to go afk, if anything else is broken poke hashar :) [20:45:50] legoktm: k thx. cu [20:45:54] jzerebecki: heh, have fun... was that because of the change I asked you to do to jenkins? [20:46:30] DanielK_WMDE_: yes i accidentally unapplied an earlier change when deploying it [20:46:41] eek [20:47:10] (03Merged) 10jenkins-bot: Split and refactor large SpecialPageTestBase method [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199913 (https://phabricator.wikimedia.org/T92375) (owner: 10Thiemo Mättig (WMDE)) [20:47:26] (03CR) 10JanZerebecki: [C: 032] "reapplying +2 to trigger zuul" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197093 (owner: 10Thiemo Mättig (WMDE)) [20:48:11] (03Merged) 10jenkins-bot: Cleanup bits in SpecialSetLabelDescriptionAliasesTest [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199260 (owner: 10Thiemo Mättig (WMDE)) [20:48:15] (03CR) 10JanZerebecki: [C: 032] "reapplying +2 to trigger zuul" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/194105 (https://phabricator.wikimedia.org/T91387) (owner: 10Thiemo Mättig (WMDE)) [20:48:57] 10Wikidata, 10Citoid, 6Editing, 10Possible-Tech-Projects, and 2 others: Create a system to store and query links to books - https://phabricator.wikimedia.org/T90852#1158134 (10CeruleanDreamer) Thank you for responding.I will check into the links that you have shared. [20:49:01] (03Merged) 10jenkins-bot: Split MonthNameUnlocalizer into Base and MediaWiki part [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197093 (owner: 10Thiemo Mättig (WMDE)) [20:52:57] (03PS7) 10Smalyshev: Refactor prefixes to be output only once, eliminate need to clean prefixes on dump [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199543 [20:53:16] (03Merged) 10jenkins-bot: Make SpecialSetLabelDescriptionAliases a 2-step process [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/194105 (https://phabricator.wikimedia.org/T91387) (owner: 10Thiemo Mättig (WMDE)) [20:54:40] (03CR) 10jenkins-bot: [V: 04-1] Refactor prefixes to be output only once, eliminate need to clean prefixes on dump [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199543 (owner: 10Smalyshev) [21:01:05] (03PS9) 10Smalyshev: Refactor prefixes to be output only once, eliminate need to clean prefixes on dump [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199543 [21:03:24] (03PS10) 10Smalyshev: T94224: Refactor prefixes to be output only once, eliminate need to clean prefixes on dump [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/199543 [21:06:12] 10Wikidata, 5Patch-For-Review: Enable header/footer generation with RdfWriter without buffering full output. - https://phabricator.wikimedia.org/T94224#1158211 (10Smalyshev) p:5Triage>3Normal a:3Smalyshev [21:09:15] 10Wikidata, 5Patch-For-Review: Enable header/footer generation with RdfWriter without buffering full output. - https://phabricator.wikimedia.org/T94224#1158223 (10Smalyshev) Not sure separate sub-writer is even needed. See the patch - it looks like just adding FINISH state is enough. But we could tweak it furt... [21:12:13] (03PS6) 10Smalyshev: T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 [21:12:21] (03CR) 10jenkins-bot: [V: 04-1] T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 (owner: 10Smalyshev) [21:13:32] (03PS7) 10Smalyshev: T93451: ontology fixes - namespace the value predicates [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200119 [21:13:52] (03Abandoned) 10Smalyshev: refactoring for better performance [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/194242 (owner: 10Smalyshev) [21:43:49] (03CR) 10JanZerebecki: [C: 031] "Will merge when I get the last in the series." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197504 (https://phabricator.wikimedia.org/T92288) (owner: 10Daniel Kinzler) [21:44:18] (03CR) 10JanZerebecki: [C: 031] "Will merge when I get to the last in the series." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197940 (https://phabricator.wikimedia.org/T93057) (owner: 10Daniel Kinzler) [21:46:25] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph - https://phabricator.wikimedia.org/T94064#1158400 (10Smalyshev) >The lexical representation where the year fraction is 0 has undefined meaning in Wikib... [21:48:53] (03CR) 10JanZerebecki: [C: 04-1] "See inline." (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197933 (owner: 10Daniel Kinzler) [21:49:27] (03CR) 10JanZerebecki: [C: 031] Update documentation for usage tracking. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197942 (owner: 10Daniel Kinzler) [21:53:31] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph - https://phabricator.wikimedia.org/T94064#1158423 (10mkroetzsch) Yes, the discussion on SPARQL has converged surprisingly quickly to the view that XSD... [21:57:19] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph - https://phabricator.wikimedia.org/T94064#1158459 (10mkroetzsch) > "we don't know what year it was but it was July 4th" Ouch. Where has this been des... [21:58:28] (03CR) 10JanZerebecki: [C: 032] Add clarification about subscriptions table deployment [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [21:59:49] (03PS10) 10Smalyshev: T92996: Add Julian Calendar support [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198419 [22:04:28] (03CR) 10Aude: "@JanZerebecki it is okay to have this deployed and then do the schema change. the code will just stop writing to the column and not care t" (032 comments) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197933 (owner: 10Daniel Kinzler) [22:04:58] 10Wikidata, 10Wikidata-Query-Service, 5Patch-For-Review: Data format updates for RDF export - https://phabricator.wikimedia.org/T93451#1158493 (10mkroetzsch) All RDF tools should be able to handle resources without labels (no matter if used as subject, predicate, or objcet). But data browsers or other UIs wi... [22:06:59] (03CR) 10Aude: "and note, it is not always okay to handle schema change this way, but these are ok." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/197933 (owner: 10Daniel Kinzler) [22:08:07] (03CR) 10jenkins-bot: [V: 04-1] Add clarification about subscriptions table deployment [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [22:17:11] (03PS1) 10Hoo man: Move "onSpecialMovepageAfterMove" out of ClientHooks [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200245 [22:19:20] (03CR) 10jenkins-bot: [V: 04-1] Move "onSpecialMovepageAfterMove" out of ClientHooks [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200245 (owner: 10Hoo man) [22:20:35] "22:17:53 ERROR:zuul.Repo:Unable to initialize repo for https://gerrit.wikimedia.org/r/p/mediawiki/vendor" :S [22:20:47] (03CR) 10Hoo man: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200245 (owner: 10Hoo man) [22:21:45] (03PS1) 10Hoo man: Move Scribunto under client/includes/DataAccess [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200246 [22:21:47] (03PS1) 10Hoo man: Move Scribunto classes to Wikibase\Client\DataAccess\Scribunto [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200247 [22:25:00] (03CR) 10Hoo man: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200245 (owner: 10Hoo man) [22:36:50] (03CR) 10JanZerebecki: [C: 032] "Retrying after Jenkins slave problem." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [22:39:26] hoo: my fault. fixed [22:39:43] :) [22:46:02] (03Merged) 10jenkins-bot: Add clarification about subscriptions table deployment [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200154 (owner: 10Aude) [22:48:40] (03PS11) 10Smalyshev: T92996: Add Julian Calendar support [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/198419 [22:49:31] Suddenly got this error: Fatal error: Class 'Wikibase\Lib\Parsers\TimeParser' not found in /vagrant/mediawiki/extensions/WikidataBuildResources/extensions/Wikibase/repo/includes/ValueParserFactory.php on line 113 [22:49:45] can't find this class anywhere - anybody knows where it is supposed to be? [22:55:17] SMalyshev: Mh... could ahve been moved/ renamed recently [22:55:22] try composer update [22:55:35] hoo: tried so. no such class found anywhere [22:55:57] it's configured as $wgValueParsers['time'] = 'Wikibase\Lib\Parsers\TimeParser'; [22:56:10] but I can't find any trace of such class existing [22:56:53] The class has been removed in f8a57f9f8f5dd59446b1b6e1b3b70c09f7fba1cc [22:59:16] Looking at that patch, I don't think we can just update $wgValueParsers['time'] = 'Wikibase\Lib\Parsers\TimeParser'; [23:01:07] SMalyshev: Nevermind, I can come up with a fix, I think [23:01:25] hoo: thanks. Shouldn't there be tests somewhere to catch such thing? [23:01:43] removing class that is in base config is kind of problematic :) [23:06:54] this seems to work: [23:06:55] $wgValueParsers['time'] = function($options) { [23:06:56] $factory = new Wikibase\Lib\Parsers\TimeParserFactory(); [23:06:56] return $factory->getTimeParser(); [23:06:56] }; [23:06:56] [23:07:01] not sure if correct though [23:08:14] (03PS1) 10Hoo man: Fix repo $wgValueParsers['time'] [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200253 [23:08:32] aude: DanielK_WMDE_: jzerebecki ^ [23:08:40] 10Wikidata, 10Continuous-Integration, 6Release-Engineering, 7Browser-Tests, 5Patch-For-Review: browsertest jobs should not be allowed to run for 10 hours - https://phabricator.wikimedia.org/T92275#1158771 (10Tobi_WMDE_SW) @hashar & @zeljkofilipin great thanks to you for fixing this! [23:08:43] SMalyshev: Yeah, I came up with almost hte same [23:08:54] except that your version losses the language passed in as option [23:08:55] (03CR) 10Smalyshev: [C: 031] "That fixes the problem for me." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200253 (owner: 10Hoo man) [23:09:41] hoo: yeah it was just quick n dirty patch, yours is correct ofc [23:10:54] (03CR) 10JanZerebecki: [C: 032] Fix repo $wgValueParsers['time'] [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200253 (owner: 10Hoo man) [23:12:32] Mh... I really wonder how we did't sport with tests... [23:12:36] * spot [23:13:00] yeah that worries me too. It is immediately apparent in the gui - date parsing just stops working [23:13:48] (03Merged) 10jenkins-bot: Fix repo $wgValueParsers['time'] [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200253 (owner: 10Hoo man) [23:16:45] 10Wikidata, 10Wikidata-Query-Service: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph - https://phabricator.wikimedia.org/T94064#1158816 (10Smalyshev) @mkroetzsch I don't think it is specified anywhere, it's just what people do. See https... [23:17:38] 10Wikidata, 10WikiGrok, 3Roadmap, 7notice: Test of WikiGrok --> Wikidata pipeline - https://phabricator.wikimedia.org/T91840#1158830 (10Liuxinyu970226) [23:40:01] (03PS1) 10Smalyshev: T94115: add property type to dump [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200262 [23:42:10] (03CR) 10Smalyshev: Moving RdfWriter and friends to a separate component called Purtle. (031 comment) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/200183 (https://phabricator.wikimedia.org/T94172) (owner: 10Daniel Kinzler)