[08:43:10] !nyan | * [08:43:10] *: ~=[,,_,,]:3 [08:43:43] !nyan | ~=[,,_,,] [08:43:43] ~=[,,_,,]: ~=[,,_,,]:3 [08:44:15] !nyan | ~=[,,_,,]3 [08:44:15] ~=[,,_,,]3: ~=[,,_,,]:3 [09:27:17] lol [10:11:46] Thiemo_WMDE: Want to talk to Jarkt about date formatting? https://www.wikidata.org/wiki/User_talk:Duesentrieb#Internationalization_of_Dates [10:19:30] someone around to give guidance on 1) checking whether a data field exists, and if it exists how to retrieve that data [10:19:52] the modules all seem to a lot more [10:53:58] sDrewth: you mean vis the api? [10:54:01] *via [10:55:06] sDrewth: wikibase doesn't really have "data fields", it has "claims" resp. "statements". wbgetclaims is probably what you want, see http://localhost/daniel/wikidata/api.php?action=help&modules=wbgetclaims [10:55:57] k, DanielK_WMDE__ I have done a ugly hack within https://en.wikisource.org/wiki/Template:Author/sandbox using module:wikidata [10:56:25] sDrewth: oh, you mean via Lua! [10:56:30] https://en.wikisource.org/w/index.php?title=Author:Harriet_Martineau&oldid=5563808 [10:56:57] daniel, I don't care how I get it [10:57:02] With Lua, it's still a bith rough... hoo is the expert, he'll probably join soon. [10:57:26] well, if you need it in wikitext, you can't use the api - or rather, you'd then need a bot to make edits to the page based on the info from the api [10:57:30] Lua is probably better [10:57:52] <- outcome focused :-) [10:57:58] actually, we should somewhere collect ideas of how the Lua interface could be streamlined, to make common tasks easier [10:59:14] sDrewth: I can try to help you for now if you wish [11:00:15] DanielK_WMDE__: I think that a something like 1) check if value exists at WD, and use it if no parameter or empty template parameter; 2) categorise where the template has a local value AND a value exists at WD [11:00:55] that allows for local override, and allows for a tidy up if a community wishes [11:01:35] matej_suchanek: thanks, I am trying to insert the image property into enWS's author template if it exists at WD [11:02:23] if you have a Lua module thaat can retrieve data from Wikidata, that's good start [11:03:10] you may create a new function that both retrieves the data and checks if the provided parameter is not empty [11:03:11] I have poked Module:Wikidata, but it isn't quite there by my use [11:03:44] if it isn't empty, return it, else return the retrieved data [11:04:11] and if there is some data and the parameter is not empty, you can also return the category [11:04:26] yes, but it as it is an image it needs the {file:...]] components [11:04:39] [[file: ]] [11:04:39] 10[1] 04https://www.wikidata.org/wiki/file: [11:05:04] ok [11:05:23] and that doesn't behave as well [11:05:30] so you should check within the module if the datavalue is an image and create a formatter for that [11:06:06] oh, I will know that it is a image, that is what I am wanting [11:06:32] yes, mainsnak table includes datatype and datavalue [11:06:47] datatype can be "commonsMedia", that's what you are looking for? [11:06:55] I am just trying to figure out the components to do the right checks in the right order and to pull back the right information (as a not very good hacker) [11:07:14] ' mainsnak table' means nothing to me [11:07:29] ah, so I will explain [11:07:42] and I get confused over the entity bits, etc. (not my KB) [11:12:47] sDrewth: you provided a link above but it seems fixed now... [11:14:27] yes and no, that is a working ugly hack, which was reverted as I was experimenting with a presentation page which I reverted [11:15:09] so I got something from module:wikidata, but it seems ugly [11:16:55] maybe I will try and lasso tpt and/or aude to customise something for the WSes [11:21:17] WDQ down? [11:23:38] sjoerddebruin: works for me [11:23:52] http://tools.wmflabs.org/autolist/autolist1.html?lang=nl&props=21&q=claim%5B31%3A5%5D%20and%20link%5Bnlwiki%5D%20and%20noclaim%5B21%5D won't load for me. [11:24:05] Now it does... [11:24:12] It's like "Hey, this door squeeks" [11:29:10] hmm, script fails for multiple images recorded, and most don't have a preferred rank :-/ [11:52:56] can we make module:wikidata only return one result if there are multiple images all ranked normal? [11:53:10] s/images/results/ [11:57:38] * sDrewth cannot tell his mainsnak from his hors d'ouvres [11:58:18] sDrewth: how should wikidata decide which one to return? [11:58:50] sDrewth: the idea is that they are all equally preferred, all should be shown. If they are not, people should go and mark one as preferred. [11:59:36] Currently, statements often don't get marked as preferred, because ranks arn't used much anywhere, so the need and function isn't obvious.- [11:59:56] DanielK_WMDE__: I understand, and I was hoping for choose higher rangked, or first of normal ranked [12:00:00] If you start using ranks to decide what's shown in an inforbox, it becomes mroelikely for people to actualyl use ranks. [12:00:12] first of normal ranked is not a good thing. [12:00:26] DanielK_WMDE__: most WS users wouldn't know WD if they fell over it [12:00:40] it imposes a meaning on the order which simply isn't maintained by the software, and can't even be controleld by users (we have no UI for changing the order) [12:00:43] for an image, first normal is better than all normal [12:00:51] I strongly disagree. [12:01:11] when you see it automatically pulled back into an image, not really [12:01:46] if you pick the first one, and I think that'S not a good idea, how would i fix the problem? [12:02:12] and if we rely on order, how will it become obvious what preferred means? [12:02:22] https://en.wikisource.org/w/index.php?title=Author:Elizabeth_II&oldid=5563842 [12:02:39] I don't expect you to fix the problem [12:03:01] there is no problem, it is populated with an image, and that is suitable [12:03:19] no, but by the spirit of the wiki, i should be *able* to fix the probelm if i care to [12:03:37] why not show multiple images? [12:03:42] what's the problem with that? [12:03:59] we just want the one, it is bling anyway [12:04:16] which on is "the" one then? [12:04:23] the preferred one? then mark it as preferred. [12:04:44] DanielK_WMDE__: but for an image it can only ever be MY preferred [12:04:52] it is not a birthdate [12:04:53] sDrewth: I do agree that we should make it much easier to display multiple images. this is possible with lua, but still way to ward to do. [12:05:04] that is something we definitly should ijmprove [12:05:41] sDrewth: well, people on wikipedia pick the image for an infobox by ropugh consensus, right? so, the same applies here. better than picking a random image, don't you think? [12:05:53] and "first" really means "random" at the moment, since there is no control over the order [12:07:14] I understand your point, but if we have a preference of many, we would just use the parameter [12:07:54] why not set the "preferred" rank in such a case? [12:08:01] and what is used at enWP may not be what is used at enWS and not the one at deWP [12:08:10] and I have seen many cases of difference [12:08:19] if that is the case, you'd indeed have to override locally. [12:08:34] I have no issue with setting the preference, just know that it won't get done predominantly [12:08:42] but none of this tells my why it is a good idea to only show the first of all the defined images. [12:08:47] people 1) don't see the little grey boses and 2) don't understand them [12:08:55] yes, because they are not used! [12:09:07] which is exactly why i think we should use ranks a lot more for display. [12:09:12] b/c I just want one image, not multiple, I am not building a gallery [12:09:23] how often are there multiple images? [12:09:23] DanielK_WMDE__: I don't disagree [12:09:32] * sDrewth shrugs [12:09:52] you can just use the first one, sure. but it takes away control from editors. [12:10:06] it becomes much harder to understand how to influence which image is shown [12:10:12] actually what would be useful is when people ADD a multiple image they are prompted to rank one as the preferred [12:10:21] yes, I agree [12:10:25] the workflow is very unclear [12:10:31] ranks are hidden in the UI [12:10:45] and the rankings are not exclusive [12:10:50] handling properties with multiple values in infoboxes is still way too hard [12:11:27] not exclusive? you mean there can be more than one preferred image? yes. in which case there is probably a very good reason for people doing this. Which is why we should show all of them. [12:11:52] multiple preferred values often indicate a disagreement, which should be exposed, not hidden [12:11:59] https://www.wikidata.org/wiki/Q9682 [12:12:06] just set two to have preferred [12:13:03] value and rank are both semi-concealed functionality [12:13:17] I wouldn't use the TIME Mag as a preferred image [12:13:33] this is not what you would generally expect when looking for an image of Queen Elizabeth, is it? [12:13:33] an example, not a permanent setting [12:14:40] okay, and there is now (1) preferred image, and module:wikidata still shows multiple [12:16:28] bugger getRawVAlue returns multiples, no matter what [12:16:42] Why doesn't it merge to the lower ID? https://www.wikidata.org/w/index.php?title=Q4320383&action=history [12:16:46] sDrewth: i honestly don't know how Module:Wikidata works. That's somethign hacked up by local admins, not part of Wikibase. [12:17:24] agreed DanielK_WMDE__ and multiple versions all over the place sucks bloated maggots [12:17:56] so fix the data, don't paint over the symptoms! [12:18:15] ??? data is fixed [12:18:27] <- transcribes [12:18:31] <- stewards [12:18:44] !<- code [12:19:06] sDrewth: well, and fix the darn Lua module, if it doesn't handle "best rank" correctly... [12:19:22] yea, "someone" needs to fix that. poke the authors of the module? [12:19:25] my coding days are WAY WAY WAY behind me [12:20:03] I did PASCAL at university [12:20:09] which is a very sad indicator [12:20:33] hehe, I remember the days of TP 6.0... [12:22:06] sDrewth: honestly, I'm not even sure what is currently the "best" way to get "best" ranks (preferred if any exists, normal otherwise). I assumed module:wikidata was handling that correctly. If that is not the case, it needs fixing - and better support from our Lua library. [12:22:31] or enWS is behind the right fix of enWP [12:22:51] and the fact that we all have different versions suck more bloated maggots [12:23:06] interwiki modules! [12:23:09] I suggest to ask Hoo about it when he joins here, and poke the people who wrote the module on-wiki. [12:23:20] yep [12:23:50] Aude and Tpt wrote the first version, and both sort of wikisource, so I need to hunt them down [12:23:54] yes, global modules, better lua lib for wikibase, better ui for ranks, structured meta-data for media files.... [12:23:59] so much to code, so little time :) [13:01:36] * hoo rages about json dumps [13:02:16] :( [13:02:43] The new dump we created overnight is even worse than yesterday's dump [13:02:48] I think I'll delete both [13:03:03] * aude cries [13:03:06] how is that even possible -.- [13:06:22] addshore: aude: [25ae4933] [no req] Wikibase\DataModel\Entity\PropertyNotFoundException from line 50 of /srv/mediawiki/php-1.26wmf16/extensions/Wikidata/extensions/Wikibase/lib/includes/EntityRetrievingDataTypeLookup.php: Property not found: P133 [13:06:42] >.> [13:06:45] whut [13:06:57] I guess we need to be more graceful there [13:07:12] Occasionally we can have references to deleted properties [13:08:08] hoo: nearly finished a integration test for the json dumps [13:08:29] oooo [13:08:34] in the dumps? [13:08:55] yep [13:09:04] :( [13:10:58] replying to RTL bugs rulez * https://phabricator.wikimedia.org/T107861 [13:11:41] Lydia_WMDE: Shall I delete the dump from yesterday night as well? [13:11:49] I mean the dump from Monday [13:11:52] hoo: if it is broken yes [13:12:10] Lydia_WMDE: Yeah... not as broken as the one from today, but still broken [13:12:47] done [13:13:12] addshore: What's master's state? Did you apply the patches against it or do we want to have a proper solution there? [13:13:27] I think apply the patch that is up for matser [13:13:47] then these integration tests, then perhaps work on it more, but ideally these tests first! [13:14:42] Ok [13:16:49] just trying to make phpunit detect and check the output of the script.... apparently expectOutputString isnt working [13:17:58] addshore: Have it write to a file in tempdir and read from that? [13:18:33] could do, will try it! [13:18:56] addshore: Will you do a hotfix for the fatal or shall I? [13:19:01] Should be easy enough [13:19:22] well... what should data type be in that case? empty string? just leave the key out? [13:19:24] DanielK_WMDE__: ^ [13:21:18] hoo: i would leave the key in [13:21:25] and make it an empty string? [13:21:44] maybe empty string, maybe 'unknown' or 'missing' [13:24:28] aude: I'll rebuild the items per site table, now that the schema change is through [13:24:34] or did you already kick that off? [13:25:10] hoo: no [13:25:21] ok :) [13:25:25] would be great if you do it [13:28:37] hoo: i wonder if mukunda didn't run scap or somethign is wrong on test.wikidata with i18n? [13:28:46] see https://test.wikidata.org/wiki/Special:ConstraintReport :( [13:28:56] it took him 3 times to get the right version [13:29:03] yikes [13:29:05] ok :/ [13:29:57] 22:07 logmsgbot: twentyafterfour Synchronized php-1.26wmf17: forgot submodule update (duration: 01m 39s) [13:30:03] ok, needs scap [13:30:12] oh, doh [13:30:13] :P [13:30:55] would be better if you could do it, but maybe i can do it quick [13:31:11] or have mukunda do it [13:31:28] I can do it [13:31:32] ok [13:31:37] but guess I need to wait for codfw to come back [13:31:43] :/ [13:31:49] * aude is at the airport :) [13:32:00] Flying back to Berlin? [13:32:03] not yet [13:50:15] * aude off [13:51:36] I still don't know what to set datatype to if the property no longer exists [13:52:30] if we could answer that, I guess I could fix the script easily and finally produce a complete dump in the right format [13:52:33] DanielK_WMDE__: Lydia_WMDE ^ [13:53:15] hoo: something like "none! [13:53:17] or? [13:54:01] hoo: https://gerrit.wikimedia.org/r/#/c/229379/1 I'll base the other patch on this test [13:54:23] addshore: I guess you need to do it the other way round [13:54:30] the test should not pass on master, right? [13:55:11] well, this test passes with current master, all be it a broken dump (no datatypes) [13:55:19] ill patch the test in the patch to add the data types [13:55:34] ok [13:55:39] but let's fix the branch first [13:57:04] Interestingly the old code just omitted it [13:57:07] //XXX: shall we set $serialization['datatype'] = 'bad' ?? [13:58:36] well, we should do that then! :) [13:58:59] ill make a patch in a second! also both patches for master are ready now (also excluding tihs patch I am about to make).... [14:04:49] also hoo patch for the latest exception @ https://gerrit.wikimedia.org/r/#/c/229380/ just got to manualls CP to the branch [14:04:51] *manually [14:06:49] :SSSSSSSS [14:06:56] the rebuild script failed [14:06:56] ? :SSSSS [14:07:22] hoo: what error? [14:07:27] duplicate key [14:07:51] We haven't (been able to) run it in several months [14:07:55] probably all year [14:08:05] so the table is probably in a very funky state [14:08:36] Last successful run of the script was on February 13 [14:09:06] Which was a Friday :D [14:09:10] That explains the mess :D [14:09:13] HAH! [14:10:06] maybe we should hot fix that as well, let me [14:10:07] see [14:10:09] * hoo sighs [14:11:40] hoo: for the branch https://gerrit.wikimedia.org/r/#/c/229381/ [14:12:09] Thanks :) [14:18:26] hoo: also https://gerrit.wikimedia.org/r/#/c/229383/ ;) [14:19:17] DanielK_WMDE__: sDrewth (semi-)struts [14:19:19] https://en.wikisource.org/wiki/Category:Author_pages_with_Wikidata_image [14:19:19] hoo: DanielK_WMDE__ asks if there is a ticket he can add his mustard to :D [14:19:21] i hope thats all the dump stuff caught.... [14:19:33] 1,596 total. and still populating [14:20:08] * Lydia_WMDE has to leave [14:21:37] * hoo rages [14:21:44] just resetted the wrong file [14:21:58] Lydia_WMDE: https://commons.wikimedia.org/wiki/Module:Wikidata [14:22:06] Now he's gone as well [14:22:52] hoo: module:wikidata needs some consistency across wikis, and REALLY a central version would be soooo useful [14:23:17] I know... I know [14:23:24] I'm saying that for years [14:23:39] hoo: on a related note - what's the Right Wayx to get best rank statements from Lua? [14:23:40] olay, so now I am supporting you :-))) [14:23:49] okay [14:24:17] hoo: for standardizing module:wikidata... any functionality that is common to most wikis should really be part of the extension, right? [14:24:32] DanielK_WMDE__: :getBestStatements( 'P123' ) ;) [14:24:45] that is part of mw.wikibase.entity [14:25:08] all documented on https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua :) [14:27:40] hoo: so, module:wikidata should start using getBestStatements, instead of returning all "valid" statements [14:28:24] probably [14:29:37] 2526 images added :-) [14:31:43] DanielK_WMDE__: addshore: Would like to backport https://gerrit.wikimedia.org/r/229385 as well [14:32:27] sDrewth: would you talk to the people who maintain module:wikidata, and point them to mw.wikibase.entity and getBestStatements ? [14:32:45] okay [14:32:49] thanks! [14:33:11] hoo: looking at the code in c:module:wikidata, it seems like we could use a generic filter facility that can be plugged in. [14:33:31] stuff like "best" or "with source" or "with start point after", etc [14:35:48] DanielK_WMDE__: Yeah, we should also have some generic stuff for reference formatting [14:35:55] SQLite :/ [14:38:42] DanielK_WMDE__: https://en.wikipedia.org/w/index.php?title=Module_talk:Wikidata&oldid=674690660 [14:38:52] get them to update, then enWS will steal :-) [14:41:46] \o/ [14:47:58] DanielK_WMDE__: How can I find out whether my last insert was successful on SQLite? [14:48:07] when using IGNORE [14:52:40] hoo: i have no idea [14:56:51] affectedRows returns 1 in that case and even insertId returns something :( [14:59:03] mh, the whole collision detection seems broken there [14:59:19] no DBError when I remove the IGNORE wtf [15:50:31] Thiemo_WMDE: https://github.com/wmde/DataTypes/pull/34 [15:52:02] DanielK_WMDE__: Do you think it would be ok to not run that test on SQLite? [15:52:06] I'm running out of ideas [15:52:25] Maybe the schema copy of MW doesn't have any indexes on SQLite? [15:55:16] Indeed :S [16:02:58] DanielK_WMDE: https://gerrit.wikimedia.org/r/229385 [16:12:12] addshore: ^ [17:57:47] so is the wikidata branch correct in wmf17 now? I'm about to push it out to group1 [17:59:51] twentyafterfour: did hoo run scap earlier today? [18:05:50] aude: I'm not sure [18:06:55] twentyafterfour: i think he did and believe everything was done right now ..... but [18:07:20] test.wikidata and test.wikipedia are giving g 502 errors [18:07:40] so something is not good and should be fixed before doing anything else [18:07:47] test2 interestingly works for me [18:10:13] hmm [18:10:39] i'm asking in operations [18:26:16] fixed [18:32:00] did something change with composer setup for wikidata? [18:32:39] I'm trying to update it and getting requirements conflicts [18:32:58] Your requirements could not be resolved to an installable set of packages. [18:33:31] SMalyshev: i think there is a new version of data model [18:33:32] http://pastebin.com/8pubLRA1 [18:33:49] aude: so what do I need to do to update it? [18:39:51] SMalyshev: maybe jzerebecki can help [18:40:03] (i am at theairport) [18:40:10] aude: ok, thanks [18:40:16] but disabling quality extension for now is probably the solution [18:40:23] i think there is a setting for it [18:40:43] aude: how one disables it from composer? just remoce the dependency? [18:40:49] yes quality constraints is still missing the update to dm 4 [18:40:56] SMalyshev: yes [18:41:18] https://github.com/wmde/WikidataBuildResources/blob/master/Wikidata.php#L34 [18:41:32] i think that would disable from composer [18:42:44] ok dropping quality & ocnstraints from composer.json seems to fix it [18:42:45] yes after disabling it in composer then disable by the global variable aude linked to [18:52:02] yeah disabling that stuff seems to make it work again [18:52:17] so master right now is not in consistent state? [18:54:01] SMalyshev: that's correct :/ [18:54:20] ok, I will be aware of it. thanks [19:07:32] SMalyshev: the conflict is resolved now. [19:32:44] jzerebecki: https://gerrit.wikimedia.org/r/229385 I would like to backport that, but still need a +2 [19:34:15] aude: ^ [19:34:19] You're back? :) [19:55:56] hoo: done [19:56:07] Thank you :) [21:57:15] DanielK_WMDE_: jzerebecki: aude: https://gerrit.wikimedia.org/r/229580 Trivial change [21:57:55] that json dump script failed for the 4th or 5th time now this week [21:57:59] Frustrating [21:58:25] oh it's on the branch, only aude could merge it anyway [22:08:45] Self merged, given how trivial it is [22:10:17] hoo: nnnnnnnnnnnnnooooooooooooooooooooooooooooooooooo you eviiiiiiiiiiil person [22:10:18] DanielK_WMDE_, jzerebecki: care to take a look on https://gerrit.wikimedia.org/r/#/c/229428/ ? [22:11:14] hoo: oh, someone wrongly backported that as part of something else? [22:11:21] SMalyshev: +1ed as I looked at it earlier, but didn't dare to approve [22:11:39] JeroenDeDauw: Addshore backported that, but we failed to realize the class name changed [22:11:58] ah [22:16:58] hoo: thanks [22:17:11] SMalyshev: done [22:18:24] DanielK_WMDE_: cool, thanks! [22:19:26] DanielK_WMDE_: and yes, we're still in beta mode, so we can pull off such things. In fact, that's why I want to get through most of them asap so we wouldn't have to break stuff later [22:19:57] stuff will change over time. [22:20:21] one thing i can foresee changing in a bit are references. i have a gut feeling they'll get guids instead of hashes [22:22:21] DanielK_WMDE_: since we don't ascribe any meaning to them shouldn't be much of an issue [22:36:01] PROBLEM - wikidata.org dispatch lag is higher than 300s on wikidata is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - pattern not found - 1422 bytes in 0.209 second response time [22:38:50] Am I deleting too much? [22:39:54] sjoerddebruin: I don't think so. seems like most edits currently are by https://www.wikidata.org/wiki/User:KrBot [22:40:05] Okay. [22:48:02] RECOVERY - wikidata.org dispatch lag is higher than 300s on wikidata is OK: HTTP OK: HTTP/1.1 200 OK - 1409 bytes in 0.114 second response time [22:48:55] jzerebecki: I fear that the warning is because of a serialization change [22:49:04] and I'm not sure what the effects are... sounds scary [22:49:10] ug :( [22:49:35] I'm inclined to make it UBN because I fear that dispatching could be (partly) broken [22:50:00] hoo: we certainly should look into it more right now [22:51:08] yeah [22:51:11] will try to verify [22:51:19] * reproduce [22:56:16] I can't [23:04:16] I no longer see that error in production with my dispatcher instance, but maybe there are just to few changes now [23:09:57] hoo: this is working on a db table, right? so the field info in it would need to be represented as null in php for this to happen? [23:10:30] It's serialized in the DB [23:10:44] and it could be anything, except of an array for that to occur [23:11:27] Warning: array_key_exists() expects parameter 2 to be array, null given in /srv/mediawiki/php-1.26wmf17/extensions/Wikidata/extensions/Wikibase/lib/includes/changes/DiffChange.php on line 95 [23:12:38] oh it says null, right [23:13:37] hoo: DiffChange.php gets $this->hasField( 'info' ) from ORMRow which operates on a normal db table, right? so it is not about serialization? [23:14:17] It's more magic than that [23:14:35] Look at the code in ChangeRow [23:16:07] ah ok :( [23:19:12] Innapropriate use of Active Record [23:19:37] Still think It'd be good to pull that stuff out of Lib and clean it up [23:20:59] hoo: so the value would be either null or the string N; or some invalid json or json deeper than the recursion limit? [23:21:13] JeroenDeDauw: Author: jeroendedauw [23:22:00] jzerebecki: Oh really?! [23:26:56] I don't see the error happening anymore [23:27:02] so let's leave it at high [23:27:09] will probably watch dispatching further [23:27:25] might have been a temporary thing, but I have no idea where that would come fomr [23:27:58] ok