[07:23:57] is anyone able to translate these to french? it is needed for merge.js tool [07:23:58] movingClaim: 'Moving claims...', [07:23:58] mergingClaim: 'Merging claims...', [07:23:58] movingSitelink: 'Moving site links...', [07:23:58] loadingMergeDestination: 'Loading merge destination...', [07:23:58] unwatching: 'Removing from watch list...', [07:23:58] deleting: 'Deleting...', [07:23:59] sendingToRfd: 'Sending to RfD...', [07:23:59] requestingStreamDeletion: 'Requesting Stream Deletion...' [07:46:48] http://www.wikidata.org/wiki/MediaWiki_talk:Gadget-Merge.js#French_translation [08:08:37] Ebraminio, I thought I'd let you know I made a dB query to find the bots that had used wbeditentity and removed aliases and reverted them all :) [08:09:06] addweekend: wow, great! [08:12:36] Turns out it was only 100 ish :p [08:13:08] As it only happens if you either send aliases back to set or do a null edit :-) and the latter doesn't really happen much :-) [09:10:48] addweekend: thank you, that is awesome [09:11:12] addweekend: thinking about reverting them / how to revert them? [09:13:38] Already reverted them :0 [09:13:41] :-) [09:14:19] Working on my bot rewrite now :-) [09:17:06] Ergh, its hard trying to add wikivoyage :/ thinking about it hurts my mind [09:20:11] I need a bigger pad of paper [09:35:16] addweekend: awesome (re revert) [09:35:39] maybe concentrate on the Wikipedias first, and then Wikivoyage? [09:36:35] It will be the same code :-). I just need to expand it to be project aware instead of just language aware :-) [09:59:56] JeroenDeDauw: one more thing [10:00:01] if you're here... [12:56:23] wikidata is timing out a lot for me today [12:56:30] anyone having similar experiences? [12:57:47] there are some general problems [12:58:00] gadgets not loading [12:58:16] DoS on api backend or something [12:59:09] woot? who would be dossing us? [13:00:25] see -tech [13:10:22] He he :) [13:10:27] Global issues, not just us :-) [13:10:50] we are down to 20 edits per minute [13:11:10] one could even watch the changes now :) [13:11:28] :D [13:11:50] Haha, great way to slow Wikidata down, turn off the api xD [13:12:58] If you want to see ingica spamming about the issues see #wikimedia-operations :) [13:23:24] addweekend: I see you like watching problems going on a large scale :) [13:40:41] New patchset: Bene; "Special page for entities without description" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/67634 [13:44:49] New patchset: Bene; "Special page for entities without description" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/67634 [13:47:41] He he, lazowik_, I'm just everywhere ;p [13:48:24] aude: ping [13:54:54] he he [14:00:27] can some wikidata sysop null edit [[MediaWiki:Gadget-definition]]? [14:00:28] [1] 04https://www.wikidata.org/wiki/MediaWiki:Gadget%2Ddefinition [14:00:38] I'm seeing https://bugzilla.wikimedia.org/show_bug.cgi?id=37228 there [14:02:56] liangent: done [14:02:59] I hate that bug [14:03:00] Denny_WMDE: perhaps the timeouts are caused by high load because of all the people visiting Wikidata after reading my blog post? [14:03:00] /me continues dreaming [14:03:22] lazowik_: pong [14:03:27] I'm here now [14:03:29] I hope [14:03:32] ^^ [14:03:45] hoo: thanks [14:04:07] should I change the $data only on adding sitelink or also on getting them? [14:04:38] JeroenDeDauw: I hope they stay as editors. After we got over9000, it didn't grow much any more :) [14:05:28] btw any server issue now? [14:05:49] I got a bunch of HTTP Error 504: Gateway Time-out at about 11:00 UTC [14:05:50] lazowik_: hmm, just realized I forgot about diffing yesterday [14:06:11] but it can be done separately? [14:06:12] liangent: yeah, they have been some problems today Wikimedia wide. they seem to be getting better [14:06:15] or is it? [14:06:20] liangent: more discussion on #wikimedia-tech [14:07:27] lazowik_: yeah sure - though it needs to be done in the same commit, else things will break [14:07:34] btw Denny_WMDE, I'm feeling aliases need fallback or at least translation too [14:07:55] lazowik_: oh no, wait, this is not true [14:07:59] liangent: no [14:08:03] I think so... [14:08:05] lazowik_: can be in seperate commits, so should be in seperate ones [14:08:16] I already wrote the code [14:08:18] and it workd [14:08:19] *s [14:08:41] liangent: the only reason of being for aliases is to help finding entities. they are never displayed. [14:08:43] hadn't run tests however [14:09:03] lazowik_: then you don't know if it works :) [14:09:21] right... [14:09:40] lazowik_: when retrieving sitelinks, I suggest you update the structure in $data [14:09:40] liangent: labels (and later multilingual text) is the only place where language fallback is needed [14:09:44] Denny_WMDE: so I have to add the same aliases again with the same content in zh-cn, even if some others added them in zh-tw [14:10:03] JeroenDeDauw: ok, diffs and patches broken [14:10:04] or at least, label lookup [14:10:27] lazowik_: also when serialization is done, this should happen. If we load an item, make some modification (not necessary sitelinks), we want to store it in the current format [14:10:59] And we want to avoid any item with both formats used saved to any palce [14:11:03] liangent: can we have the other steps first before we do translation of aliases for lookup? [14:11:28] JeroenDeDauw: but it's hard to overcome [14:11:33] but it seems having the same amount of work between implementing label lookup w/ fallback/translation & alias lookup w/ fallback/translation [14:11:40] it would need to be similar to db update [14:11:56] lazowik_: I'm not following you [14:12:11] oh, saved [14:12:12] ok [14:12:39] I thought you wanted to change all of them at once [14:12:57] Denny_WMDE: I don't have really look into it (wikidata has done too much storage abstraction), but do we have a standalone table for search keys? [14:13:10] it is in wb_terms [14:13:29] not exactly stand alone, but close [14:14:00] Denny_WMDE: any other usage besides search? [14:14:16] quick access of labels and descriptions for display, i think [14:14:23] without having to parse the entity first [14:16:24] Denny_WMDE: and is everything related to a given entity flushed there when the related entity gets edited? [14:18:32] depends on the implementation of the store [14:18:42] for the current sqlstore that we use, that is indeed the case [14:18:57] but there's a todo to try to be a bit smarter about it and change only what is needed [14:20:29] JeroenDeDauw: but if I'm to do it on deserialization [14:20:46] do I need to do it on gettting/saving sitelinks? [14:20:53] *adding [14:21:00] well to achieve "quick access of labels and descriptions for display", I guess I still need to put fallback/translate-resolved data there, and it's tied to search/lookup [14:21:10] and almost the same between fallback & aliases [14:21:12] Denny_WMDE: ^ [14:21:35] I mean "labels and aliases" [14:22:05] where have you been putting them so far? [14:22:38] also they can only be put there if they are user-independent, obviously [14:22:46] Denny_WMDE: I haven't put them anywhere so far [14:23:11] everything is resolve on the fly now [14:23:14] lazowik_: there is no need to update already created serializations - this would be non-trivial to do, and not serve much purpose if we retain compat with them anyway [14:23:37] liangent: that can work for display [14:23:42] right [14:23:47] but doesn't work for search [14:23:54] "also when serialization is done, this should happen. If we load an item, make some modification (not necessary sitelinks), we want to store it in the current format" [14:24:02] could you then explain that? [14:24:15] I'm missing something... [14:25:01] lazowik_: if we have an Item object, as soon as anything accesses $data['links'] in a way that is affected by the new structure, then update the whole structure. [14:25:01] This thus includes getting a sitelink, it also includes re-serialization. It does not include removing a sitelink, as this just cares about the key, which is unchanged. [14:25:14] liangent: but how could it work for search? [14:25:21] ok [14:25:59] wouldnt you translate the query to the fallback and then execute the translated query on the fallbacked language data? [14:26:10] JeroenDeDauw: so maybe I'll just add a function that does that? [14:26:24] that is the only way it could work due to user-specific fallback chains [14:26:52] lazowik_: the word for updating the structure would be in its own method yes, and this one would be called from several places [14:27:11] lazowik_: perhaps its worth doing something like the stubbing code for claims [14:27:19] see getStubbedClaims and unstubClaims [14:27:30] * lazowik_ looks [14:27:46] Denny_WMDE: I was thinking about putting resolved data into term_text? thus the goal is to have every term_language populated. but this introduce expensive operations on edits... [14:28:07] "wouldnt you translate the query to the fallback and then execute the translated query on the fallbacked language data?" << well this look nice [14:28:26] and flip translation source and dest for translated languages? [14:28:27] liangent: can we set up a time this week for one hour where we i have full attention to you and the other way around? what about tuesday 1pm UTC ? [14:29:20] flip translation source and dest for translated languages? <- didn't understand that [14:29:49] Denny_WMDE: tuesday 1pm UTC is fine [14:30:31] good [14:30:34] we can continue though [14:30:50] Denny_WMDE: I mean, I'm typing zh-cn search keys; wikidata / mw translate it to zh-tw then lookup in zh-tw terms [14:31:04] yes [14:31:10] first it looks in zh-cn [14:31:21] and if it doesnt find anything, translates to zh-tw and looks there [14:31:46] maybe even if it finds something [14:31:56] if it doesnt find enough [14:32:02] and then continues to zh [14:32:03] because of possible ambiguation [14:32:06] and to en [14:32:14] until it has enough results [14:32:19] all the way through the fallback chain [14:32:27] "until it has enough results" right [14:32:56] what's the current algorithm to sort results? [14:33:04] none [14:33:10] that's why our search is so bad [14:33:23] i actually made something on friday to be a bit better at that [14:33:32] but it is in review currently [14:33:38] Denny_WMDE: link? [14:33:45] current or new? [14:33:53] new [14:34:12] https://gerrit.wikimedia.org/r/#/c/73405/ + https://gerrit.wikimedia.org/r/#/c/73420/ [14:34:35] but really, it has to be replaced by something like solr or elasticsearch in the medium run [14:35:30] shall i add you as a reviewer? [14:35:33] Denny_WMDE: ok now "until it has enough results" doesn't work [14:35:55] because another result with a much bigger weight may appear later in the fallback chain [14:36:20] we have to think how weight plays into this [14:36:43] ie. I may want to write where (term_text='X' and term_language='zh') or (term_text='A' and term_language='en') order by term_weight desc [14:36:57] (this may break index) [14:37:20] term_weight is not indexed anyway in that implementation [14:37:20] db index [14:37:31] the ranking is done postdb [14:37:58] sounds terrible, and it is, but the indexes on this one are very big already [14:38:17] you mean on wb_term table? [14:38:19] we have to be careful in anything that makes this db and index bigger [14:38:21] yes [16:52:02] Is there any progress indicator for the transfer of language links to wikidata? What remains? [16:53:04] LA2: I made this analysis last week http://simia.net/languagelinks/2013.html [16:53:08] http://tools.wmflabs.org/addshore/addbot/iwlinks/ [16:54:03] Reedy 's link is better though :) [16:54:15] La2 I'll have more up to date stats next week :-) [16:54:25] about 2.4 Mio links left, i.e. 99% have been removed [16:54:34] After I have decided if the list I have just made is right or wrong :p [16:56:13] thanks [16:57:42] are the remaining ones the impossible ones, like town/city where there is an old interwiki conflict? [17:06:33] Denny_WMDE1: Your list reports, for sv.wikipedia, the article [[:sv:Trästeklar]], but the reported language links are within a [17:06:33] [1] 04https://www.wikidata.org/wiki/:sv:Tr%C3%A4steklar [17:09:25] oh, ignore the lists [17:09:29] they are still from last year [17:09:39] i only have numbers [17:09:51] only addmobile 's list have the actual article names [17:12:16] Reedy: the list informs me that https://es.wikipedia.org/wiki/Categor%C3%ADa:Localidades_de_Chile [17:12:29] still has interlanguage links, and there is no link to en.wikipedia [17:12:37] Not my list ;) [17:12:39] the obvious candidate would be https://es.wikipedia.org/wiki/Categor%C3%ADa:Localidades_de_Chile [17:13:03] eh, ... [17:13:23] https://en.wikipedia.org/wiki/Category:Populated_places_in_Chile [17:13:43] is there a tool to see why these are not united? [17:15:19] ah, be-x-old: is overlapping [17:16:08] and nl.wikipedia is too [17:16:34] and ro.wikipedia [17:17:24] in many cases with this subject the problem exists due city and town are mixed up [17:18:08] yes [17:18:24] so should we allow 3-4 languages of Wikipedia to delay the work of addbot? [17:18:47] I solved all interwikiconflicts in categories on nl-wiki [17:18:55] nl-wiki category namespace is interwiki clean [17:19:50] I just counted, about 30.000 articles have local interwikis on the Dutch Wikipedia, that is a bigger concern to me [17:19:50] La2 addbot only does the simple stuff ;p [17:19:58] Romaine: well, other languages still link to https://nl.wikipedia.org/wiki/Categorie:Stad_in_Chili [17:20:10] LA2: I know [17:20:32] addmobile: is it possible to run your bot with nl-wiki as base to start? [17:23:41] Romaine, I'm rewriting it currently, if you pop back during the week I'll tell you the plan and what order it looks at things :-) [17:24:51] many users are solving interwikiconflicts, so that is been taken care of, but all other local interwikis still needs to be moved [17:25:06] so any help it that is greately appreciated [17:25:35] The new code should be up by next week :-) so we will see how good it is :-) [17:27:24] ok thanks [17:43:15] JeroenDeDauw: I'm still not convinced [17:44:01] writing the code to cleanStructure maybe is not right [17:44:09] but calling the function from there? [17:44:38] actually it is structure cleaning... [17:47:04] some articles describe concepts that are mixed up, e.g. a garden tractor / two-wheel tractor / tiller. Is there any cross-language discussion on how best to sort out such concepts? [17:48:46] town/city/place/locality is a similar kind of conflict [18:02:25] New patchset: Jeroen De Dauw; "Update README file" [mediawiki/extensions/Ask] (master) - https://gerrit.wikimedia.org/r/73627 [18:02:39] Change merged: Jeroen De Dauw; [mediawiki/extensions/Ask] (master) - https://gerrit.wikimedia.org/r/73627 [18:03:36] lazowik: feel free to write up what you think is a good solution - I'll be happy to review [18:03:57] ok [18:09:00] New patchset: Jeroen De Dauw; "Update README" [mediawiki/extensions/Diff] (master) - https://gerrit.wikimedia.org/r/73629 [18:09:18] Change merged: Jeroen De Dauw; [mediawiki/extensions/Diff] (master) - https://gerrit.wikimedia.org/r/73629 [18:10:48] New patchset: Jeroen De Dauw; "Update dependencies in readme file" [mediawiki/extensions/Diff] (master) - https://gerrit.wikimedia.org/r/73630 [18:10:57] Change merged: Jeroen De Dauw; [mediawiki/extensions/Diff] (master) - https://gerrit.wikimedia.org/r/73630 [18:56:07] JeroenDeDauw: see what you've written here : [18:56:08] https://github.com/wikimedia/mediawiki-extensions-WikibaseDataModel/blob/master/DataModel/Entity/Entity.php#L475 [19:57:54] lazowik: you could add it in there if you really want to [19:58:08] added function [19:58:09] lazowik: I'm however not a fan of that, as this is getting called in the constructor [19:58:19] You'll be calling this ever time an Entity gets constructed [19:58:20] and that function there [19:58:25] Even when you don;t care about the sitelinks [19:58:37] otherwise diffs will be broken... [19:58:53] diffs relay directly on $data, don't they? [19:59:06] No [19:59:34] See getDiff and entityToDiffArray [19:59:36] in Entity [19:59:39] I'm off now [19:59:43] ok [19:59:46] bye! [20:00:04] I'll call the function from add/remove sitelinks [21:09:58] why are a parts of system messages in English, while my preferences is another language? [21:10:38] the whole page is in Dutch, besides "Wikipedia pages linked to this item" [21:10:39] ? [21:11:41] Has that message been translated? [21:12:03] in the past it was in Dutch [21:12:13] Hm. [21:12:26] Was it locally overridden? [21:13:08] mmm, on translatewiki it is not in /nl ??? [21:13:59] I see, software update [21:14:08] previously it used [21:14:12] https://translatewiki.net/wiki/MediaWiki:Wikibase-sitelinks/nl [21:14:45] that's weird... [21:14:46] now it uses https://translatewiki.net/wiki/MediaWiki:Wikibase-sitelinks-wikipedia/nl [21:14:53] I added content now [21:15:08] :D [21:15:14] right, that got changed because of wikivoyage [21:15:25] I think it is because they make Wikidata ready for other projects as well [21:17:01] translated the Wikivoyage one as well [21:42:12] i've found a bug on rollbacks that will cause crash on wikidata, is it known and reported? [21:44:16] I think there is one, lemme search [21:45:07] ebraminio: https://bugzilla.wikimedia.org/show_bug.cgi?id=49338 [21:53:39] legoktm: hmm. yes, I think this is, anyway, please test edit rollback on http://www.wikidata.org/w/index.php?title=Q13406268&action=history [21:54:02] yeah [21:54:05] thats the same thing i think [21:56:15] legoktm: this was my duty to check if is reported or not :P http://xkcd.com/386/ [22:03:01] Might look at that in the morning if I remember [22:11:40] addmobile: do you have some spare time from 17 August to 18 August? [22:12:34] Haha, well, I can tell you I have nothing in my calander :p [22:12:57] would you mind coming over to Bonn? :P [22:13:05] (with ICE) [22:13:16] your WMDE would pay, ofc [22:13:53] To Bonn? *googles* [22:14:35] MF-W, SPQRobin and me will meet us there and maybe you are interested, too :P [22:14:59] Why Bonn? :p [22:15:14] ask SPQRobin [22:15:34] he doesn't want to teVel that long from Belgium [22:15:39] *travel [22:15:44] we looked for a location between where I live and where MF-W/Vogone live [22:15:57] otherwise I'd have to travel too long [22:16:16] Haha, you'll have to remind me closer to the time :-) [22:16:32] where do you live (more or less)? [22:16:46] Berlin [22:16:59] he is in the Wikidata development team [22:17:05] ah [22:17:35] makes sense being in the wikidata channel :p [22:23:02] addmobile: Bonn is the former capital of Germany, btw :P [23:12:31] hi [23:12:45] Is there anyone who can help me? [23:13:05] I would like to get a list from wikidata [23:13:07] dubaduba: With what? [23:13:22] containing all articles from hungarian wikipedia [23:13:37] which do not have links to other language wikis [23:13:42] is it possible? [23:14:30] Only with querying the database, not via the user interface or API. [23:15:13] how can I query the DB? [23:16:10] You can only do that if you got a toolserver or tool labs account and some knowledge of (My)SQL [23:17:01] thanks [23:17:22] do you know if there are any existing tools online to do that? [23:17:39] I doubt it