[10:18:51] Daniel_WMDE: the item HAS an en_-label_ and sitelink since the first version [10:19:40] Merlissimo: uh... where do you see that? to be honest, i did not confirm in the json structure. but i don't see the label when looking at that revision [10:20:09] Merlissimo: if the label was indeed there, the bug is of course valid - and indeed a bug. [10:20:35] my bot always creates labels for each language that has also an sitelink when creating items [10:21:20] only on modifications no label is set/changes [10:21:29] Merlissimo: as it should be. does it strip the parentecy part? [10:21:34] i'm looking at the json now [10:22:33] parentecy part? [10:23:22] it scripts brackets and everything after a comma [10:23:38] ok. that was just a though [10:24:23] so... ok, you are correct. the label is there. which means that the display is not only incorrect in the diff view, but also when just looking at the revision [10:24:25] strange [10:25:51] Merlissimo: ok, so it seems that the page title is only replaced with the item label if the current revision is shown. it's broken for all old revisions, as far as i can tell. [10:25:58] i'll re-open the bug and change the description [10:27:26] modification of existing items is a new feature of my bot since yesterday [10:28:44] Merlissimo: if i may ask... which items are you importing, and to what purpose? are you planning to just import as much "stuff" as possible? or is it some selection? [10:31:05] my interwikibot is running normal on all wikipedia related wikis (which includes commons/incubator wikipedia testwikis and wikispecies) and my detects strong connected components. and i am submitting these string connected components to wikidata [10:33:22] Merlissimo: ok. so, basically, over time, it will import most of what's on wikipedia. [10:33:41] i don't think we have enough disk space for that on the test wiki... guess we'll find out ;) [10:34:12] i already filled the database some days ago [10:34:49] Merlissimo: be aware though that the data from the test wiki will likely not be transferred to the live site. So i'm not sure if this is really worth the effort [10:35:02] testing with lots of data (real) is good of course... [10:35:30] we'll have to look into the disk space and performance issues. [10:35:35] Jens told me in berlin, that it was hard to find conlfict free test cases, and i have these confict free interwikis groups [10:36:00] but i think raising disk space is a minor problem [10:36:31] well, the labs infrastructure isn't as flexible as i'd wish it to be, but yea, disk space should work. [10:36:49] enough ram to fit large database indexes could be a problem though. and that would make the wiki horribly slow. [10:37:04] Merlissimo: we are investigating, and will be discussing this next week when i'm in berlin again [10:37:21] for now, keep it running. having these test cases is sure very useful [10:37:23] yes load and ram could be a problem [10:37:48] ram, mostly, really, since redering and diffing wikidata is much cheaper than wikitext [10:38:49] Merlissimo: we may ask you to limit or filter the stuff you import by some random crieria, so we can have a smaller but still useful and mostly random test set. would that be ok with you? [10:39:19] on the traffic diagram you can see that my bot was inactive for some hours after version change: http://ganglia.wmflabs.org/latest/?r=week&cs=&ce=&c=wikidata-dev&h=wikidata-dev-1&tab=m&vn=&mc=2&z=medium&metric_group=ALLGROUPS [10:39:20] as i said, this is still to be discussedm but one possibel outcome is that we don't want to have *everything* on the test site right now. [10:39:38] filtering is not a problem [10:40:10] ok, great. could be by first-letter-of-hash-of-title or something. [10:40:21] currently my bot only submits groups with a minimum size of 2, but if my bot is too fast for wikidata requests only really big groups are checked and submitted [10:40:25] hm, 10k/s traffic should be fine [10:40:33] load stays <0.2 [10:41:16] group size could be one criteria, though i'd rather go for something more random. we'll see. [10:41:32] i think the limiting factor is going to be ram, i.e. index size, more than anything else [10:41:54] so we may want to limit the total number of items on the wiki [10:42:04] we'll let you know [10:48:36] Daniel_WMDE: some people calculated last year that there are about 16 million unrelated articles on all wikipedias (in total there are more than 22 mio. articles). so you'll have about 16 mio item some day [10:53:02] Daniel_WMDE: maybe it would make sence to filter if it contains an special wiki (e.g. only item that also contain dewiki), so the maximum number would be the number of articles on one wiki having at least on langlink [10:54:06] Merlissimo: we ant at least those 16 million, and are likely to have many more. we are planning for several hundred million - but that's for the live site. [10:54:47] the test site will probably not be able to handle that much. the vm has very limited memory. 1GB, I think, maybe 2GB. The production database servers have 24G and more. [10:55:05] my bot is limited by the s3 cluster write limits [10:55:22] in speed, yea. speed is not that much of an issue [10:55:38] i'm thinking about limiting the total number of items. [10:55:47] so no change to add those many item within some weeks while doing normal interwiki bot work [10:55:55] and what the criteria should be. i'll discuss it with the team in a few days. [10:56:06] yea, i know. [10:56:21] no urgent action needed. i just wanted to let you know [10:56:30] let'S talk next week some time [10:56:35] i'm afk for now, family stuff [10:58:01] cu [14:21:59] New review: Jeroen De Dauw; "Why are you preventing creation of SiteLinks objects with sites that are not present in the sites ta..." [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/16200 [16:13:22] New review: Jeroen De Dauw; "As I mentioned on the internal list and the other commit in which you made this change: the site cod..." [mediawiki/extensions/Wikibase] (master); V: 0 C: -2; - https://gerrit.wikimedia.org/r/16201 [16:52:18] New patchset: Jeroen De Dauw; "fix cp error" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16223 [16:52:28] Change merged: Jeroen De Dauw; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16223 [16:59:17] New patchset: Jeroen De Dauw; "fix cp error" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16224 [16:59:28] Change merged: Jeroen De Dauw; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16224 [19:13:19] Change abandoned: Daniel Kinzler; "abandoned in favour of core change" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16201 [19:14:59] New review: Daniel Kinzler; "@Jeroen: We need the Site object in order to do two things:" [mediawiki/extensions/Wikibase] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/16200 [19:53:39] New patchset: Daniel Kinzler; "Refactored link normalization, moved to SiteLink" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16200 [20:15:58] New review: Daniel Kinzler; "This breaks diffs for all items! See https://bugzilla.wikimedia.org/show_bug.cgi?id=38555" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/15967 [20:17:46] New review: Daniel Kinzler; "Seems fine. " [mediawiki/extensions/Wikibase] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/16134 [20:17:46] Change merged: Daniel Kinzler; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16134 [20:18:15] Daniel_WMDE: poke [20:18:25] JeroenDeDauw: heya [20:19:21] Daniel_WMDE: I actually noticed that diffs between revs broke because of that change yes [20:19:27] Daniel_WMDE: the fix has already been merged in [20:19:39] Daniel_WMDE: at least if the issue you where referring to was a fatal error [20:19:52] Daniel_WMDE: there is still an issue with the thing not getting displayed correcty [20:20:00] JeroenDeDauw: no, not a fatal error [20:20:06] it just doesn't work [20:20:12] it shows the json diff [20:20:24] Daniel_WMDE: yes, that's the display issue [20:20:36] Daniel_WMDE: I will look into this now [20:20:42] the reason is that ItemDiffView doesn't derive from EntityDiffView [20:20:56] Daniel_WMDE: I noticed while working on getting the display of undo diffs correct, but this turns out to be less trivial then I though [20:21:26] Daniel_WMDE: it does... [20:21:40] Daniel_WMDE: class ItemDiffView extends EntityDiffView { [20:22:23] JeroenDeDauw: ItemContentDiffView [20:22:30] ItemContentDiffView extends \DifferenceEngine [20:22:51] JeroenDeDauw: which is the class actually used. [20:23:00] why do we have ItemDiffView and ItemContentDiffView [20:23:02] ? [20:23:08] is that intentional? [20:24:41] Daniel_WMDE: yes [20:24:43] New patchset: Jeroen De Dauw; "fix display of diffs between revs - spotted by danielk" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16232 [20:24:46] Daniel_WMDE: there you go [20:25:19] Daniel_WMDE: we could want to display diffs for entities loose from the content [20:25:23] for instance on the client [20:25:25] or whatever [20:25:56] yes... perhaps... maybe... [20:27:27] JeroenDeDauw: please remember to put bug numbers into your commit messages. i keep forgetting that too, but it helps. [20:29:22] ok, the ItemDiffView vs ItemContentDiffView thing has me confused. otherwise, i'd have fixed it myself. [20:29:37] New review: Daniel Kinzler; "fixes bug 38555" [mediawiki/extensions/Wikibase] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/16232 [20:29:37] Change merged: Daniel Kinzler; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/16232 [20:33:42] Daniel_WMDE: send you a mail about the diff stuff I'm having problems with [20:34:09] JeroenDeDauw: will probably not look into it tonight, maybe tomorrow. definitly monday. [20:35:49] JeroenDeDauw: the error indicates that it failed to load a revision. but i'd have to dig deeper to find out where and why. [20:36:08] * Lydia_WMDE waves [20:36:08] Daniel_WMDE: yeah... [20:36:26] hi Lydia_WMDE! [20:36:58] Daniel_WMDE: I'm getting that when I let ItemEditAction derive from EditAction rather then ItemViewAction and then click an undo link - which suggests it's broken for CH in general [20:37:04] It's a Lydia_WMDE [20:37:09] it is! [20:37:09] :P [20:37:30] currently adding new members to kde eV - one of the more fun parts of being on the board :P [20:37:56] Lydia_WMDE: after kicking them out? [20:37:59] * JeroenDeDauw hides [20:38:02] lol [20:38:11] the kicking out part comes next week [20:38:35] still need to see who missed two assemblies in a row and needs to be kicked [20:38:57] but i need data from claudia for that so can't do it now [20:39:25] Daniel_WMDE: you did not address all my concerns about the SiteLink stuff - is this intended to be followed up by a new patchset? [21:49:19] Daniel_WMDE: poke [23:55:28] New review: John Erling Blad; "A couple of comments that probably should be handled but isn't really blockers for our use of the co..." [mediawiki/extensions/Wikibase] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/16200