[00:37:33] How would we like a WD: namespace alias for the project namespace? [00:37:49] (like WP: for Wikipedia: on Wikipedia) [01:04:56] http://dumps.wikimedia.org/wikidatawiki/20121026/ [01:04:56] Woo, database dump ;) [01:06:42] {"label":{"en":"Barack Obama","fr":"Barack Obama","ar":"\u0628\u0627\u0631\u0627\u0643 \u0623\u0648\u0628\u0627\u0645\u0627","ru":"\u0411\u0430\u0440\u0430\u043a \u041e\u0431\u0430\u043c\u0430","nb":"Barack Obama","it":"Barack Obama","de":"Barack [01:06:42] Obama","be-tarask":"\u0411\u0430\u0440\u0430\u043a \u0410\u0431\u0430\u043c\u0430","nan":"Barack Obama","ca":"Barack Obama"},"description":{"en":"President of the United States of America [01:06:46] * Reedy barfs [01:09:23] * Jasper_Deng hands Reedy some towels to clean up [01:14:38] So... Full history for Q1-Q100.... 76.1MB [01:15:15] 7z turns that into 887KB [01:15:29] I herd u liek bloat; so I added JSON to your XML... [01:19:11] | Data Base Name | Data Base Size in GB | [01:19:11] +----------------+----------------------+ [01:19:11] | wikidatawiki | 0.756133630872 | [02:55:40] New review: SPQRobin; "Well, the documentation is right in that it may be both a Language object and a string, to make it e..." [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/31650 [04:38:18] Hmm, is there a bug that was already reported regarding the favicon of wikidata.org? [04:43:42] I don't know if it's reported or not, but I can confirm that it doesn't display for me [04:43:51] erm, actually, it does [04:44:12] Hydriz: what's the bug? [04:44:36] PiRSquared17: The favicon seems to show the word "office" in it. [04:44:48] Hydriz: purge your cache [04:45:04] and restart/reset your browser, should be fixed [04:45:16] PiRSquared17: Ah, great. Thanks. [04:45:48] Simple problem -> simple answer [04:46:05] :) [05:00:17] How can I add babel? There are no babel templates [05:00:43] the parserfunction one doesn't work inside tables [08:55:47] Lydia_WMDE: Hi! [08:55:50] I'e written a page about the integration of external ids into Wikidata: https://www.wikidata.org/wiki/User:Tpt/Support_of_external_links [08:55:52] Can I have your input about it before propose it to the community ? [08:56:04] hey Tpt! [08:56:07] i'll have a look [08:56:24] Thanks :-) [08:57:56] Tpt: looks good - you probably want input from DanielK_WMDE_ (and Denny_WMDE when he's online later) [09:00:31] Tpt: in case i have not mentioned this yet: it's awesome you're taggling this! :) [09:02:03] Lydia_WMDE: Thanks. I'll ask DanielK_WMDE_ [09:02:07] :) [09:02:26] hello. Denny Vrandecic sent us a change to "configure babel category for Wikidata" . I have deployed the change on the cluster :-) [09:03:01] so if he happens to be in the WMDE office, can you let him know please ? :-] [09:03:03] hashar: oh nice - what's possible now? [09:03:04] related change is https://gerrit.wikimedia.org/r/#/c/31608/ [09:03:15] Lydia_WMDE: I have no idea :-]]]]]]]]]]]]] [09:03:16] will let him know [09:03:16] hah [09:03:16] ok [09:03:36] I just know it is not going to kill the cluster so I have deployed it hehe [09:03:45] *lol* [09:03:46] ok [09:03:47] thanks [09:04:17] seems to be a PHP Extension so user can list the languages they know on their userpage [09:04:27] (much better than using templates if you ask me) [09:04:30] *nod* [09:05:05] though the fact that I know english/french should be in wikidata.org :D [09:05:15] something like query=languages&user=hashar --> en|fr [09:06:13] hehe [09:06:51] I am back to deploying moaar changes [09:06:52] have a good morning :) [09:06:53] have fun [09:44:58] DanielK_WMDE_: poke [09:48:00] DanielK_WMDE_: Can I have your input about https://www.wikidata.org/wiki/User:Tpt/Support_of_external_ids before propose it to the community ? [09:59:34] New patchset: Aude; "move autocomment back to repo; not used yet in client and causing test issues with client tests" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31815 [10:10:07] Tpt: sounds solid to me - I agree on the rationale and the implementation. [10:12:50] Tpt: there is one thing to keep in mind though, which is important wrt storage in the database and also UI design: there is a fundamental difference between the wikipedia-links and the external IDs. wikipedia-links point to *docuemtns* with the same *topic*. external identifiers are alternative URIs for identifying that topic, they are not (primarily) URLs of documents. [10:13:06] Although such URIs often do resolve to descriptive documents, that's not a requirement, and it's semantically important to note that these IDs do not reference documents, but topics. [10:13:32] (in case the poic is a book or a website, it itself *is* a document - confusing....) [10:13:38] *the topic [10:14:44] Tpt: DanielK_WMDE_ : hey [10:14:47] moin [10:15:33] i was thinking of implementing external identifiers using normal statements with a property of the type monolingual text [10:16:04] the property could then have further descriptions which are used for exporting etc. [10:16:40] this would allow users to extend the list in a straightforward way using the mechanisms that we will have anyway [10:17:12] and we wouldn't need to think about issues like "what if there are several identifiers for x?" [10:17:38] or "what about if the identifiers are indeed not unique?" (like ISBNs) [10:17:58] external identifiers arn't unique per item. though i think the same external id shouldn't be used for multiple items. not sure about that, though [10:18:38] but it can happen [10:18:53] Yea. Anyway. "monolingual test" would be wrong, the ID doesn't have *one* language, it has *no* language. So, it's a string, not a text at all. [10:18:53] due to errors in the external system [10:19:00] we need such a type anyway, for zip codes, etc [10:19:08] yea. [10:19:08] zip code is the same thing [10:19:29] anyway - if we just use plain text IDs, we can't generate links. Links are very useful for this kind of thing, especially for verifying [10:19:50] the property definition could contain an URL pattern [10:19:51] we can put that information in the property [10:19:55] yes, exactly [10:20:05] yes, that is also a possibility [10:20:12] though we wouldn't get any nice auto-completion, etc [10:20:28] auto-completion on viaf ids? [10:20:35] it's simpler, and easier to extend, but less powerful [10:21:03] anything that we would want to do there could be done with external tools [10:21:08] Denny_WMDE: if there's a decent search/suggest api for viaf, you'd start typing your ID, and would see a list of matches along with their descriptions [10:21:20] so you have an eyeball verification of what you are entering [10:21:32] i guess [10:22:06] Denny_WMDE: perhaps add your idea as an alternative proposal to Tpt's page? Or you two talk about it and acome to an agreement :) [10:22:16] i will [10:22:16] New review: Jeroen De Dauw; "Which issues was this causing? Just moving the files will leave us with the same issue for installs ..." [mediawiki/extensions/Wikibase] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/31815 [10:22:30] i just wanted to make sure in the chat that it is known that i have alternative ideas :) [10:22:30] I think that we can split identifiers in two categories: [10:22:33] 1 Identifiers to resources that point to documents petty same as the wikipedia pages (by example: viaf page about a person). This identifier must be unique. [10:22:35] 2 Identifiers that are related to the topic of the document (example: ISBN identifier that identify publication of the book, not document about the book). [10:22:45] Tpt: thank you, by the way! [10:23:23] Tpt: I agree, but I would prefer not to have that split in the system, but be built on top of the system [10:23:34] i will comment on your suggestion on-wiki [10:23:37] New review: Aude; "Autocomment (at least the test) require stuff in the repo. That won't work on client installs. Unt..." [mediawiki/extensions/Wikibase] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/31815 [10:23:59] Tpt: Dennys idea is more generic and allows new identifiers to be used without the need to write code. that'S a big advantage. [10:24:06] the question is - what big advantage does the code-based solution offer? [10:24:06] DanielK_WMDE_: could you give me a status on the branch? (could go private) [10:25:01] Denny_WMDE: nope, lost track of that on friday over fixing bugs. last i checked, there was still quite a bit of client stuff to be reviewed [10:25:07] and then a bunch of things to backport. [10:25:17] maybe too much to backport. [10:25:31] the branch is now nearly a week old. [10:26:04] do we have a list of stuff to backport? [10:26:34] I see two advantages of the software solution: [10:26:39] 1 URI resolving; you can get a Wikidata entry from the identifier. [10:26:39] 2 Validation of ID and protection of the uniqueness of the ID. [10:26:46] DanielK_WMDE_: are you suggesting to make a new branch instead? [10:27:14] Tpt: but the IDs might not be unique [10:27:32] there are many examples of external systems that have assigned the same ID to different entities by error [10:27:55] forcing uniqueness constraints on that will lead to issues [10:28:13] Denny_WMDE: it's a possibility. but i'd love to pass the responsibility for the branch stuff back to anja. Or is she not available? [10:28:21] checking uniqueness a posteriori is much easier and lighter on the system [10:28:22] she is [10:28:36] ok, pass it to her with a status update [10:28:51] I'd prefer her to take the lead on this then. I'll help out of course [10:29:05] I talk only about ids that might be unique if there is no bugs in the database. [10:29:05] to answer your qestion: no, we don't have a list [10:29:12] we pretty much have to audit the git log [10:29:20] :P [10:29:23] ok, thanks [10:29:50] Tpt: that would mean that in case there *is* a bug in their database, we can't use their ID. [10:30:27] DanielK: Yes [10:30:39] For not unique Ids like ISBN the Denny's solution is I think the best one. [10:31:17] Tpt: perhaps we should go with dennys solution first, then, and see if we still need the code based one. [10:32:06] Tpt, Denny_WMDE: It could also be possible to mark properties as unique, and enforce constraints this way (or at least highlight and find violations of the constraint easily) [10:32:34] * DanielK_WMDE_ likes that idea [10:33:46] I agree with this solution if there will be a way to validate the ids. [10:35:46] Tpt: the wikipedia way would be to use bots for that [10:36:40] New patchset: Aude; "adding some i18n to the recent changes lines" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30998 [10:36:49] New review: Jeroen De Dauw; "Ok, makes sense." [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/31815 [10:38:15] DanielK_WMDE_: Yes, It may be a solution. [10:38:28] New patchset: Aude; "move autocomment back to repo; not used yet in client and causing test issues with client tests" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31815 [10:39:44] New review: Jeroen De Dauw; "Autoloads still not done" [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/31815 [10:43:34] Let the software have as little assumptions as possible [10:43:43] for the sitelinks in phase 1 we need the assumption of uniqueness [10:43:50] but for external IDs we do not [10:44:11] so we can leave the responsibility for checking their validity to the community [10:44:11] i.e. the bots [10:44:41] New patchset: Aude; "add injectRecentChanges setting to enable or disable the external RC stuff" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30948 [10:45:57] Denny_WMDE: do we have a current writeup of well-known properties we may want properties to have besides the data type? like a url pattern for generating a canonical URI or authoritative URL. [10:47:12] no, we are not there yet [10:48:32] might be good to start collecting ideas [10:48:49] New patchset: Aude; "move autocomment back to repo; not used yet in client and causing test issues with client tests" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31815 [11:03:30] DanielK_WMDE_: 41667 now seems to be non-existent in my localhost, but it could be that I'm actually is testing this in a wrong way.. [11:08:31] New patchset: John Erling Blad; "(Bug 41214) Changes to use variant length prefixes and fragments" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/28527 [11:10:09] New patchset: John Erling Blad; "(Bug 41214) Changes to use variant length prefixes and fragments" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/28527 [11:24:09] New review: John Erling Blad; "Seems like failures in the tests are still there." [mediawiki/extensions/Wikibase] (master); V: -1 C: 0; - https://gerrit.wikimedia.org/r/30963 [11:24:53] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30809 [11:25:19] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30981 [11:26:44] New review: Daniel Kinzler; "@john: can't reproduce this. can you please look what *is* in the RC feed on your box? Maybe that gi..." [mediawiki/extensions/Wikibase] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/30963 [11:32:38] New review: John Erling Blad; "Ooops I did it again.." [mediawiki/extensions/Wikibase] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/31374 [11:32:38] Change merged: John Erling Blad; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31374 [11:35:18] is Sk!d by any chance 'round here? [11:38:29] Sannita: doesn't look like it - anything someone else can help with? [11:39:42] thanks Lydia_WMDE, you're always kind :) no, I kinda need him, I just wanted to get some advices on pywiki(pedia|data)bot framework [11:39:54] ah :) [11:42:56] AnjaJ_WMDE_: http://pastie.org/5188903 [11:43:39] these are on master of wikibase and don't worry about the testSquashToNFC tests which are because of my configs [11:45:13] http://pastie.org/5188930 [11:55:50] AnjaJ_WMDE_: http://pastie.org/5188984 is what i get on ubuntu [12:04:59] New patchset: Aude; "adding some i18n to the recent changes lines" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30998 [12:06:30] New patchset: Daniel Kinzler; "(Bug 41667) remove onPageContentLanguage handler." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31818 [12:09:37] New patchset: Tobias Gritschacher; "JS refactoring: Api.js (DO NOT MERGE)" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31206 [12:11:18] aude: i'm currently working on the getPageLanguage stuff [12:12:02] http://git-animals.tumblr.com [12:12:03] awesome tumblr ;) [12:12:32] DanielK_WMDE_: should it really return wfGetLangObj( $pageLang ); [12:12:47] New patchset: Tobias Gritschacher; "JS refactoring: Api.js (DO NOT MERGE)" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31206 [12:12:54] it is already a lang object? [12:13:23] New patchset: Aude; "modify watchlist query conds to include external changes" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31820 [12:13:49] trrrrommelommelommmmeee... waiting for tests.. [12:14:06] DanielK_WMDE_: anything you need reviewed? (i can poke at the OAI stuff) [12:14:35] or should i help fix tests? [12:15:17] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30998 [12:18:46] DanielK_WMDE_: how do i make phpunit not report the skipped tests? [12:20:43] aude: in suite.xml, set verbose to false. also, set strict to false to disable timeouts [12:20:53] ok :) [12:21:01] there are literally hundreds of parser tests skipped on core [12:21:22] aude: the OAI stuff would be helpful for search, but i think other stuff takes prio over that, ask denny. [12:21:31] ok, maybe tomorrow [12:22:20] https://gerrit.wikimedia.org/r/#/c/29973/ would be good to get reviewed <-- maybe hashar can take another look [12:22:21] aude: you could poke at 41491, if JeroenDeDauw isn't working on that [12:22:26] ok [12:22:39] aude: yes, indeed [12:22:53] and then I0b1db6d7 of course [12:22:55] oh looks like I forgot to review that one :-D [12:22:58] the tests are blocking your other patch since there is dependency [12:23:19] hashar: would be helpful :) [12:23:21] yes, i hoped that the dependency would speed up the review for the test cases :) [12:23:40] also, it's good to have test cases when fixing complicated stuff like this. [12:24:10] agree [12:24:25] aude: other than that, i think i can take care of the bugs i mentioned in my mail this morning. then looks at gerrit a bit. [12:24:32] damnit php [12:24:36] hehe... [12:24:36] if only we could define our own keywords [12:24:50] where( 'foo' => bar ); [12:25:06] DanielK_WMDE_: ok [12:32:30] New review: Tobias Gritschacher; "How exactly is this supposed to work?" [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/30948 [12:36:46] New patchset: Daniel Werner; "Cleanup for doxygen comments, added some namespace usages" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31822 [12:43:21] Change merged: Jeroen De Dauw; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31822 [12:47:27] DanielK_WMDE_: am unable to reproduce https://bugzilla.wikimedia.org/41491 [12:47:30] maybe JeroenDeDauw can look ^ [12:49:27] New patchset: Jens Ohlig; "Reorganized wbsearchentities" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31824 [12:52:56] aude: maybe Nikerabbit himself can shed some light on this. [12:53:51] ok [12:53:51] jeblad_WMDE: re isCountable(): i don't expect this function to stay this simple. having a test for the current implementation makes sure we assert future behavior in the test cases. [12:54:51] To not merge something because it may be more complexin the future really don't make sense.. [12:54:54] So, time spent to set up a trivial test isn't wasted, because you'll not have to do (and think about it) later when isCountable becomes more complex [12:55:13] aude: can't reproduce it either [12:55:43] jeblad_WMDE: to not merge something because it does not have tests is actually policy. to let that slide in some truely trivial cases may be ok, but in this case i would have preferred to see a test case. [12:57:29] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31820 [12:58:37] New patchset: Daniel Kinzler; "(Bug 41667) remove onPageContentLanguage handler." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31818 [12:59:06] New patchset: Daniel Werner; "Some more doxygen cleanup" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31825 [13:01:11] New patchset: Henning Snater; "JS refactoring: Api.js (DO NOT MERGE)" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31206 [13:06:03] New patchset: Aude; "add injectRecentChanges setting to enable or disable the external RC stuff" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30948 [13:06:28] New review: Aude; "tests are coming in a separate patch" [mediawiki/extensions/Wikibase] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/30948 [13:06:54] still reviewing the edit page test s:-D [13:07:57] hashar: some of my comments in https://bugzilla.wikimedia.org/show_bug.cgi?id=41352 might be useful [13:08:00] New patchset: Henning Snater; "JS refactoring: Api.js" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31206 [13:08:07] explaining what's supposed to happen [13:08:23] aude: I am trying to fix up the newline madness and get rid of trim() [13:09:06] typo in my comment: "He likes ices cream." => "He likes ice cream" but otherwise it should help [13:09:21] https://bugzilla.wikimedia.org/show_bug.cgi?id=41352#c9 [13:24:05] New review: Tobias Gritschacher; "needs rebase" [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/30948 [13:28:55] JeroenDeDauw: Hi! [13:28:57] About this https://gerrit.wikimedia.org/r/#/c/31508 , do you see a way to get the entities without label without having a table that list all the entities ? [13:32:09] New review: Aude; "looks good now, works nicely if coming from Special:ItemByTitle or coming directly to Special:Create..." [mediawiki/extensions/Wikibase] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/31308 [13:33:03] Change merged: Aude; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31308 [13:34:24] Tpt: thanks for making createitem better [13:34:53] Denny_WMDE: what should we do with the bug? should i mark it resolved? [13:35:00] https://bugzilla.wikimedia.org/40509 [13:35:46] aude:Thanks ! [13:36:27] Tpt: hey [13:36:28] aude: assigned [13:36:44] ok, got it [13:36:59] Tpt: let me look at it again - guess I was a bit sleepy when I reviewed that :) [13:37:07] Tpt: aude JeroenDeDauw jeblad_WMDE thanks [13:38:46] Denny_WMDE: huh, for what? [13:39:01] Tpt: guess such a table is indeed needed [13:39:02] eating cucumber? =D [13:39:16] so he won't get tempted! =) [13:39:19] Tpt: at least if it's not acceptable to miss entities that do not have a term at all in any language [13:40:17] JeroenDeDauw: Yes. This table will be also useful to implement "items without links". [13:40:37] Indeed [13:40:45] Tpt: the name is odd though wb_entities_per_page [13:40:52] we have a one to one map [13:41:03] How can we have several entities per page? [13:41:44] Yes it's one to one map. [13:41:57] Bug 40509 .. I'm not sure about this solution [13:42:45] why do we need a table for a one-to-one mapping that is clear from the name? [13:43:34] But we need some stats on whats in each entity [13:43:44] New patchset: Aude; "add injectRecentChanges setting to enable or disable the external RC stuff" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30948 [13:44:05] * jeblad_WMDE needs coffee [13:44:25] New patchset: Aude; "add injectRecentChanges setting for external RC stuff" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30948 [13:44:30] JeroenDeDauw: Tpt: I would suggest it is OK to miss entities that have no labels at all if it makes things easier [13:44:54] they would be caught by a empty item special page, no? [13:44:56] ah wait [13:45:00] Tpt: what about wb_entity_props ( entity_id, entity_type, prop_name, prop_value ) [13:45:01] this table could then be used for various kinds of lookups on stuff such as "has no links" [13:45:01] assuming there is at least one such property set for each entity it can then also be used for the label lookup you are doing [13:45:08] a page can have sitelinks but still have no labels at all [13:45:13] which means it could be non-empty but without labels [13:45:14] sucks [13:46:41] ...the feeling when you ee a really ugly bug in your own code, and you know it will take time to fix it.. #darn [13:47:35] The problem is that it's less flexible. With a wb_entities_per_page we can do all the queries we want (enties/item/property with/without a condition. [13:47:37] Make a table with some rudimentary stats about the entities [13:50:03] We have also to pay attention to people that will want to make research on Wikidata database and will want to do queries we haven't think about. [13:51:59] Change merged: Aude; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31818 [13:52:50] The table as I've done allow also to make easily the link between the wikibase tables and the storage of pages done by MediaWiki. [13:53:32] Tpt: sure, I agree we need it [13:53:50] Tpt: and that a prop table is not ideal since it needs an extra condition [13:53:57] Disagree it's less flexible though [13:54:27] Tpt: if you think having just a plain page id to entity id map table that's fine with me [13:55:02] I'm just wondering that if we end up with a props table later on anyway, such a map could easily be modelled in there [13:55:15] So perhaps we could just not have the simple map table at all [13:55:26] aude: DanielK_WMDE_ : I am a bit busy with analytics team right now. But definitely intend to send something regarding the edit page tests :-] [13:55:44] Tpt: however like I said, I don't object to having it, just wondering [13:55:54] Tpt: needs a better name then it has not though [13:55:57] *now [13:56:23] JeroenDeDauw: Do you have an idea of name ? [13:57:13] Tpt: you could just drop the plural and end up with wb_entity_per_page [13:57:21] hashar: ok [13:58:13] JeroenDeDauw: Ok. I'll change it. [13:58:44] and move to TermCache the SQL query of SpecialEntityWithoutLabel. [13:59:22] Tpt: fantastic :) [13:59:33] When this patch will be merged, I'll implement SpecialItemsWithoutLinks and some other ones if it's needed. [14:01:52] Tpt: could you also write a test for the new query? that makes review and maintenance a lot easier [14:02:25] I don't know how to write tests :-( . [14:04:04] Tpt: it's not hard, there are plenty of examples [14:04:11] * DanielK_WMDE_ is going afk for a while [14:05:25] Tpt: yeah, if you can figure out how to do the query you did, you can definitely figure out how to write a test for it :) [14:05:30] Tpt: examples here https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/Wikibase.git;a=tree;f=repo/tests/phpunit/includes/store;h=908ab9065656efd69e2033f4b1d35c2d5528b94a;hb=HEAD [14:06:02] Tpt: the tests are registered in Wikibase.hooks.php in registerUnitTests [14:06:14] Tpt: https://www.mediawiki.org/wiki/Manual:PHP_unit_testing [14:07:35] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/30948 [14:07:45] Tpt: for finding pages with no links, I'm thinking it might be nice to instead of doing a query against the existing link table have the mentioned entity_props table with a "link count" property [14:08:05] That way we can also find "entities with few links" [14:08:49] And maintaining these props is very easy [14:09:03] stats on links, stats on statements, stats on whatever.. [14:09:33] Extract items with unusual low number of references compared to the claims [14:10:39] It's a very good idea I believe. :-) [14:11:59] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31815 [14:13:25] New patchset: Henning Snater; "JS refactoring: Api.js" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31206 [14:14:54] JeroenDeDauw: I've an other question: Why to set @since 0.3 here: https://gerrit.wikimedia.org/r/#/c/31508/4/repo/includes/specials/SpecialWikibaseQueryPage.php [14:14:54] This patch can be added to phase 1 version, can't it ? [14:14:55] New patchset: Jeroen De Dauw; "Added ValueParser JavaScript" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31830 [14:15:53] jeblad_WMDE: and of course "entities edited by the crazy Norwegian aka jeblad"? :p [14:15:54] * JeroenDeDauw hides [14:16:14] Meh, that actually should be in page props [14:16:36] entities containing "cats" and "fries".. ;p [14:17:20] Tpt: that version is the version of the software. 0.3 does not mean phase 3. We are now in the process of getting 0.2 ready for deploy, and this is still only phase 1 stuff [14:17:57] But seriously,.. it should be possible to find entities with missing specific languages from the new pages lisyt [14:17:59] JeroenDeDauw: Ok, Thanks. [14:18:05] jeblad_WMDE: oh, good point. Denny_WMDE1, I hope are you convinced this is our no 1. priority now :) [14:19:03] Of course it is! [14:20:21] Tpt: jeblad_WMDE: later on in phase3 we can perhaps move the entity props table functionality by having system properties, much like we will probably do as with labels and descriptions [14:20:31] New patchset: John Erling Blad; "(Bug 41581) Add an autocomment with label and value" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31831 [14:20:51] Although the implementation might be the same for the naive sql store anyway ... [14:21:43] JeroenDeDauw: Dunno,.. I just know what I want, but not who to bribe.. [14:24:13] New patchset: Jeroen De Dauw; "Added ValueParser JavaScript" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31830 [14:24:40] We need a better solution for Autocomments when they are not built as part of api calls.. Perhaps something object-ish.. I've heard that is an popular new way to code fancy stuff... [14:25:39] New review: Tobias Gritschacher; "this needs rebase." [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/31004 [14:25:46] jeblad_WMDE: nah, let's stick to procedural, that worked fine for many decades and people are still using it anyways :) [14:25:53] those OOP hipsters! [14:25:59] is https://gerrit.wikimedia.org/r/#/c/31818/ going to be committed soon [14:26:08] I want my mediawiki files ;-; [14:26:08] jeblad_WMDE: if you want we can has a look at it together [14:26:38] I like GOTO [14:26:47] Cobol with objects and GOTOs! [14:26:55] * jeblad_WMDE <3 [14:29:37] Ajraddatz: aude merged it [14:29:39] New review: Tobias Gritschacher; "needs a rebase." [mediawiki/extensions/Wikibase] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/31831 [14:29:53] jeblad_WMDE: Cobol, you mad? whitespace! [14:29:55] it hasn't fixed the problem then :/ [14:33:30] where? [14:33:55] http://www.wikidata.org/w/index.php?title=MediaWiki:Cite_text as an example [14:34:05] all the mediawiki pages are still giving errors [14:37:55] New patchset: John Erling Blad; "(Bug 41581) Add an autocomment with label and value" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31831 [14:38:40] Ajraddatz: It is a working fix in master, probably it will be in the branch.. #hope [14:43:11] New review: Jeroen De Dauw; "Good initial step, could use some tidy follow-up though." [mediawiki/extensions/Wikibase] (master); V: 0 C: 2; - https://gerrit.wikimedia.org/r/31824 [14:46:01] aude: DanielK_WMDE_ I have sent a new patchset for the edit page tests https://gerrit.wikimedia.org/r/29973 [14:46:15] still show up the /usr/bin/diff3: No newline at end of file lines though :( [14:51:50] New review: Jeroen De Dauw; "53 failing tests with this change, 50 without. Might not be the change, but with master being so bro..." [mediawiki/extensions/Wikibase] (master); V: -1 C: 2; - https://gerrit.wikimedia.org/r/31824 [14:56:58] New patchset: John Erling Blad; "(Bug 41581) Add an autocomment with label and value" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31831 [15:01:01] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31831 [15:02:11] plop [15:02:26] hashar: commented on the editpage test thingy [15:04:11] New patchset: Anja Jentzsch; "(Bug 41667) remove onPageContentLanguage handler." [mediawiki/extensions/Wikibase] (0.2.x) - https://gerrit.wikimedia.org/r/31832 [15:05:47] New review: Anja Jentzsch; "Merging b9e3b3d21dccc8e26a6328b277683f002080b5b1 into 0.2.x" [mediawiki/extensions/Wikibase] (0.2.x); V: 1 C: 2; - https://gerrit.wikimedia.org/r/31832 [15:05:47] Change merged: Anja Jentzsch; [mediawiki/extensions/Wikibase] (0.2.x) - https://gerrit.wikimedia.org/r/31832 [15:11:10] DanielK_WMDE_: replied on https://gerrit.wikimedia.org/r/29973 [15:11:19] DanielK_WMDE_: that is all cause of the newlines madness :/ [15:12:18] hashar: just trim it all :) [15:12:59] :-] [15:15:32] folks! just a reminder that the next office hour (in german) will be in 45 mins in #wikimedia-office [15:15:42] the english one will be tomorrow [15:15:48] same place, same time [15:15:49] yay! office hours [15:16:03] :) [15:16:30] New patchset: Henning Snater; "JS refactoring: Edit button event" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31833 [15:18:59] New patchset: Jeroen De Dauw; "Added test group" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31834 [15:19:11] Change merged: Jeroen De Dauw; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31834 [15:23:32] New patchset: Jeroen De Dauw; "Reorganized wbsearchentities" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31824 [15:23:50] New review: Jeroen De Dauw; "Rebased in last PS" [mediawiki/extensions/Wikibase] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/31824 [15:23:51] Change merged: Jeroen De Dauw; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31824 [15:24:38] JeroenDeDauw: yes, pleas? [15:27:02] johl: https://gerrit.wikimedia.org/r/#/c/31824/ [15:27:04] please to has a look at ze commentz [15:29:18] jeblad_WMDE: so wants to haz a look at teh autocomments stuff together? [15:33:03] Change merged: Daniel Werner; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31830 [15:34:18] New patchset: Henning Snater; "JS refactoring: event for remove button" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31835 [15:41:01] New patchset: Jeroen De Dauw; "Follow up to 5fe2b5ec39b4b8c2812cd56ceec9cb814aebb438: minor tweaks" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31836 [15:41:22] New patchset: Jens Ohlig; "wbsearchentity continuation fixed." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31837 [15:46:18] doh! i'm an idiot :D office hour of course not in 15 mins but in 1h 45 mins -.- [15:49:20] Lydia_WMDE: that's not being an idiot, it's a smart way to get 90 minutes during which you can watch cats jump into boxes and fall over [15:49:29] +1 [15:49:36] JeroenDeDauw: haha [15:49:44] JeroenDeDauw: nah - time for a kde eV board call [15:49:51] but maybe.... [15:49:53] :P [15:52:21] Lydia_WMDE: you don't wanna watch cats? http://i.imgur.com/H9a2T.jpg [15:52:38] Sannita: i always do! :P [15:52:45] ...seven seconds away... #singing [15:52:56] Lydia_WMDE: you could also order a Döner, delivered to you by a flying robot drone! http://www.doenercopter.de/ [15:53:08] johl: lol [15:53:13] not a fan of döner... [15:53:15] -.- [15:55:25] San Francisco has a taco copter... [15:56:25] Lydia_WMDE: http://www.buzzfeed.com/summeranne/the-time-a-cat-adopted-four-baby-hedgehogs [16:01:02] New patchset: Jeroen De Dauw; "Improved array param documentation" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31839 [16:02:00] New patchset: Jeroen De Dauw; "Improved array param documentation" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31839 [16:05:43] New patchset: Jeroen De Dauw; "improved array param documentation" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/31841 [16:36:03] Is https://gerrit.wikimedia.org/r/#/c/31818/ actually applicable to the currently wmf deployment branch? [16:36:22] Looking at the conflicts, a lot of the code seems to not exist in the branch currently.. [16:42:40] New patchset: Daniel Kinzler; "Patch from Wikibase beta1 to beta2 (DO NOT MERGE!)" [mediawiki/extensions/Wikibase] (wikidata-wmfphase1beta) - https://gerrit.wikimedia.org/r/31849 [16:43:08] +25643, -11004 [16:43:10] Awesome ;) [17:03:07] Lydia_WMDE: when you're done with office hours, I've got a proposal for you [17:03:28] Sannita: shoot [17:04:13] Lydia_WMDE: are you available for a written interview (in english of course) about Wikidata, why Wikidata, why make an open/editable/free database and stuff? [17:04:34] I've been requested an article about WD, but I'd like to make it like an interview [17:04:48] sure [17:04:50] New patchset: Henning Snater; "JS refactoring: save and cancel event" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31851 [17:05:36] ok, so I'll let you know as soon as the boss confirms my view on the article [17:05:57] :) [17:06:03] best send questions by email [17:06:11] Lydia_WMDE: they already said it's ok :D [17:06:18] of course, I'd have sent it by mail [17:06:19] i'll be traveling from thursday to wednesday [17:06:27] cool [17:06:35] love those guys [17:06:35] i'll try to get to it as soon as i can [17:07:16] no prob, I have to prepare this thing by wednesday at the most, so I'll give you the questions asap [17:08:12] Change abandoned: John Erling Blad; "die?" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31650 [17:08:22] goo [17:08:22] d [17:08:31] (asap = tomorrow, probably) [17:14:57] New patchset: John Erling Blad; "(Bug 41580) Filter down the number of terms" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31853 [17:19:02] DanielK_WMDE_: What should we do about backporting https://gerrit.wikimedia.org/r/#/c/31818/ ? [17:20:09] I would say it should be backported [17:20:33] I would also like a pony [17:20:40] * Reedy pets jeblad_WMDE and gives him a carrot [17:20:56] wrinsk! [17:21:06] When cherry picking it into that branch, a lot of the code that is changed/removed isn't there already [17:25:14] alright this time i get it right :D [17:25:29] office hour (german) in #wikimedia-office in 5 mins [17:26:54] New patchset: Henning Snater; "JS refactoring: Removing EditableValue reference from EditGroup" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31856 [17:59:58] New review: Jeroen De Dauw; "Code looks good, although I'd get rid of the array_values. Not sure if we want to include the descri..." [mediawiki/extensions/Wikibase] (master); V: 0 C: 1; - https://gerrit.wikimedia.org/r/31853 [18:00:06] ;p [18:12:27] * Reedy is creating an XML export of wikidatawiki [18:13:42] Reedy: I am curious if this will work. It should, but I am not sure we ever tested it [18:13:50] Special:Export works [18:14:06] [01:14:54] Full history for Q1-Q100.... 76.1MB [18:14:06] [01:15:10] 7z turns that into 887KB [18:14:22] [01:08:09] {"label":{"en":"Barack Obama","fr":"Barack Obama","ar":"\u0628\u0627\u0631\u0627\u0643 \u0623\u0648\u0628\u0627\u0645\u0627","ru":"\u0411\u0430\u0440\u0430\u043a \u041e\u0431\u0430\u043c\u0430","nb":"Barack Obama","it":"Barack Obama", [18:14:22] "de":"Barack Obama","be-tarask":"\u0411\u0430\u0440\u0430\u043a \u0410\u0431\u0430\u043c\u0430","nan":"Barack Obama","ca":"Barack Obama"},"description":{"en":"President of the United States of America [18:14:38] It just looks horrible [18:14:51] I logged https://bugzilla.wikimedia.org/show_bug.cgi?id=41790 to this extent [18:15:37] thanks [18:15:37] Obviously, JSON wrapped in XML is very compressible [18:15:44] * aude vomits :o [18:15:56] I'll see if I can put this dump somewhere in the meantime [18:16:02] sure we can make it better [18:16:03] http://dumps.wikimedia.org/wikidatawiki/20121026/ [18:16:10] saw that one [18:16:11] I think we have a few weeks before another one is due to be started [18:16:36] the issue is just XML and Unicode serialization, I'd say [18:16:44] which file is it? [18:16:59] * aude only saw the main page there [18:17:08] I used Special:Export on the wiki to export obama [18:17:14] ah, okay [18:17:14] and then another to do q1-100 [18:17:17] nice [18:17:31] Currently using dumpbackup on the cluster to generate a more up to date one [18:17:38] 2012-11-05 18:17:33: wikidatawiki (ID 4476) 11216 pages (32.3|130838.7/sec all|curr), 312900 revs (899.9|1166.5/sec all|curr), ETA 2012-11-05 18:17:58 [max 335657] [18:17:52] Takes about 10 minutes it seems [18:18:02] Done [18:18:08] Place your bets on the XML size [18:19:48] hmmm [18:19:55] that's without the changes table [18:20:04] 650 mb [18:20:25] one CD [18:20:25] Indeed.. I'm not sure if that'd back it up anyway [18:20:26] And then also guess 7zipped size [18:20:26] This is a fun game ;) [18:20:46] 10 MB? :) [18:21:36] lol [18:21:46] ehm [18:22:18] 550 mb, just a rough guess [18:22:19] :P [18:23:42] oh wait [18:23:43] 10 MB for the 7zipped, 650 MB for without [18:23:50] it will be less more [18:24:28] I think around 10 MB, jist like denny [18:24:31] reedy@fenari:/tmp$ du --si wikidata.* [18:24:31] 27M wikidata.7z [18:24:31] 1.4G wikidata.xml [18:24:37] oooh [18:24:51] wow [18:26:39] osm = 18 gb compressed as pbf, 23 gb as bz2 and ~300 gb uncompressed [18:26:39] for comparison [18:27:14] I'm not even sure what compression level 7z used [18:27:28] * aude no idea how big the wikipedia en dump, but know it comes in parts [18:28:16] aude: there is also one that recombines the parts [18:28:16] sure [18:28:19] aude: not sure how many of these downloads are ever completed :) [18:28:32] true [18:28:50] The current revisions of NS 0 are usually large enough ;) [18:29:05] hihi [18:29:30] aude : Recombine all pages, current versions only. [18:29:30] enwiki-20121101-pages-meta-current.xml.bz2 17.1 GB [18:29:42] dcrochet: not bad [18:30:28] the osm database is managable, as long as you have enough space for it [18:32:51] https://www.strongspace.com/shared/l7s8vq51lg < If anyone is interested [18:41:10] nomnomnom [18:41:10] data [18:41:33] lol :P [19:04:53] plop plop [19:10:00] * Lydia_WMDE hands Nikerabbit a carrot [19:15:39] hello [19:16:42] what's the use of namespace : Query, Property and TimedText ? [19:17:32] Grondin: query and property are coming later in wikidata [19:17:48] and TimedTExt? [19:17:48] too? [19:17:53] timedtext is for the video/media extension and not sure why it's on wikidata [19:17:58] shouldn't be [19:18:11] it's new on wikimedia sites [19:18:21] like commons [19:18:22] ok [19:18:31] thanks' [19:18:47] we're just reserving the property and query namespace for now [19:18:54] wut [19:18:57] but those come in phase 2 and 3 [19:20:22] aude: where can I read something more about TimedText, please? [19:22:47] Sannita: http://www.mediawiki.org/wiki/Extension:TimedMediaHandler [19:22:51] it's for commons, not wikidata [19:23:02] aude: ty [19:23:37] it think it was an error to enable it for wikidata, especially since there are no uploads [19:25:16] true, do we need uploads right now? I don't think so [19:25:28] no [19:25:33] we can upload to commons [19:26:09] exactlt [19:26:11] *tly [19:26:40] Lydia_WMDE: thanks [19:30:09] DanielK_WMDE_: poke [19:30:25] JeroenDeDauw: *eek* [19:30:25] Where to hook in if I has a namespace in which pages of 2 content models reside? [19:30:43] Where the content model can be derived from the title [19:31:00] (Looking at how to update EP to use CH) [19:31:00] JeroenDeDauw: ContentHandlerDefaultModelFor [19:31:16] it's called from ContentHandler::getDefaultModelFor( Title ) [19:32:10] DanielK_WMDE_: awesome [19:34:42] Daaaah https://www.mediawiki.org/wiki/Manual:Hooks/ContentHandlerDefaultModelFor [19:35:23] isn't a bot supposed to be creating these from hooks.txt= [19:35:23] ? [19:35:32] or does that just do the hook list, not the individual pages... [19:41:06] DanielK_WMDE_: no idea [19:41:10] thought it was done by magic [19:41:13] adding it manually now [19:49:45] DanielK_WMDE_: manual magic? neat! [19:53:49] JeroenDeDauw: it'S the wiki way ;) [20:12:42] Denny_WMDE: https://www.wikidata.org/wiki/Wikidata_talk:RFC/Inclusion_of_not-article_pages [21:22:03] Hi. Why I get "Our servers are currently experiencing a technical problem." when trying to look on some pages in Mediawiki namespace? [21:22:13] Known issue [21:22:16] [[MediaWiki:Mainpage]] [21:22:16] 10[1] 10https://www.wikidata.org/wiki/MediaWiki:Mainpage [21:22:58] Reedy: Aaah. Thanks. [21:23:02] https://bugzilla.wikimedia.org/show_bug.cgi?id=41667 [21:36:10] /!\ data #14000 saved /!\ [21:37:36] woohoo [22:00:12] !no [22:00:42] !no is https://bit.ly/dennyno [22:00:42] Key was added [22:00:46] !no [22:00:46] https://bit.ly/dennyno [22:00:49] * JeroenDeDauw hides [22:01:12] !no | Denny_WMDE [22:01:12] Denny_WMDE: https://bit.ly/dennyno [22:01:28] :D [22:06:36] Change abandoned: John Erling Blad; "Sombody else should fix this" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31853 [22:08:03] Change abandoned: John Erling Blad; "(no reason)" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/15868 [22:08:42] Change abandoned: John Erling Blad; "(no reason)" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/25547 [22:12:58] New review: Jeroen De Dauw; "? :(" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31853 [22:58:05] Change restored: SPQRobin; "Maybe I wasn't clear. You can keep this change and just include the doc update and the removal of "L..." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31650 [22:58:19] New patchset: SPQRobin; "(Bug 41667) Fix for onPageContentLanguage" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31650 [23:00:41] Change abandoned: SPQRobin; "Oh, never mind. It got removed in change 31818." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/31650 [23:07:14] Hello peeps. What up?