[18:00:14] o/ [18:00:25] \o [18:00:30] #startmeeting Wikidata office hour [18:00:30] Meeting started Tue Nov 14 18:00:30 2017 UTC and is due to finish in 60 minutes. The chair is Lydia_WMDE. Information about MeetBot at http://wiki.debian.org/MeetBot. [18:00:30] Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. [18:00:30] The meeting name has been set to 'wikidata_office_hour' [18:00:40] Hey folks :) [18:00:49] how are you? [18:00:57] who's here for the wikidata office hour? [18:01:07] i am! [18:01:12] * Lucas_WMDE is alive and well :) [18:01:19] I'm doing good and I'm here for that, surprisingly enough. [18:01:28] \o/ [18:01:59] Ok then let's get the party started [18:02:25] let's take a look at what happened over the last 3 months or so since the last office hour [18:02:59] we've done quite some usability work [18:04:06] specifically we've improved the search in the entity selector. that should now be considerably better. it can however still be tweaked more. let me know if you find cases where the ranking is still not good. This is the ranking for the results when you search for an item in the top-right search boy or when adding a new statement. [18:04:56] we rolled out the constraints gadget that shows you when a statement is violating a constraint so this is easier to spot and hopefully fix than it was previously [18:05:20] give it a try if you have not already and tell us what you think - specifically me and Lucas_WMDE [18:05:59] we've also removed language fallback indicators for language fallbacks as these were pretty annoying when using wikidata in swiss german for example [18:06:20] (for related languages) [18:06:50] on top of that we've done a lot of talking to editors to get an overview of all the ways people are editing wikidata and where their biggest issues are [18:07:07] Lucas_WMDE: heh right - sorry, forgot that important part :D [18:08:11] I am here as well [18:08:24] in this area the most important thing coming up is building out the constraint gadget and then turing it into proper code so we can turn it on for everyone as well as looking into shape expressions to see how they can be used to make the constraints checks even more powerful [18:08:33] * spinster says hi [18:08:52] right then there is the big topic of structured data on commons [18:09:04] :D [18:09:09] ;) [18:09:10] we've continued the work on the underlying technology - specifically multi content revisions [18:09:20] still a lot of work to do there but it is making progress [18:09:39] and spinster will be doing an office hour next week to talk more about all the other work she and the team have been doing [18:09:56] Yep, 21 Nov @ 18:00 UTC [18:10:02] oh, cool! hadn’t heard about that [18:10:25] go and attend if multimedia files and structured data are your thing :D [18:11:05] alright next was the articleplaceholder. We've rolled it out to two more wikis: sqwiki and bnwiki. [18:11:24] If you know a wiki that would like to have the articleplaceholder set up let me know. [18:11:33] (unfortunately can't roll it out on large wikis yet) [18:12:16] we've also put quite some work into support for all the other sister projects. for example there is now the wikidata concept monitor: http://wdcm.wmflabs.org/ [18:12:35] \o/ [18:12:37] it helps you better understand how the different projects are using wikidata's data. [18:12:53] if you find something interesting/weird/funny/... in there i'd love to hear about it [18:12:56] No font fallback :( [18:13:06] sjoerddebruin: ? [18:13:30] There is no font stack in the CSS, just the font Liberation Sans that might not be on every computer. [18:13:42] ah will let Goran know [18:14:13] besides that we also did a lot of fire fighting to make change dispatching better [18:14:36] this is what is responsible for notifying all the wikipedias and other projects about a change on wikidata that is relevant for them [18:15:24] because of a lot of activity on wikidata we've been reaching the limit of what the current system could handle and have made some changes that buy us time but we'll need to look into more fundamental changes it seems [18:15:39] Things are doing well for 12 days now. :) https://grafana.wikimedia.org/dashboard/db/wikidata-dispatch?refresh=1m&orgId=1&from=now-14d&to=now [18:16:02] Is there any more thoughts about what those fundamental changes will be [18:16:13] (feel free to ignore that question if its out of scope for the meeting) [18:16:30] as part of that we also started making the system more smart about only showing changes that are really relevant for that wikipedia or other project. before there was way too much in the watchlist and recent changes that wikipedia editors were not interested in [18:16:50] unfortunately we need to roll that out slowly so we can monitor the performance [18:16:58] so right now it is only on greek wikipedia and a few others [18:17:04] sjoerddebruin: it has been 1️⃣2️⃣ days since the last dispatch incident :D [18:17:25] bawolff: not really. we'll have to figure that out and probably have a discussion at the dev summit about it [18:17:45] ok. I'm very interested in what the long term plan will be [18:17:51] *nod* [18:17:55] me too :D [18:18:00] Its a complicated issue. Anyways, look forward to hearing about it at dev summit [18:18:08] it is [18:18:16] @Lydia Any indication on how much time was bought? [18:19:01] Andra: it is really hard to say because we've been making all kinds of tweaks that might have small or ig impacts. also it is highly dependent on the number and kind of edits happening on wikidata [18:19:26] I'd say months rather than years though [18:19:43] :( [18:19:51] jep indeed [18:20:04] but then again: growing pains ;-) [18:20:15] Indeed [18:20:34] alright then on to the query service [18:20:52] we've done more work on expereimenting with letting people click together queries without knowing sparql [18:21:08] we've added support for showing the geoshapes from commons in the map visualisation [18:21:37] and you can now easily get code snippets for your query to integrate it in your programs in all kinds of programming languages [18:22:18] Yes I have seen that, NICE! [18:23:06] Another thing we worked on was adding full URIs for external identifiers to our RDF exports. this means we'll hopefully soon show up in the Linked Open Data cloud visualisation. i'm looking forward to seeing how we'll be positioned there. [18:23:20] http://lod-cloud.net/ [18:23:34] Good evening. Probably like if a nuke went off [18:23:42] lol [18:24:54] and then we also released a docker image for wikibase which hopefully makes it easier for people to start coding on wikibase and in the future also set up their own instance of wikibase. [18:25:31] i'm really looking forward to the time when we'll have a nice ecosystem of many different wikibase installations out there all connected with wikidata but concentrating on their specialty data [18:26:27] I like the improved search Lydia_WMDE! And the patch to solve pasting seems to be up for deployement too. Just wondering, when do we get strings in search? :-) [18:26:39] multichill: \o/ [18:26:53] strings in search: that is a good question. SMalyshev is probably best to answer that [18:27:21] which brings me to the last big development topic: lexicographical data and wiktionary [18:28:04] that is what took most of the time over the last quarter and we now have a demo system at http://wikidata-lexeme.wmflabs.org of the current state. we've demoed that at Wikimania and have been working on the feedback since then. [18:28:24] right now most of the edits you make there are not persistently stored. that is what we are tackling at the moment [18:29:03] ok on to all the other important things that happened: [18:29:31] there is now a page to help other wikimedia projects use the data in wikidata: https://www.wikidata.org/wiki/Wikidata:How_to_use_data_on_Wikimedia_projects [18:29:55] and another one that adresses common questions/fears/... at https://www.wikidata.org/wiki/Wikidata:Wikidata_in_Wikimedia_projects [18:30:07] both of these could use some more help if you have some spare time [18:30:43] a bunch of new tools have been developed. [18:31:00] Lydia: Maybe worth mentioning I am Currently brainstorming with Rhizome and the Finish name project on building such an ecosystem of collaborating Wikibases [18:31:06] https://tools.wmflabs.org/wikidata-todo/sparql_rc.php lets you enter a sparql query and then gives you edits related to the result set. [18:31:22] so you can for example query for all politicians in a country and then see what edits happened to their items [18:31:37] Andra: awesome. [18:32:15] here is another one: https://tools.wmflabs.org/wikidata-todo/wd_edit_stats.php [18:32:35] it lets you enter a sparql query and a time range and then gives you statistics about the edits in that timeframe [18:32:48] \o/ already tried and adopted ! [18:33:20] https://angryloki.github.io/wikidata-constraint-violations/ shows you how the constraint violations for a specific property are developing over time. very useful to see the impact of a cleanup drive or monitor imports [18:33:44] Lydia here is the Gdoc collecting the ideas: https://docs.google.com/document/d/1ax3um4KJWXK5TZHQjaphpXH-bBonLo9D1RfuLd6IWB4/edit?usp=sharing [18:33:49] thanks! [18:34:16] here is another helpful tool for cleanups: https://lucaswerkmeister.github.io/wikidata-ontology-explorer/ [18:34:31] that lets you visualize parts of the ontology and see where things are not right [18:34:38] Also I can confirm that the docker image really works. I have deployed several since its release [18:34:44] :D [18:34:45] sweet [18:34:48] \o/ [18:35:21] There is more good stuff in the birthday presents section on https://www.wikidata.org/wiki/Wikidata:Fifth_Birthday [18:35:23] check it out [18:35:39] oh and there is now an alexa skill for Wikidata :D https://www.amazon.de/metaphacts/dp/B0745KLCFX/ [18:36:25] I think some people first need to load the German skill before they can use that one Lydia [18:36:26] wow, wikidata is 5 [18:36:31] If you speak a language that is not well-represented on wikidata you should totally join the Wikidata flashmop: https://www.wikidata.org/wiki/Wikidata:Flashmob [18:36:35] It seems just yesterday it was announced [18:36:51] multichill: i thoguht it also works for english now [18:36:51] Dutch is so under-represented... [18:36:59] bawolff: :D i know right? [18:37:06] growing up so fast! [18:37:09] sjoerddebruin: :P [18:37:25] Alexa is still mainly a english/german thing, isn't it? [18:37:43] i'm not sure how many languages they cover atm [18:37:49] but definitely english and german [18:38:01] (individual skills might know less) [18:38:15] speaking of 5 years old! [18:38:35] we celebrated Wikidata's 5th birthday with a whole conference and I really really loved it [18:38:52] * VIGNERON loved it too [18:39:15] Yes it was an awesome event [18:39:23] https://www.wikidata.org/wiki/Wikidata:Fifth_Birthday has lots of nice texts, presents and more [18:39:25] <3 [18:39:35] https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Documentation/Proceedings has documentation from the conference [18:39:45] https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Documentation as well [18:39:54] that has videos, slides, notes etc [18:40:00] It was also the first conference I ever attended with really no Internet issues whatsoever [18:40:12] *lol* [18:40:15] very good [18:40:52] that was actually one of our big worries but the venue was awesome and super helpful (Tagesspiegel, a local publishing house) [18:40:56] I was wondering if there are plans to upload the video registrations to Commons? [18:41:00] yes they will be [18:41:06] :-) [18:41:23] not sure when though (Léa's department and she's on vacation now so I guess when she is back= [18:41:39] She surely deserves that vacation :D [18:41:44] :D totally! [18:41:54] until then you can get them from the chaos computer club [18:42:09] who were fabulous in recording everything [18:43:45] alright and my last part is blog posts and papers [18:44:11] here are some that talk about using wikidata's data to build cool things: [18:44:12] https://blog.wikimedia.de/2017/07/25/data-partnerships-with-wikidata-beatunes/ [18:44:12] https://blog.mapbox.com/exploring-the-world-with-wikidata-and-openstreetmap-30f1bfe954d3 [18:44:12] https://blog.wikimedia.de/2017/08/21/data-partnerships-in-wikidata-project-durchblick/ [18:44:29] and a paper about references on Wikidata: https://eprints.soton.ac.uk/412922/ [18:44:59] and one analyzing wikidata's language coverage: http://opensym.lero.ie/wp-content/uploads/2017/08/a14-kaffee.pdf [18:45:24] alright. that's it for the reporting part of the office hour [18:45:35] on to comments/questions/other topics you want to talk about :) [18:45:54] Lydia: There was an issue with the Gdoc share setting its open for all to comment now [18:46:04] thanks! [18:46:59] thank you for the updates, Lydia! [18:47:07] you're welcome! [18:47:18] Glad to see some focus on stuff on Wikidata itself again, it was a lot of non-visible things lately. [18:47:53] :) [18:48:44] Can I ask a question on the Docker image here? [18:48:56] all questions allowed :D [18:48:58] so go ahead [18:49:14] * Lydia_WMDE wonders if addshore is around [18:49:22] thanks Lydia_WMDE! Like you, I'm looking forward for the Linked Open Data cloud visualisation update ("One Database to link them all" ^^) [18:49:29] Idle for like one minute ;) [18:49:31] We are considering using the image as a test box for our bots. [18:49:36] Envlh: :P exactly [18:49:52] I'm actually wondering if we should ask people to test more on test.wikidata.org and local images... [18:49:57] I still don't like the Wikidata sandbox :P [18:50:22] sjoerddebruin: i guess it depends on what kind of tests [18:50:31] large scale import tests: definitely! [18:50:34] but this would require that the property number to be persistent, but now it seems that the p number are assigned automatically, can this be made manually? [18:50:40] If you want to do some tests with clients, yeah. [18:50:42] bigger modeling: probably [18:50:56] Sjoerd the issue with sandbox is that sometimes tests involves actual item and property numbers [18:51:06] Lydia_WMDE: yes! [18:51:16] addshore: see Andra's question please :) [18:51:22] *reads up* [18:51:40] "We are considering using the image as a test box for our bots." >> Sounds like a good idea [18:51:44] It is possible to exclude the sandbox items from queries, unless you've used some kind of service? [18:52:28] "but this would require that the property number to be persistent, but now it seems that the p number are assigned automatically, can this be made manually?" >> As in you would require the same property on the docker container each time you use it? [18:52:28] envlh/Lydia: It has been genewiki's slogan from the start (making wikidata the hub in the (biological) linked data cloud) ;) [18:52:36] Oh right: [18:52:37] Or the same property Ids as on wikidata.org? [18:52:54] addshore: the same property ids as on wikidata.org [18:53:04] and even in some case on item numbers [18:53:10] When can we expect better property suggestions? :) [18:53:11] but maybe that is pushing it [18:53:17] So, that isnt really possible right now [18:54:00] :( [18:54:19] I did plan on incorporating https://github.com/filbertkm/WikibaseImport into the docker image, but then https://github.com/filbertkm/WikibaseImport/issues/14 is an issue that would still need fixing [18:54:20] * DanielK_WMDE_ missed the office hour [18:54:33] Andra: you could however just have a mapping in some configuration file? [18:54:35] sjoerddebruin: since we need to concentrate on the change propagation improvements i fear that is on the backburner for a while :/ [18:54:54] I've seen some refactoring though, so it's something. [18:55:01] yeah [18:55:41] addshore: WikibaseImport *maps* IDs, it doesn't preserve them. [18:56:04] addshore: to preserve, you can use a plain XML import. [18:56:27] why not just do that? [18:56:35] DanielK_WMDE_: yup, although the plain XML dump then has issues with the hook up to the query service [18:56:52] or, at least, it isn't as simple as it sounds / I would need to document more stuff around that area [18:56:56] addshore: it does? like what? [18:57:24] With the simple playground docker images, you create items, they are in the query service via the updater, if you import a dump, that wont happen [18:57:25] yea, the xml import isn't safe, it produces a lot of red links, and it should only be done if no ids are created locally. [18:57:45] so, it's not for general use. that'S why it's deisabled per default [18:57:48] for initializing a testing environment, it should be fine [18:58:05] WikibaseImport also has a task filed for json dumps https://github.com/filbertkm/WikibaseImport/issues/7 which would be great [18:58:20] ad WikibaseImport creates recent changes entries, which means the updater picks up the changes [18:58:30] addshore: the query services needs to be re-loaded from a dump after na xml import... [18:58:36] DanielK_WMDE_: yup [18:58:47] addshore: no, better: just provide an RDF dump along the XML dump. import both [18:59:09] Andra: if you don't want to use the query service bits of the docker stuff you can try just loading XML dumps of the items / properties that you need [18:59:39] addshore: actually the query service is essential in the test environmnet [18:59:58] DanielK_WMDE_: well, the rdf dump still has to be munged, it can't just be thrown into blazegraph in 1 go. [19:00:30] DanielK_WMDE_: you can dump zips of the all the mysql and blazegraph data, and send them to someone else to load into their own docker containers ;) [19:00:38] Andra: are you editing all relevant items? then that would be enough to get them into the query service and you should be ok with an xml import [19:00:55] Lydia: Indeed! [19:01:22] Andra: It might be worth you filing a ticket under the #wikibase-containers project on phabricator to discuss further, as I know you are not the only person with this usecase. [19:02:05] infact Andra https://phabricator.wikimedia.org/T180216 [19:02:51] addshore: Will do. We are also trying to see if we can get a workshop going on this: https://docs.google.com/document/d/1ax3um4KJWXK5TZHQjaphpXH-bBonLo9D1RfuLd6IWB4/edit [19:03:35] sweet [19:03:44] any other topics/questions? [19:04:17] Some help cleaning up https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/Mandatory_constraints/Violations would be great. :) [19:04:18] no more topics from me. thanks for the office hour! [19:05:01] alright then let's wrap that up and all help sjoerddebruin clean up constraint violations :D [19:05:03] I warmly invite everyone to next week's Structured Commons IRC office hour again! Nov 21, 18:00 UTC [19:05:09] thanks everyone for coming! [19:05:40] Thanks all [19:05:46] spinster: will it be on #wikimedia-office ? [19:05:57] Yes [19:06:13] good, I'll be there ;) [19:06:14] * Lucas_WMDE waves goodbye [19:06:27] #endmeeting [19:06:27] Meeting ended Tue Nov 14 19:06:27 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) [19:06:27] Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-11-14-18.00.html [19:06:27] Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-11-14-18.00.txt [19:06:27] Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-11-14-18.00.wiki [19:06:28] thank you all! [19:06:28] Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-11-14-18.00.log.html [19:33:48] Andra: google doc looks good. let me know if you need anything i can help with