[08:42:33] https://grafana.wikimedia.org/dashboard/db/wikidata-dispatch?refresh=1m&orgId=1&from=now-12h&to=now :/ [08:42:51] Wait, wikidata itself is 37 minutes behind on itself? [10:03:43] hello [10:51:48] I don't suppose anyone knows how to detect when event handlers get added in js? trying to work out how to remove the scroll effects in the query service [10:52:15] $(window).unbind("scroll"); works in the console, but if I tell it to run that on page load, it doesn't work, I assume because it's too soon [10:53:02] nikki: your best option for now is probably to clone https://github.com/wikimedia/wikidata-query-gui and open the webpage locally [10:53:08] should work after `npm install` [10:53:26] that doesn't fix links I click on though :/ [10:54:15] yeah :( [10:54:25] we really need to deploy the fix soon [10:57:19] unless it's being removed again, that won't get rid of the scroll effects though :P [11:20:30] well you could check out an older version… but that’s not a great long-term solution [11:21:08] you could also help us to figure out how to make the scrolling less annoying :) [11:21:27] I personally think in general it’s quite nice, being able to instantly get to the query result [11:21:41] especially for the map view, because that captures the scroll event [11:21:50] so scrolling to get the full map into view used to be super annoying [11:26:23] the problem is that what I consider usable does not seem to match what other people consider usable :( [11:27:32] I can't imagine any scroll effects that would be an improvement on the way it works by default, because I know how scrolling works normally and anything that's different will be unexpected behaviour [11:32:05] and I don't want it to hide the query and run button just because I want to scroll down slightly, and the way it scrolls more than I told it to makes it hard to keep track of where I'm looking, because things are moving of their own accord [11:36:58] the best solution from *my* point of view is to stop trying to do fancy scroll effects, but I expect everyone else to think the fancy scroll effects are cool and useful, because that's usually how it works [11:40:42] hence trying to find a way to remove the things that bother me without insisting that other people can't have their fancy effects :P [11:41:38] well we could always check some cookie before adding the scroll handler and skip it if the cookie is set [11:41:57] and then you could run `Cookies.set('dont-scroll', true)` or something like that in the console (once) [11:42:19] but that would really only be useful to you, the one person who knows about it :) [11:42:25] (and anyone else who reads the source code) [11:43:24] yeah, if there's going to be options like that, it'd make sense to expose them in the interface [11:47:18] I do love it when I find options like that (I was so happy when I found one in duckduckgo to stop the header appearing and disappearing... I even sent them some feedback saying how pleased I was), but I feel bad asking for features if they're just for me (at least so far I haven't seen other people complaining about too many effects) [11:48:31] and so far most of the things that have been distracting have been solved with liberal use of "display: none"... can't exactly do that with the query box or the results though here XD [12:00:39] I do have a page locally which I made a long time ago which I can use (it has no features, unless you consider "more purple" a feature) and that works for most queries I want to do, but that's no help when I click a link [13:55:37] dispatch lag already over 2 hours again, with 50k pending for enwiki -.- [14:01:25] looks like all the bots and quickstatements users are back editing as normal [14:02:48] so I can't say I'm surprised... [14:04:01] of course, if we had automatic descriptions, we wouldn't need to run bots and quickstatements constantly to add the same description to tons of items, but noooo, we were told manual descriptions are better [14:07:46] I do find it ironic that the first goal of wikidata was to centralise interwiki links so we didn't need an army of bots to maintain them across languages, and now we need a different army of bots to maintain something else across languages [14:25:03] even if we only supported automatic descriptions for categories, templates and disambiguation pages, we have 6 million of those [14:28:47] translating three strings into another language and getting 6 million descriptions instantaneously for that language would be awesome, way better than finding the right bot and getting them to do 6 million edits [14:28:55] I can dream :P [14:29:34] somebody should make dream comes true [14:30:36] yeah, that'd be nice too [14:35:48] nikki: I definitely agree with you on descriptions, it really does not make sense to run bots to fill them [14:37:03] it is useful to be able to input descriptions manually, but there should be a fallback on autodescriptions when the field is empty [14:37:09] * nikki nods [14:55:17] FYI https://twitter.com/matej_suchanek/status/890218059323117568 [15:40:14] dispatch lag over three hours now [15:40:39] and sjoerddebruin hasn’t even said anything about the lowered dispatch lag on project chat yet as far as I can see [15:40:49] so presumably all those bot operators were also watching grafana [15:41:00] but now they’re not noticing that dispatch lag is rising again? [15:41:04] I don’t get it [15:42:31] they probably assumed it was safe and stopped checking [15:42:45] they don't know that it's rises so quickly again [15:43:15] https://grafana.wikimedia.org/dashboard/db/wikidata-edits?refresh=1m&orgId=1&from=now-24h&to=now agrees [15:45:01] if you look at RC, you will see those users who were asked to stop... [15:48:31] I’m looking at http://wikidata.wikiscan.org/?menu=live&filter=all&sort=edit&date=24&list=users [15:49:38] looks like Mr.Ibrahembot is throttled now, so at least there’s that :) [17:43:04] PROBLEM - puppet last run on wdqs1002 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [18:10:24] RECOVERY - puppet last run on wdqs1002 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [18:22:43] the number of edits is higher than the days before [18:22:51] http://wikidata.wikiscan.org/gimg.php?type=edits&date=201707&size=big [19:44:52] the number of edits is one thing, has the amount of time increased for enwp to get all the changes in ? [20:36:10] DanielK_WMDE_: Do you think it's time to throw away the legacy MySQL dispatch locking [20:36:27] The lockmanager version can also use MySQL as backend, so there's hardly a point [20:39:03] I like the mention of dispatch in those sentences :P [20:40:05] I would like to refactor/ simplify it so that we can make it better and faster after [20:40:45] This is not really related to the performance improvements we plan, but it's just not needed IMO [21:51:14] DanielK_WMDE_: ping about https://gerrit.wikimedia.org/r/#/c/339575/ [21:53:22] hi everyone [21:53:50] i found a duplicated item [21:53:57] and proceeded to merge them [21:54:11] cool [21:54:18] but.. i also checked the user that created it [21:54:30] and picking 3 random items [21:54:38] i found out that they where all duplicates... [21:54:44] what can we do? [21:55:22] !admin [21:55:49] https://www.wikidata.org/wiki/Special:Contributions/Mr.Ibrahembot [21:59:04] all? [21:59:08] Zeroth_: tell the user on https://www.wikidata.org/wiki/User_talk:Mr.Ibrahembot ? [21:59:28] all that i checked sjoerddebruin [21:59:39] there are hundreds of edits [21:59:51] i saw only 5 or 6 at random [22:00:37] I checked one, it has the same name as the municipality. But most municipalities aren't the same size as the settlement in them? [22:00:58] Also: settlements are stable. Municipalities can always merge etc. [22:02:01] are you saying they are different entities? [22:02:12] and that one only exists in wikipedia in ceb? [22:02:36] I think so. [22:02:39] I saw the same happening for Belgium. [22:02:49] * hoo|away calling it a day… good night [22:02:55] hoo|away: sleep well <3 [22:03:03] Thanks :) [22:03:20] settlements aren't stable, they grow and absorb other places [22:03:38] Well, they are more stable than municipalities. [22:04:13] We still have various Dutch items combining settlement and municipality, that shouldn't be. [22:04:30] i think that it's highly unlikely that thouthands of entities of places only existed in wikipedia in ceb [22:04:31] municipalities are way easier to work with for me, they actually have borders [22:04:48] what makes you say that they are different entities? [22:05:07] Can you give me links of things you think are duplicates? [22:06:32] https://www.wikidata.org/wiki/Q33438754 [22:06:37] https://www.wikidata.org/wiki/Q396566 [22:06:48] (again, random item) [22:07:11] the coordinates are way off, though [22:07:37] yes [22:07:44] im seeing thant [22:07:46] let me check [22:08:36] Q396566 already contains a ceb link, with a weird disambugation though [22:08:46] Seems about a river, wtf [22:08:52] Zeroth_: if ceb wikipedia decided to import a catalog of municipalities by bot.. in the past .. it might be plausible [22:09:14] But EmausBot did weird stuff here: https://www.wikidata.org/w/index.php?title=Q396566&action=history [22:09:19] a bot imported a load of geonames stuff to cebwiki [22:09:42] it's similar with all the "water holes" and "waterfalls" and "mountains" that only exist in "sv" [22:09:42] im worried that its not filtered [22:09:52] i mean, that it's not checked [22:09:55] for duplicates [22:10:01] Old interwiki's... https://ceb.wikipedia.org/w/index.php?title=Golinja_%28suba_sa_Krowasya%2C_lat_45%2C32%2C_long_16%2C02%29&type=revision&diff=9785620&oldid=9772769 [22:10:02] or items that already exists [22:10:23] so if you play wikidata game to identify what is a person and what isn't you will notice those few languages that seem to have ALL the geonames that aren't categorized yet [22:11:12] I still don't think those two are the same, but there was a wrong cebwiki link on the second one... [22:11:50] another example [22:11:50] https://www.wikidata.org/wiki/Q33439785 [22:11:55] https://www.wikidata.org/wiki/Q3045933 [22:12:25] ir refers to the same town [22:12:29] it* [22:12:36] and has the same coord. (or almost) [22:12:36] That one seems plausible. [22:12:41] But you need very advanced bot programming to detect this. [22:12:44] at least the items have statements... if someone is going to create duplicates, I'd rather have duplicates we can *find* [22:13:13] They do have the same geonames ID, so they will show up in reports [22:13:21] lots of those bot-created-from-geonames articles have wikidata items with lots of statements still missing :( [22:13:49] At least this bots adds statements... [22:13:57] so the process is upload bunch of items and check later it they are duplicated? [22:14:34] (im genuinely asking) [22:15:14] yikes https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/P1566#.22Unique_value.22_violations [22:15:15] and another [22:15:18] https://www.wikidata.org/wiki/Q33439760 [22:15:22] https://www.wikidata.org/wiki/Q3449824 [22:15:27] that page has been a mess ever since lsjbot started :( [22:15:36] I tried working on it, but the bot was too much for me [22:15:38] again guys, im picking items at random [22:16:00] Yeah, at least the bot should have a geonames check [22:18:19] I'm not sure that those two are the same either, one is a town which is like municipalities elsewhere (i.e. can contain multiple settlements, including the main one which the town is named after) [22:18:57] population seems plausible though [22:19:14] (haven't checked closely, so maybe it is... but it's difficult for a bot to know) [22:20:00] and we have lots of badly-assigned geonames ids too, so even checking existing ids is likely to lead to sitelinks being added to the wrong items [22:20:09] it's all a mess :( [22:20:43] nikki, the arts. in ceb says "town" [22:20:46] in all of them [22:21:14] "lungsod" (means town) [22:21:31] so, all the items created are of towns, not municipalities [22:23:28] the word town can mean both a type of settlement and a type of municipality [22:24:27] "town limits" in US [22:26:41] guys, don't you think its more plausible that a bot its uploading a dataset withouth checking the geonames than all of the other conditions you are saying are met? [22:27:26] please check by your own means the dataset upload [22:27:41] like http://www.geonames.org/4960133/cardville.html claims to be in the town of greenbush, so because Q3449824 is marked as the town, we would mark that settlement as being in Q3449824... but Q33439760 is for the settlement in the town [22:29:16] I do think it is creating some duplicates, but I don't see any way of avoiding that, because our existing geonames ids are full of mistakes [22:30:13] ok [22:41:43] sjoerddebruin: if you think the unique value one is bad, have you checked the no country one? :P [22:41:49] :( [22:43:13] anyone know what the criteria for accepting external ID properties are, if there are any? [22:43:39] I don't think we have any formal criteria [22:43:53] how about informal criteria? [22:44:09] people like having formatter urls [22:44:54] stable ids are good, datasets with lots of ids are good, people don't like things which aren't publicly accessible [22:45:26] any problems with "too much like URLs instead of identifiers"? [22:45:44] (ie "/people/BobShmoe") [22:46:23] I don't like things which are url fragments, but I'm pretty sure we've added some like that [22:47:00] okay then. thanks. [22:48:22] plenty of property examples with slashes in them: http://tinyurl.com/yak4a2nq [22:48:27] even multiple [22:49:41] good to know [22:50:17] wait what where did the option for showing 1000 results go in the dropdown? [22:50:33] it's definitely there in my older tabs :( [22:51:18] I think it only shows up when there are nearly that many results [22:51:43] I wish I could just tell it to always show all results [22:51:49] like it used to [22:54:39] phew... got 1000 back [22:54:40] thanks [22:55:03] that'll teach me to touch that option [22:55:32] never gonna touch it again... unless someone adds an option to show even more at a time :P [23:03:17] WikidataFacts: btw I was looking at the ui code you linked earlier... do you know whereabouts it decides whether to enable certain views? [23:05:33] (like you can only select the map if you have coordinates, and the charts if you have... numbers I assume) [23:19:23] nikki: I think the `visit` function in the result browsers (usually near the end) determines that, but I’m not sure how it works [23:19:52] ah