[00:36:43] New patchset: Daniel Werner; "Move jQuery.ui.suggester into ValueView extension" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57448 [00:37:05] New patchset: Daniel Werner; "Moves jQuery.ui.suggester from Wikibase into ValueView extension" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/57449 [00:48:47] is there a property for official positions? for when a person has held the position [00:49:11] maybe [00:49:23] * sDrewth sharpens a stick [00:49:23] * legoktm checks what barack has [00:49:49] dog's breakfast -> https://www.wikidata.org/w/index.php?title=Special%3AAllPages&from=&to=&namespace=120 [00:50:07] https://www.wikidata.org/wiki/Property:P39 [00:50:12] office held (political) [00:50:35] k, thx [00:51:17] ive found the fastest way to find a property without trying to search through WD:P is go to the most popular instance of the item you're trying to add to, and chances are someone will have already added it :P [00:52:55] legoktm: usually (in wikis) many are right but fail on helpful; here you are helpful, but that is just so wrong [00:53:45] ahahaha [00:53:47] :) [00:54:07] someone/some group are going to need to lovingly tend WD:P if people are going to do it right [00:54:13] right [00:54:17] its near impossible to keep up [00:54:26] * legoktm writes a bot [00:54:59] legoktm: it almost needs to be expandable like CategoryTree [00:55:20] just click a twistie and drill your way down [00:55:36] hmm [00:55:40] thats a pretty good idea [00:56:00] you click the property, and it adds that and waits for the data bit [00:59:49] https://www.wikidata.org/wiki/Wikidata_talk:List_of_properties#Missing_Properties [00:59:51] * Hazard-SJ goes to do some coding now [01:00:49] :) [01:00:57] know JS Hazard-SJ? [01:01:04] heh [01:01:25] legoktm: Not really [01:01:30] :P [01:01:31] ok [01:12:16] legoktm: what do you think about letting the bot check logs up to one hour before run time? [01:12:35] for...? [01:12:55] expecting that whatever moves/deletions to be reverted would have been done by then, legoktm [01:13:50] ohoh [01:13:57] for the zomgadminbot? [01:14:34] hmmmmm [01:15:02] i suppose, but its not really necessary imo [01:15:07] Whatever you want to call it :P [01:15:09] just use the exclusions i said to :) [01:15:20] has anyone else commented since i did? [01:15:24] legoktm: OK [01:15:46] legoktm: I'm also inverting the order that my bot reads the logs in [01:15:50] ah [01:15:52] smart [01:16:17] legoktm: I got one move problem about it sometime ago, I assumed that was the cause :P [01:16:48] And my bot is going to have to run more frequently too :P [01:20:15] squatted. [01:21:39] New patchset: Jeroen De Dauw; "Minor tweak to entry point code" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57453 [01:26:11] legoktm: What do you think about adding checks to avoid deletion of non-orphaned pages? Support or oppose? [01:26:24] yes [01:26:50] my bot also checks for [01:26:58] instance of --> cultural something [01:27:06] i forget what its called [01:28:28] legoktm: You just used page.getReferences, right? [01:28:39] yeah with namespaces=[0] [01:34:20] trunk doesn't have that option AFAIK [01:35:51] well there's the problem ;) [01:35:58] {{oppose}} uses trunk ~~~~ [01:35:59] 10[1] 10https://www.wikidata.org/wiki/Template:oppose [01:37:56] :P [01:38:39] legoktm: The good thing is that your bot might still get work from mine ;) [01:39:36] legoktm: is there a botMayEdit function in rewrite? [01:39:46] course [01:40:29] I'm going to write a botMayDelete for my code :P [01:40:59] and if it returns false, it goes to RFD [01:41:27] separate user account right? :) [01:51:54] legoktm: I haven't created the account yet, but I have a name in mind if I need it [01:52:02] :D [01:55:25] legoktm: I plan to make a "mock deletion log" :D [02:11:09] legoktm: Does http://pastebin.com/D1zWds0c look okay to you? [02:12:05] whats the point of [02:12:06] if not self.canDelete: [02:12:06] return False [02:12:35] authors = list() <-- why not just make that a set from the beginning? [02:12:36] legoktm: I have a variable in __init__ called self.canDelete [02:12:44] ok [02:12:54] you can use len() over a set [02:13:04] no need to go from list --> set --> list [02:13:44] if u"wbsetsitelink-remove" in summary: <-- are you gonna check "and not user in admins"? [02:14:19] legoktm: Reload the page for the update [02:15:21] authors.add [02:15:25] set uses add not append [02:15:29] legoktm: it returns False for that by default in that case, so you're saying I should exempt admins from having it return dalse? [02:15:36] right [02:15:52] you dont delete things that have links removed cuz of vandalism [02:15:59] but admins dont vandalize :P [02:16:28] legoktm: So what if all the edits were made by different flagged bots? Coud I delete in that case? [02:16:41] i uh.... [02:17:01] Do you trust other people's code? ;) [02:17:33] legoktm: Are you another person? ;) [02:17:43] * Aren't [02:18:58] hmm? :P [02:19:01] I mean [02:19:05] I trust my own code [02:19:09] and probably a few others [02:19:25] but i wouldnt whitelist every bot... [02:19:54] * Hazard-SJ goes to exempt admins [02:22:33] New review: Jeroen De Dauw; "(1 comment)" [mediawiki/extensions/DataValues] (master) C: -1; - https://gerrit.wikimedia.org/r/57048 [02:25:17] New review: Jeroen De Dauw; "(4 comments)" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/57049 [02:25:36] * sDrewth slaps WD for its refusal to create a new item and saying that it exists [02:26:42] or maybe I am just being throttled [02:26:48] sDrewth: what item? [02:27:49] got it now, but twice it told me NO [02:27:58] >not gonna do it [02:28:30] oh wow [02:28:31] hmm, this refuses Knight of the Order of Australia [02:28:38] we've created 300k items today [02:28:41] it presumably matches the category [02:29:03] sDrewth: on enwiki thats a redirect [02:29:04] legoktm: that would be when your bot was throttling the lab server last night [02:29:06] pointing to https://www.wikidata.org/wiki/Q1141149 [02:29:19] New patchset: Jeroen De Dauw; "Remove clutter constructor comments" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/57464 [02:29:44] legoktm: sure, but while enWP may do it that way, it isn't always going to work in WD where it can need specifics [02:29:51] right [02:29:55] so um [02:29:57] especially when you get to infoboxes [02:30:06] I will just trick it [02:30:50] create it with a near name, and then rename when created [02:34:46] New patchset: Jeroen De Dauw; "Add exception message and use correct exception type" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/57465 [02:34:51] legoktm: ? [02:35:11] i dont think i pinged you... [02:35:20] you didnt :P [02:36:34] i need a special query or script :P or both :) as I found a problem :D [02:37:56] oh [02:37:57] hi [02:38:01] whats up? [02:38:31] durign moments of lag / laxlag exception it would seem my scripts has created duplicated in my db! I need to find / create a script / query that will remove duplicates where the collum 'lang' and 'article' are the same only leaving the most recent behind acording to collum 'checked which is a timestamp :D [02:39:44] how about... [02:39:57] i give you a query that gives you the duplicate rows [02:40:02] and you figure out which ones to trash? [02:40:07] i already have that :. [02:40:11] oh [02:40:34] * legoktm is feeling lazy though [02:41:08] ... [02:41:13] :D [02:41:22] * addpub always feels lazy [02:41:37] give it 24 hours and it might be done ;p [02:41:42] * legoktm puts addpub to bed [02:41:53] :P [02:42:30] after doing some more queries to my db I realise that the db contains many duplicates because of the alter to the db that I never managed to do because it was to big :P [02:43:05] so now I need a script to clean up after all of the error, which I am guessing will remove about 1.5million rows from the db :P [02:43:10] ish x] [02:44:32] im gonna go to sleep now [02:44:58] legoktm: care to look at http://pastebin.com/D1zWds0c first? :P [02:45:14] meh ima go sleep too and fix this tomorow :D [02:45:26] i just realized ive been up for 24hours [02:45:28] addsleep, addwake! [02:45:28] Hazard-SJ: sure [02:45:35] Thanks :D [02:45:37] Hazard-SJ: shhh [02:45:47] I got all of your pms and messages on talk [02:45:58] I still havent looked at a diff between the two lists you have [02:46:01] I saw :) [02:46:15] but I think there is still something missing :P [02:46:38] Hazard-SJ: is getVersionHistory a generator? [02:46:39] literally I have been flooded with crosslang talk messages :/ the last few weeks have been a right *&*** [02:47:15] whether or not it is, its probably just best if you iterate over it once [02:47:28] legoktm: It's in wikipedia.py [02:47:37] in Page [02:48:18] addsleep: And the code already takes long enough to run :S [02:48:41] hehe, I know, it does have to check a hell of a lot of stuff :P [02:48:51] do you have to code public somewhere? :) [02:49:32] Hazard-SJ: http://dpaste.de/LzsJT/raw/ [02:49:35] addsleep: I'll put it on GitHub soon [02:49:35] ALSO [02:49:37] 4 SPACES [02:49:42] now is property "subdivides into" generic or specifically part of location? [02:50:05] Hazard-SJ: looking at the size of the lists there must be some error somewhere in my opinion :P [02:50:09] sDrewth: how have other people used it? [02:50:36] legoktm: Notepad++ uses tab by default [02:50:40] it is inundated with location, as it sits there in the guidance [02:51:18] Hazard-SJ: sofixit :) [02:51:27] but not sure how else to subdivide a "termn" [02:51:28] sDrewth: No clue [02:51:35] New review: Jeroen De Dauw; "The general idea here is good though the code has some serious issues as well, including some incorr..." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/55515 [02:51:50] i wish I had more time :/ [02:52:02] gnite everyone [02:52:08] :/ [02:52:15] but its only 3:52 am :/ [02:52:30] legoktm: Why are the four spaces preferred, though? :) [02:52:32] * legoktm has been up for 25 hours now [02:52:46] New review: Jeroen De Dauw; "Also, this code is lacking tests... Not cool" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/55515 [02:52:51] legoktm: I dont remember when I woke up :P [02:52:54] rschen7754: is property "subdivides into" generic or specifically part of location? [02:52:57] Hazard-SJ: PEP8 standard. Also not all text editors handle tabs the same way, but they all handle spaces correctly [02:53:00] but it was some time ago :P [02:53:09] * Hazard-SJ tries to keep legoktm up for 26 hours [02:53:17] GNITE [02:53:33] gnite legoktm [02:53:41] * addsleep slaps Hazard-SJ [02:53:46] punch drunk! [02:53:48] * addsleep goes to compare two lists [02:53:56] * addsleep slaps sDrewth [02:54:18] shhh [02:55:44] * ihaveamac word [02:56:09] :/ [02:56:14] * addsleep slaps ihaveamac [02:56:20] * ihaveamac pokes addsleep [02:56:30] shhh [02:56:54] f-it, if there are subsidiary parts, and two way pairs, then it would be helpful to show the freaking pairs [03:02:08] New patchset: Jeroen De Dauw; "Remove StringFormatter" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57466 [03:03:07] :D [03:03:52] sDrewth: not sure, i haven't done a lot of wikidata in hte last few weeks [03:05:59] New patchset: Jeroen De Dauw; "Improve docs and removed incorrectly placed todo" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57467 [03:06:12] Change merged: Jeroen De Dauw; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57467 [03:10:44] New patchset: Jeroen De Dauw; "Remove StringFormatter" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57466 [03:11:40] New patchset: Jeroen De Dauw; "Remove StringFormatter" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57466 [03:13:19] rschen7754 :O [03:14:12] ? [03:17:02] not done a lot of wikidata in the last few weeks? :O [03:17:31] yup [03:22:15] I couldnt sleep [03:22:43] In other news, my category-item creation bot is currently running on every wiki except enwiki [03:23:10] sDrewth: And I stuck them into SGE so it shouldnt overload anything :) [03:28:43] I'm editing faster than 500epm [03:33:33] We should hit 10mil in the next 2 days if we can sustain this rate. [03:36:14] legoktm: coibot will catch up one day ... edit-backlog: 2633 files (1499827 lines) [05:05:56] Do we have something for "member of band" yet? Just "part of"? [05:44:58] superm401, maybe "member of"? [05:45:17] Hahc21, does that exist in general? [05:45:27] I thought there was only member of party/member of sports team. [05:45:32] ugh no [05:45:43] we'd need "member of band" too [05:46:03] Alright, I'll propose it. [05:46:18] I kind of get why they're separate, but I'm not sure it makes sense. [06:06:38] Hahc21, proposed at https://www.wikidata.org/wiki/Wikidata:Property_proposal/Person#member_of_musical_group [06:06:57] Great [06:06:59] :D [06:07:05] I shall support that [06:08:12] Okay, supported, and I consider we do need it [06:17:26] hi [06:18:09] helllo [06:52:54] New patchset: Jeroen De Dauw; "Cleanup of DataType related formatting code" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57466 [07:33:09] New patchset: Jeroen De Dauw; "Cleanup of DataType related formatting code" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57466 [07:35:10] !nyan [07:35:10] ~=[,,_,,]:3 [07:42:14] New patchset: Raimond Spekking; "Fix typo" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57476 [07:43:50] someone please poke me when hoo shows up [07:43:54] also: good morning [07:46:57] https://www.wikidata.org/w/index.php?oldid=19819425&rcid=19840769 Woooohooooooo [07:47:09] https://www.wikidata.org/wiki/Q10000000 [07:47:24] we´ve hit 10 mil now :P [07:47:32] congrats legoktm [07:47:37] Lydia_WMDE: ^^ [07:47:53] woah! [07:48:31] I was watching the livestream of the RC for a time [07:48:39] and saw we were close to it :p [07:48:45] hehe [08:34:12] o,o [08:34:36] well it seems to have stopped [10:40:37] Change merged: Aude; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57476 [10:47:59] legoktm: hey, you there? [11:20:46] did i do anything wrong? [11:35:40] Schisma: any reason you are asking? [11:38:03] Lydia_WMDE: I mean in my proposal [11:38:19] Schisma: sorry, which proposal? [11:39:53] ah [11:40:03] my irc client crashed it seems ^^ [11:40:30] http://www.wikidata.org/wiki/Wikidata:Property_proposal/Place#Openstreetmap_Relation_ID [11:40:38] this proposal [11:40:59] looking [11:41:39] Schisma: data type should probably be string [11:42:01] and you can add a link to openstreetmap to the source part [11:42:15] and add your name to proposed by :) [11:42:16] there is no such thing as an int? [11:42:30] not at the moment [11:42:39] but string is fine for what you want there i think [11:42:52] other than that it looks fine to me [11:48:06] would it really be an int? [11:48:21] i.e. is 0123 the same as 123? [11:48:26] if there are ints, it would be the same [11:48:31] if they are strings, it would not [12:25:20] Denny_WMDE1: http://www.wikidata.org/w/index.php?title=MediaWiki:Common.js&action=history [12:25:33] trying again, legoktm? [12:26:07] language selector thing comes from commons [12:27:35] thx, that is really helpful aude [12:27:37] http://www.wikidata.org/wiki/Wikidata:Administrators%27_noticeboard#Import_AnonymousI18N.js [12:28:01] Hey, and admins here? [12:28:07] any admins here? [12:28:47] I am thinking about blocking legobot as it seems to currently be the cause of a major lag [12:29:46] also it is currently creating a lot of items for categories that do not even have language links - I am not exactly sure what the advantage is of those [12:30:08] addshore: Moe_Epsilon ^^ [12:30:28] benestar|away_: ^^ [12:30:31] hmm [12:30:37] legoktm around? [12:30:44] yeah, i tried to address him [12:30:53] if he would throttle the bot a bit this could help [12:31:03] or at least let us know how long this is going to run probably [12:31:19] if it is done in a few hour, heck, ok. but if this goes on for a few days… :-/ [12:31:21] is it really causing lag? :/ [12:31:26] we are not sure [12:31:30] it is our best guess right now [12:31:38] we're currently at 5 hours lag or so [12:31:46] so it's pretty bad from 0 last night [12:31:53] he is currently creating items in an extremely high speed [12:32:04] about a million new items since yesterday [12:32:12] which is an increase of 10% [12:32:14] bad lego, he should know that XD [12:32:27] i am not blaming him, in no way [12:32:46] and we are not sure it is even the root of the fault [12:32:54] Id say wait a while and see if he turns up :) apparently he went to sleep at the same time as me so should pop in soonish [12:33:00] but i would like to switch it off, so we can see if this is the case [12:33:17] I am waiting for two hours [12:33:18] where's the problem then? [12:33:24] the lag has gone an hour worse since then [12:33:33] you can see it here [12:33:34] http://www.wikidata.org/wiki/Special:DispatchStats [12:33:34] mabdul: changes not going to the wikipedias [12:33:42] so pages not purging and so on after changes [12:33:55] eman blocking and unblocking a bot is exactly for invented for such cases: if bot op not available - blog it [12:33:57] and not showing up in the recentchanges and watchlists [12:34:25] *i mean [12:34:30] hmm Denny, what are the possible causes of increased dispatch lag? [12:34:51] im guessing new links being needed on the clients from wd? [12:35:08] addshore: any change on wikidata basically [12:35:15] that affects a wikipedia [12:35:32] we have a few other possible culprits in sight [12:35:39] one was the search index creation [12:35:40] 5h is very much actually... [12:35:44] but we stopped that last night [12:35:52] so it was not the case [12:35:57] just on wikidata, or would changes on the clients meaning they would need more links from WD have the same affect, or would that not change the lagtime? [12:36:05] another one was the fact that the dump is currently being created [12:36:11] but this should only be on client [12:36:37] changes on the clients would have (almost) no effect on the lag [12:36:48] it is something on wikidata [12:37:05] and the massively increased edit rate is our best guess right now [12:37:32] is there any harm in stopping it now and seeing if that helps? [12:37:39] addshore: would you confirm my public account ([[User:wikignome]])? [12:37:40] 10[2] 10https://www.wikidata.org/wiki/User:wikignome [12:38:15] there might be, there might not be :P [12:41:41] it seems, legobot is currently making more than a thousand edits per minute [12:42:03] this is a bit heavier than we expected, causing the lag [12:42:14] (in our current best guess, i have to add) [12:43:15] anyway the whole dispatch process can be sped up? :D [12:44:11] * addshore tries stopping his bot to see if off wd edits do make a difference [12:44:36] addshore: your edits are nowhere near as many as legoktm [12:44:44] it's over 1000/min [12:44:47] we are talking about more than a thousand edits per minute on a single wiki :-/ [12:44:51] http://www.wikidata.org/w/index.php?title=Special:RecentChanges&limit=1000&hidebots=0 [12:44:54] amazing [12:44:56] Lydia_WMDE: same for me, just crosswiki so noone notices, [12:45:05] addshore: no that is different [12:45:11] it hits different wikis [12:45:12] addshore: your changes have been running for weeks [12:45:26] they are OK, we catched up during the last week [12:45:37] started a run on de not too long ago [12:46:08] OK, so any admin who is willing to block legobot? otherwise i will do it for technical reasons in two minutes [12:46:28] Denny_WMDE1: link, I'll take care of it [12:47:12] [[user:legobot]] [12:47:13] 10[3] 10https://www.wikidata.org/wiki/user:legobot [12:47:24] Sannita: [12:47:25] http://www.wikidata.org/wiki/Special:Block/Legobot [12:47:35] I put a message on Legobot's talk page [12:47:37] Sannita: thank you [12:47:50] legoktm: sorry! but it's needed [12:48:04] blocked for 1 month, let me know if and when I can unblock it [12:48:28] hopefully he'll adjust it before that and then it can go again :) [12:49:03] Sannita: the community is free to decide to unblock him anytime [12:49:13] we need to check now if it cause the block [12:49:20] ok [12:50:07] need to go afk, see you later [12:50:30] the numbers are already improving [12:53:09] .. [12:56:46] http://www.wikidata.org/wiki/User_talk:Legoktm#I_requested_a_block_for_Legobot [12:58:00] ok, sorry for pressing for this. [12:58:10] i will go offline now. [12:59:53] edsu: you there? [13:00:05] the lag is stilll gowing [13:00:15] or do i read that special page wrong? [13:00:36] mabdul: do not trust the Lag time [13:00:43] look at the number of pending changes [13:00:49] also growing [13:01:06] ? the average seems going down? [13:01:11] mabdul: the average in the last row is going down [13:01:16] that's the important onew [13:01:32] the others are moving only discreetly [13:01:40] which might look like a growth [13:01:42] uhm yeah you're right [13:02:22] we still have more edits than the english wikipedia per minute :) but at least it is not 10 times as much ;) [13:02:59] I have nearly cleared the lowp alone with the help of addbot and legobot :) [13:03:14] cool [13:03:38] Hi everyone, a fast question: Is or will there be an easy method to list the pages of some Wiki[pedia] not linked to/from Wikidata? [13:03:56] jem-: yes [13:04:00] sorry we have to leave for a meeting [13:04:02] back in some [13:04:26] Ohh [13:04:49] Inopportune me [13:05:22] :) [13:06:08] Ok, Lydia_WMDE, I'll read later... [13:06:18] Denny_WMDE1: i'm here now, but i see you are not [13:06:35] Lydia_WMDE: i noticed legobot doing a high volume of edits this morning on my report [13:17:01] edsu: the bot is blocked now [13:18:54] mabdul: yes, i thought that might happen, just watching it [13:19:21] can you see this? http://www.stathat.com/stats/SZbf [13:20:56] or maybe this? http://www.stathat.com/sh/SioSBn4CxlSKZEo7 [13:23:03] looks like it peaked at 20 req/sec [13:29:45] wow [13:34:16] i was surprised the website was as responsive as it was when it was under that load [13:35:03] i was also surprised stathat didn't decide i was abusing them as well, since each bot hit on wikidata triggers a POST to stathat's api :) [13:40:03] * addshore likes the look of stathat [13:41:57] * addshore is going to love this! [13:52:24] Change merged: Henning Snater; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/57270 [14:06:30] Change merged: Aude; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/56271 [14:08:53] aude: Around= [14:08:59] hi hoo [14:09:14] hi :) [14:09:43] aude: I thought about the update repo on page move bug... have you made up anything for that yet? [14:09:51] nope [14:10:32] seems like a good task for a bot, although if we can find a way to do it better in the extension, then good [14:10:37] We can either do it in JS, which wouldn't catch API moves, mass moves, ... or we can build a soft dependency towards CentralAuth and use some server side magic [14:11:03] hmmmm [14:11:44] i'm not sure we always want the repo to be updated for page moves... [14:11:50] sure there are use cases for not doing it [14:12:16] Many? [14:12:22] so maybe have a checkbox that enables updating the repo [14:13:02] no idea how many [14:14:09] I really hate this bug... several users already complained about the repo and the Wikipedias getting inconsistent [14:14:33] * aude nods [14:14:54] if it can be a checkbox (on by default), but that can be checked off, that could work [14:15:14] * aude tries to think of other ideas [14:16:24]