PinkAmpersand: i think only the {{{result}}} param needs to be collapsed [05:20:25] its been such a long time when i wrote that monster :( [05:22:37] actually no, include the {{{POST}}} too [05:22:42] that thing is a weird table [05:27:13] yurik: ok, think i got it. want me to save it? [05:27:26] yes please :) [05:27:33] thanks!!! [05:27:56] oh, want me to include a way that people can make it so it /doesn't/ collapse? [05:28:05] how? [05:28:08] did you save it? [05:28:24] no, not yet. and i mean like a "collapse=no" parameter one could add [05:28:44] it should be collapsed by default [05:28:49] oh [05:28:57] sure [05:29:17] is the POST inside or outside? [05:32:48] inside [05:33:13] if you want, I can also add a "collapse=result_only" [05:35:24] yurik: ^ [05:36:16] PinkAmpersand: ideally i would like the collapsed portion to be ONLY the results, but always [05:36:33] the header (the query as sent to the API) should always be visible [05:36:38] oh. i thought yo usaid you wanted {{{Post}}} too [05:36:51] right right, post should also go inside [05:36:54] sorry [05:37:11] #mediawiki-template-support ;) [05:37:29] duh: didn't know there is a channel for that :) [05:37:30] ok. but i meant "result_only" would make it so that the post /doesn't/ collapse, but the result still does [05:37:35] there isnt :P [05:37:38] its now this channel [05:38:03] PinkAmpersand: btw, it seems that some users of that template don't include the results at all [05:38:12] not sure how it should handle them [05:38:16] very few [05:38:33] i am not sure its needed really, up to you [05:39:15] ok. and i'll throw in a parser so it's not collapsing empty space if people leave the {{{result}}} blank [05:44:06] PinkAmpersand: thanks! i can't wait for the LUA to make the whole thing readable :) [05:44:20] LUA scares me :( [05:44:37] PinkAmpersand: trust me, nothing can be scarier than template magic [05:44:48] well, maybe except dealing with the parser code [05:45:05] i like my {{#if:{{{foo|}}}||{{{#ifeq:{{{bar|}}}|{{{baz|}}}||quux}}}} [05:45:15] exactly my point! [05:45:28] and i thought perl was bad [05:45:52] can we all just switch to c# or python please? [05:46:18] i will even tolerate java, even though by now it's so far behind, its not even funny [05:47:02] but all that stuff is so... weird. idk. template markup is the only programming language I can actually consistently understand [05:47:20] its not a programming language [05:47:24] its just a markup language [05:47:36] well i suppose technically its programming [05:47:44] but horribly convoluted programming [05:49:29] idk, i find javascript way more complicated [05:50:56] I don't think it's about it being complicated, PinkAmpersand [05:51:19] Java is arguably more complicated. But it is much more orderly in its rules and syntax. [05:51:24] (for example) [05:56:35] PinkAmpersand: there are some weird artifacts -https://www.mediawiki.org/wiki/API:Query [05:57:05] how so? [05:57:07] the first box - extended content should be "API Result" or something similar [05:57:14] and the The following content has been placed in a collapsed box for improved usability. [05:57:16] is not needed [05:57:53] is it possible to remove it? [05:58:19] yurik: you could edit {{Collapse top}} or create an alternate version, but otherwise, no [05:58:32] (but ok, I'll add a heading to the boxes) [05:58:37] ok, its fine than [05:58:43] PinkAmpersand: thank you!!! [05:59:35] PinkAmpersand: just out of curiosity, do you think it is possible to make the actual api?... call as the header? [05:59:54] yes, it would be [05:59:58] shall i? [06:00:07] that's what i tried to do myself [06:00:16] but it broke with the divs :( [06:00:21] lemme see what i can do, then [06:00:22] if you can [06:00:46] but if you are busy doing something else, i really don't want to take any more of your time :) [06:02:15] nah it's fine [06:10:45] btw Yurik I'm probably going to put your deletion request for P31 on hold [06:10:55] need consensus to develop [06:10:58] Jasper_Deng: i think his request isnt a CSD [06:11:02] its more of a XfD [06:11:10] since its not on WD:RfD, I don't see any rush. [06:11:12] yeah, exactly [06:11:38] Jasper_Deng: i didn't see any {{proposeToDelete}} - that's what my intention was [06:11:57] * duh comissions PinkAmpersand for a new template. [06:12:03] lol [06:12:19] http://www.mediawiki.org/wiki/Requests_for_comment/Wikidata_API#action.3Dwbgetentities [06:12:32] PinkAmpersand would make a fine admin [06:12:40] my initial thoughts about changing the wikidata API [06:12:48] hm [06:12:56] yurik: should that happen on mw or wikidata? [06:13:14] duh: it will be a change to the extension [06:13:24] true [06:13:31] although i suspect i will have to make a few modifications to the ApiPageSet class [06:13:44] to allow for the extension to specify which pages it wants to work with [06:14:05] because the titles & sites arguments specify a wkidata specific search [06:14:46] btw did you see that bug in the module manager thing today? [06:14:55] with POSTing not being enforced? [06:15:02] no, where?!? [06:15:17] lemme find it [06:15:21] krenair already fixed it [06:15:22] not good, me like no bugs [06:15:52] i'll pull and see the latest [06:15:56] https://gerrit.wikimedia.org/r/#/c/49344/ [06:16:03] https://bugzilla.wikimedia.org/show_bug.cgi?id=45017 [06:17:38] good thing it got caught [06:17:53] need to see how it got removed [06:17:55] bleh :( [06:18:07] don't like bugs, especially when they relate to security [06:18:34] yurik: yeah i can't figure out how to make that thingy the header. i'll try to do it some other time, maybe [06:18:54] PinkAmpersand: i think its because of the class in div [06:19:08] that's what foiled my own attempt last time [06:19:44] PinkAmpersand: thanks for the change! [06:19:55] doc is soo much more readable now [06:20:33] duh: what do you think about the proposal? [06:20:41] I have to read it a bit more [06:20:50] just look at the results [06:20:53] the diff [06:21:27] I just finished a script that parses a dump, checks if the content is up to date, and fetches from the API if necessary, and yields all of it together in a generator. [06:21:54] sounds ... complicated :) [06:22:07] why are you playing with the dump? [06:22:30] I need to fetch the page content of ~2.5 million pages. [06:22:41] ...=gigabytes.... [06:22:54] why content though? [06:23:00] The bot is editing pages [06:23:04] Let me get the link [06:23:22] is that the bot that deletes langlinks? [06:23:25] https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/Legobot_28 [06:23:25] yes [06:23:38] the rest of the script is super easy [06:23:46] fetching the page content was hard. [06:25:00] duh: is that a modification of the interwiki bot? [06:25:05] no [06:25:07] my own script [06:25:31] because that's where it handles all the possible cases with one->many [06:25:41] i mean - many pages in one to one in another [06:25:49] and all sorts of other weird cases [06:26:42] Eh [06:26:58] But around half the links already exist [06:27:11] do you do deletion in all sites at the same time? [06:27:28] lol PinkAmpersand. [06:27:32] yurik: no, it only runs on enwiki [06:27:37] enwiki + wikidata [06:27:49] bleh... might run into major problems [06:27:53] how so? [06:28:00] it checks that wikidata == local before removing [06:28:00] because of interwiki conflicts of many->one [06:28:08] duh: re RFP? [06:28:14] PinkAmpersand: yeah :P [06:28:19] :D [06:28:22] yurik: but that gets handled on wikidata [06:28:27] there are already enough of those problems [06:28:55] that's true, but IMO the bot should be an extension of interwiki bot, and delete only when it detects that all sites have identical lists, and delete them everywhere at once [06:29:07] i don't think we should rush with the deletion until everything switches [06:29:38] in the mean time i will see if i can get the langlinks to return if the data is coming from local or the repo [06:29:57] no [06:30:00] this way we can quickly narrow the troubled ones [06:30:02] the problem is [06:30:06] humans are removing at their own pace [06:30:13] but not all humans verify its correct [06:30:21] they just assume "oh wikidata, i can delete these" [06:30:30] humans should not remove at all [06:30:31] which is why this is important to do it right [06:30:34] but they are. [06:30:35] why are they removing ?? [06:30:41] that we should stop [06:30:42] to make wikidata show up [06:30:45] no [06:30:47] thats a the problem [06:30:51] since local suppresses wikidata [06:30:58] and interwiki bots are no longer running [06:31:03] our langlinks are becoming out of date [06:31:13] and have to be replace with wikidata, via removal [06:31:14] does wikidata append to the local? [06:31:23] they do merge [06:31:27] it's not one or the other [06:31:34] in that case there is no problem [06:31:38] merge, with local overwriting wikidata [06:31:41] there is [06:31:42] exactly [06:31:44] say on local [06:31:50] you have [[de:Page1]] [06:31:58] on wikidata that gets updated to [[de:Page2]] [06:32:10] the link on enwiki will still show [[de:Page1]] [06:32:16] even though its been fixed on wikidata [06:32:22] and since no bots are running, it wont get fixed [06:33:13] ok, so what you are saying is that we need to get the langlinks info ASAP [06:33:23] yeah, as i edit road articles i'm removing the interwikis, since i know over 90% of US road articles have items [06:33:37] yurik: yes. [06:33:41] including all the ones in california [06:33:49] so my bot will do both [06:33:57] just make sure you monitor all those articles [06:33:57] it will create the wikidata item if needed [06:34:07] because if there is a rogue bot readding them, it has to be stopped [06:34:08] and then remove langlinks when confirmed [06:34:17] yurik: all interwiki bots on en have been disabled [06:34:34] i'm worried about international ones :) [06:34:44] with a global bot flag [06:34:45] wait [06:34:47] rschen7754: wtf [06:34:48] its editing [06:34:56] 06:19, 16 February 2013 Rschen7754 (talk | contribs) unblocked DarafshBot (talk | contribs) (updated) [06:35:00] 06:23, 16 February 2013 (diff | hist) . . (+33)‎ . . m Maritime republics ‎ (r2.7.1) (Robot: Adding fa:جمهوری دریایی) (top) [rollback: 1 edit] [rollback] [vandalism] [06:35:07] :( [06:35:12] just block it [06:35:17] yeah [06:35:33] page slow loading [06:35:41] yurik: because of pywikibot update, they wont check against en anymore [06:35:57] true, but they might not have updated yet :) [06:36:12] not much we can do [06:36:15] call me paranoid [06:36:20] we can monitor that [06:36:31] is there a list of known IW bots? [06:36:36] we set up https://en.wikipedia.org/w/index.php?title=Special:AbuseLog&wpSearchFilter=524 [06:36:38] yurik: no. [06:36:48] we created the list via database query and manual list [06:37:11] and then i (with help from rschen7754) went around the wikiverse and notified all bot owners on their home wiki about the change [06:38:01] https://en.wikipedia.org/w/index.php?title=Paris_Fashion_Week&diff=538467417&oldid=526339980 [06:38:04] bugaga [06:38:17] yurik: that bot has issues [06:38:30] 06:35, 16 February 2013 Rschen7754 (talk | contribs) blocked DarafshBot (talk | contribs) (account creation blocked) with an expiry time of indefinite (nope, still broken) [06:38:30] 06:19, 16 February 2013 Rschen7754 (talk | contribs) unblocked DarafshBot (talk | contribs) (updated) [06:38:30] 00:14, 16 February 2013 Rschen7754 (talk | contribs) blocked DarafshBot (talk | contribs) (account creation blocked, autoblock disabled) with an expiry time of indefinite (interwiki bot that was not updated) [06:38:42] please push the button [06:39:21] it took that op 3 tries to get through BRFA [06:39:40] technically it's not re-adding links [06:39:43] but still [06:39:45] doesnt matter [06:39:57] it had in the past [06:40:02] meaning its a liability [06:40:20] i really only trust Merlissimo's bot staying unblocked on enwiki [06:40:34] ok, is it dead? [06:40:38] blocked? [06:40:45] yeah, i'm an enwiki admin [06:41:04] oh, good [06:41:09] super! [06:41:54] ok [06:41:57] time to finish this bot [06:42:02] the hard part is done :P [06:42:32] good luck :) [06:42:43] just make sure it deletes only when the wikidata has a 100 match :) [06:42:49] well [06:42:53] it also will attempt merges [06:43:00] how so? [06:43:16] if en:page1 & page2 --> he:pageA [06:43:24] which one makes it to wikidata? [06:43:37] it checks for a 75% match [06:43:45] well at least right now it will [06:43:48] it will be suc a mess :((( [06:43:52] i probably will need to change it [06:44:13] ugh i got that exception error again [06:46:15] Riley: have some free time? [06:46:31] duh: Yes [06:46:37] wanna help generate more stuff for https://www.wikidata.org/wiki/User_talk:Legobot/properties.js ? [06:46:51] https://en.wikipedia.org/wiki/Category:Cities_in_the_United_States_by_state is probably a good starting place [06:47:06] duh: some of the road ones are probably easy to do [06:47:10] at least in the US [06:47:10] rschen7754: add them! [06:47:22] duh: that would require work to figure out the cats :P [06:48:52] get started! [06:48:56] there are so many categories to exploit [06:50:29] yeah, i still have to get phase 2 set up for USRD [06:50:44] trying to finish up phase 1 too :/ [06:50:54] my bot will finish it up :) [06:52:02] i need a good way to resume from a dump if it dies... [06:52:42] well, unfortunately it can't add commons files with variables [06:53:11] is it a predictable patern? [06:53:14] pattern* [06:56:47] duh: typically [06:56:57] https://www.wikidata.org/wiki/Q19183 [06:57:06] there's a few other fields that i still need to add [06:58:18] hm [06:58:22] ill look at it another time [06:58:27] it looks possible [06:58:33] but probably easier with a standalone script [06:58:39] rather than modifying my current one [06:59:10] my next step will support parsing templates [07:03:33] do i need my own repo to use wikidata client? [07:04:56] no [07:05:02] you can technically use the wikidata repo [07:06:09] good. but it seems like i do have to use a ton of extensions just to enable this baby [07:06:39] yeah [07:30:50] bleh, got a weird error after installing the client [07:31:13] http://pastebin.com/uaTczcdD [07:34:08] duh: does your bot support multiple statements for a property? [07:34:18] yes [07:34:27] ok [07:34:57] you just have to add it as multiple templates [07:35:01] do i need to setup remote db access? [07:35:12] i copied all the settings from http://www.mediawiki.org/wiki/Extension:Wikibase_Client [07:35:14] yurik: why not use the wikidata-vagrant? [07:35:19] i use it [07:35:28] ohh, wikidata-vagrant [07:35:29] sorry [07:35:37] i use the regular mvw one [07:35:43] link? [07:36:02] there are multiple ones [07:36:18] silkemeyer or abta [07:37:01] silkemeyer is the one i used [07:37:06] i just never finished setting it up [07:37:10] aude told me to use that one though [07:37:15] thx [07:37:43] bleh, but i already got the regular one setup :( [07:38:05] and it can't use multiple ones from the same repo [07:42:17] get another computer :P [07:43:10] https://www.wikidata.org/wiki/User_talk:Legobot/properties.js [07:43:13] so much potential [07:43:23] just need people to help gather categories. [07:46:01] duh: any ideas -- http://pastebin.com/EbjtpYJf [07:46:11] it complains on LoadBalancer->getConnection(-1, Array, 'wikidatawiki') [07:46:22] #2 /srv/mediawiki/includes/dao/DBAccessBase.php(61): LoadBalancer->getConnection(-1, Array, 'wikidatawiki') [07:46:24] yeah [07:46:33] are your foreign dbs set up properly> [07:46:42] or however that works [07:47:37] well, i followed the instructions on the ext page (link above) [07:47:46] and just copied all the settings from there [07:48:48] hm [07:48:54] what if [07:48:56] you remove [07:49:00] $wgWBSettings['repoDatabase'] = 'wikidatawiki'; [07:49:00] $wgWBSettings['changesDatabase'] = 'wikidatawiki'; [07:49:12] and include $wgWBSettings['repoUrl'] = 'http://wikidata-test-repo.wikimedia.de'; [07:49:18] duh: is your bot recursive? [07:49:50] rschen7754: no, however if theres a good reason, i can enable a recursion option [07:49:57] :/ [07:50:03] recursion is just a bad idea in general [07:50:10] for enwiki categories [07:50:13] i mean will you do subcats [07:50:22] do you want me to? [07:50:33] duh: http://pastebin.com/JJa9E7xw [07:50:34] how deep? [07:50:37] this is what i use [07:50:38] 1-2 [07:50:44] i think a few states may use it [07:50:47] yurik: get rid of [07:50:47] $wgWBSettings['repoDatabase'] = 'wikidatawiki'; [07:50:47] $wgWBSettings['changesDatabase'] = 'wikidatawiki'; [07:51:22] rschen7754: ok and do you want only articles at level 2, or articles at levels 0, 1, 2? [07:51:28] 0,1,2 [07:51:34] ok good, thats easy :) [07:51:47] just add |recursion=2 [07:51:53] ok [07:51:55] but it will only work for your name [07:51:59] as requestor [07:52:00] ooo [07:52:04] thx [07:52:15] ill whitelist people if they need it [07:52:20] i dont trust most people with recursion [07:52:36] had to re-run the interlang script again, but worked, thanks! [07:53:06] :D [07:54:32] duh: spoke too soon - the site works, but it doesn't show any interwiki links [07:54:37] hm [07:54:45] are the page names the same? [07:54:47] i tried creating a page that i know exists in wikidata.en [07:54:51] Childebert I [07:54:56] (was a random page) [07:55:02] oh right [07:55:05] null edit the page [07:55:06] it shows "add links" [07:55:13] its like a 5 minute delay caching or something [07:55:20] i created it from scratch [07:55:29] try null edit again? [07:55:30] delay on the test machine?? [07:55:36] yeah caching or something [07:55:37] idk [07:55:47] it was on the village pump [07:55:50] it just showed "add links" [07:55:57] hm [07:55:58] not sure then [07:56:06] clicking it shows "unexpected error occured" [07:58:20] no clue... [07:59:31] * Moe_Epsilon waves [08:01:06] duh: oooh... bleeeeh.it uses origin header (CORS) for some reason [08:01:14] * duh waves to Moe_Epsilon [08:04:23] when i click "add links", it issues an origin=localhost api call to the test wikidata [08:05:20] ah [08:35:15] duh: also for the statements does your bot create pages? [08:35:32] if you set |create=blah [08:35:37] ok [08:35:46] (where blah is anything) [08:42:06] duh: ok finally added one [08:43:38] :D [08:43:54] i think there's a few hundred [08:44:02] yay! [08:44:19] a few hundred more property sets, or few hundred in that one? [08:44:29] what i think we'll do is once it gets approved [08:44:38] if you have a request, you stick it on the talk [08:44:40] there's a few hundred in there [08:44:52] then a sysop who is not the requestor, will copy it over after checking it [08:44:58] ok [08:44:59] yeah i looked at the category [08:45:18] it looks good to me [08:45:22] cool [08:45:59] i would have added the two-digit ones but there's too many ignores that would be needed [08:46:22] how so? [08:46:36] too many oddballs in the category [08:46:52] oh hmmmm [08:46:56] some of these pages are redirects [08:47:15] thats a quick fix [08:49:34] done [08:50:27] thx [08:50:47] though those probably go to other things in the category [08:52:28] yeah [08:57:23] rschen7754: https://www.wikidata.org/w/index.php?diff=6674588&oldid=6673766&rcid=6687157 :DDD [08:57:38] wow [08:57:52] you'll need a bot just to generate the list :P [08:58:36] i used a script to generate it [08:58:39] now im checking it [09:00:23] ahh stupid florida [09:00:33] those are all subcat by county [09:18:33] phew, checked them all [09:30:33]