[09:08:43] Is there any way to import mediawiki dump faster using importDump.php, [09:08:44] As it took around 3 hours to dump 8k records and entire week for april dump. [09:12:37] vivek, of English Wikipedia? [09:19:09] @gry yes English wikipedia [09:51:50] vivek: while it's not a bad choice, there is #wikimedia-tech that might be a bit more familiar with these dumps [09:57:45] The fastest method may be setting up external storage. If you could have sufficient memory on your machine to put the entire database on it, except text, mount the database in ramdisk. Set up 2 database instances, one in memory and another on disk. Point external storage on disk. That would be the fastest throughput you can achieve [09:59:08] If you don't have sufficient memory for this, set external storage on a different instance as well, and give that instance low memory, while the main instance should have the maximum memory as you can [17:57:19] Hi folks, I'm hitting up against the HTTP querystring length limit trying to use the query action on the Action API on a large number of titles. Is it possible to perform a query by POST request, so I can use multipart/form-data? [17:57:32] Yes [17:57:49] POST-ing the list of titles should work fine [17:58:32] Awesome, thanks Reedy :) Is that true of all the GET endpoints, or just query? I'm actually trying to make a PR to the MW client library I'm using to get multipart working with it, so it'd be useful to know if that can safely be included for everything if the parameters are chunky [18:03:26] Naypta: Define all GET endpoints? [18:03:52] Certainly everything via /w/api.php should work like that [18:04:15] I mean, the endpoints that are marked as being GET requests on mediawiki.org [18:04:18] Ah cool, so long as it all does [18:04:31] I'm only looking at the Action API bits and bobs [18:04:50] IIRC, that api basically mungles GET and POST parameters [18:04:54] I think this is documented somewhere [18:05:23] I thought I had read something of that ilk somewhere, but I looked and couldn't find anything obviously [18:05:57] My only advice, is if you're doing write actions, is be careful how you deal with the token [18:06:21] how'd you mean? [18:07:11] From memory... don't put your token at the end of the GET, put it at the end of the POST [18:07:24] That way, if your whole query doesn't go through, you don't get broken edits [18:07:29] A great example is editing [18:07:34] you're sending 100K characters [18:07:47] Ahh yeah to avoid cutoff [18:07:52] If your token is in the GET part, and your POST gets truncated, your edit may not be as you expect [18:08:04] If your token is at the end, and MW doesn't get it, it won't go through [18:08:13] Yeah that's already implemented thankfully :) (personally I use MD5 on all my edits to be sure anyway, but *shrugs*) [18:08:32] Sure, but it's extra safeguards [18:08:40] Oh yeah definitely [18:08:41] And some of us have been doing this since before they were in ;P [18:08:59] The less risk of [expletive-ing] things up the better as far as I'm concerned :D [18:09:03] Little things that are minimal (or no) extra effort, and can give some benefit [18:09:07] Yeah absolutely hehe [18:09:27] Thanks for the help as always, I really appreciate it :) Surprised you're not completely sick of me by now! [18:09:59] If people are polite and ask reasonable questions, and/or can help explain what you mean... Most people don't mind [18:10:09] It's if you're rude, or come in expecting people to do your "work" for you all the time [18:10:39] Plus, we all know a lot of the docs suck, so getting it from peoples knowledge is sometimes easier/better [18:12:13] I've seen worse docs in my time ;) [18:12:22] At least MW has some... [21:32:32] Naypta: Reedy: pretty sure token is the one field that must be passed in the POST body [21:32:44] but everything else can be in either GET/POST [21:35:07] survey says you're right, legoktm: https://en.wikipedia.org/w/api.php?action=edit&format=json&title=Wikipedia:Sandbox&appendtext=Hello&token=obviously%20not%20a%20real%20token [21:53:53] [19:08:41] And some of us have been doing this since before they were in ;P [21:54:14] legoktm: I think that was a change made due to the issue I detailed causing crappy edits :)