[08:14:37] hi, i need to crawl the wikipedia, how fast in page/sec. can i do it? [08:14:52] 1 site/min. is not much for 30 M sites [17:23:18] ilux: o/ You will be heavily throttled if you exceed the volume of a regular browser, so I'd suggest to check https://dumps.wikimedia.org/ if you need the whole content. [18:05:12] elukey, thx, i know it the better way, i have to check it.. [18:29:53] ok i have to write something for that but it will be ok [18:30:17] the download server is very slow, can i help with a mirror? [19:08:51] we are running a small network