Answer the question
In order to leave comments, you need to log in
Why is parsing so slow?
Scab now site 2 million pages. On average, 10 pages are parsed per minute.
I use multithreading and proxy. Multithreading type several identical scripts with different page ranges. At what, that in one stream, that in several the speed is the same.
I use paid proxies.
The channel is not crowded, as you can watch full hd videos in parallel. When viewing, the speed neither increases nor decreases.
Enough RAM, processor too.
I'm using phpQuery, but I don't think it's the problem, since the processor is not clogged.
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question