Answer the question
In order to leave comments, you need to log in
How to parse Google search results without blocking (PHP + cURL)?
I parse the output of Google (only the first page of the output), after about 30 requests, Google dumps the captcha. Is it possible to parse without blocking, without using a proxy? It is necessary that ~ 1500 requests be processed in no more than three hours.
I set pauses between requests, sent browser-like headers.
Answer the question
In order to leave comments, you need to log in
There is an XMLRiver
service
~ 1500 requests can be collected in 10 minutes.
No, and moreover, a proxy, and even more so a VPS / VDS, may already be present in the list so that the captcha appears on almost every second request.
На днях написали статью "How to check which URLs have been indexed by Google using Python"
Тут имеется в виду парсинг по списку URL, можно подшаманить и парсить по запросу.
ссылка
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question