Answer the question
In order to leave comments, you need to log in
How to parse Yandex Blogs (blogs.yandex.ru) using a proxy?
Greetings!
I am writing a parser in PHP + cURL.
It is required every hour to parse the results of the issuance of blogs.yandex.ru for 950 requests.
To solve this problem, obviously, anti-captcha and a proxy are required.
If everything is clear with anti-captcha, then there are questions to the proxy:
1. How slow is the use of a proxy to get search results, given the required number of requests?
2. How many proxies and what delay is required for this task so that the proxies are not banned?
3. Which proxies are better to use: individual or general?
I must say right away that the Yandex.Blog API will not work due to the fact that there is a limit of 70 requests per hour.
Answer the question
In order to leave comments, you need to log in
Good afternoon.
1. Proxy proxies are different. Some are nimble and trouble-free, others work in the morning, not in the evening. Of course, everyone has their own response time. You can simply switch to another proxy when the timeout is above 5-7 seconds.
2. IMHO, only empirically it must be determined.
3. I'll tell you from my own experience - there are more troubles with publicly available ones. I bought proxy lists. But their purchase does not eliminate problems - the same delays, some may not work at all. In general, a proxy is not always a panacea.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question