M
M
Matvey Istomin2020-06-15 20:15:38
In contact with
Matvey Istomin, 2020-06-15 20:15:38

How can I speed up parsing?

I collect information on all public groups on VKontakte (including media - audio, photos, videos, comments, notes, records ...)
It makes no sense to create a bunch of accounts, because VKontakte stops actions aimed at collecting information. Those. if you execute the photos.getAll method many times , then after some time VKontakte will give an error 29 instead of data.
Since I collect information by groups, I can parse the site without any API. It works, but if you run at least two threads, then everything breaks.
The question is, can this process be accelerated?
I know that you can take a VPS and do something already.
There is no point in using public proxies, they are banned. Can buy. What do you think?
Can I use Tor Proxy? Are they banned or not?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
Dmitry Bashinsky, 2020-06-16
@BashkaMen

unfortunately, either normal proxies are needed, or another way to make requests from different ip.
For example, if you had your own botnet (a bunch of machines executing commands), you could give the task of getting data from the page.

P
Puma Thailand, 2020-06-16
@opium

Proxies cost a penny

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question