Answer the question
In order to leave comments, you need to log in
Can a site be blocked "by hardware"?
I made a parser of a rather large site, after many requests (~ 10k) with excellent proxies (each request changes), I get a captcha site.
It is worth noting that the site is not parsed through the web version, but through requests (python)
Each request generates different headers
How do I I realized that the ban is for iron? The same parser worked fine on the server for about the same time (it did not issue a captcha), while I ran it on my PC and the result was zero.
Further, the site does not let you in without captcha at all, I remind you that the proxies are of really high quality and there are no suspicions about them, this is something with hardware, but in my understanding the site cannot ban like that since it does not have access to java, js, flash, webrtc
Answer the question
In order to leave comments, you need to log in
It can't, it's just data exchange.
What headings have been changed? Maybe not enough new ones are being generated or something.
Either the site does not like the load that appears -> captcha is enabled for everyone, or just a faster ban began to make bad user agents. Bad proxy / many requests from one agent = ban both
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question