Answer the question
In order to leave comments, you need to log in
Does Google public DNS have limits?
Good evening.
I need to parse information from about 3 million sites, literally go to the site and download several pages (index and a couple of internal ones).
So, since I want to do this multithreaded, my script for some reason (definitely not because of the code, because everything is 100% OK with the code) is very slow. I think it's because of Google Public DNS limits. What are the options for solving the parsing problem in this case? (A proxy is also an option, but I want to take a back seat to this option, as it is completely impractical).
Answer the question
In order to leave comments, you need to log in
Try to make a couple of DNS requests while the script is running.
Surely the problem is still in the code, only you do not know about it.
Use profiling.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question