Answer the question
In order to leave comments, you need to log in
How to download all domains in some zone, for example, .ae?
You can sort through the dictionary or brute force, check by who is, or just connect to port 80, but this is long and something tells me that there is a faster and better way. There are databases on the Internet, but they are paid, maybe I'm looking badly and can I just see the full list somewhere easily and simply?
Answer the question
In order to leave comments, you need to log in
I once faced a similar problem.
I had to identify all 6 character domains in the .com zone . I
acted according to the following plan:
1) I threw a program on go that collected all possible combinations for me, it turned out ~ 2.4 billion domains
2) I wrote a multi-threaded program in go that interrogated them. First of all, of course, by DNS, if there is an answer by DNS, then there is no point in raping whois. But with DNS, too, not everything is so simple, after a certain number of requests, DNS starts to think that it is being DDoSed and can start to blunt. So I collected a list of all possible variations of public dns servers + what the provider gave and polled them randomly.
3) If there was no answer by dns, then I turned to whois, through a bunch of different proxies and sockets, because they ban very quickly. Because whois is even more capricious than DNS. Moreover, whois can also deceive, give out that the domain is free, but in reality it is busy. And I had to use the API of some registrars to check.
If my memory serves me right, a database was assembled in about a week.
It was about 3 years ago, I don't know if anything has changed since then.
It is possible to download the zone file, but only in the unlikely event that the zone owner has not covered AXFR/IXFR with a list of addresses from which this can be done.
No other way.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question