Answer the question
In order to leave comments, you need to log in
How to ignore all domains except those specified in robots.txt?
In order to avoid getting into the index of developer forks of the project, as well as all the different domain variations like www. prefixes, you need to set a rule in robots.txt to ignore all domains except the specified one. How to do it?
And also how to make ignoring some directories also on this domain?
Answer the question
In order to leave comments, you need to log in
If you mean under
all-different domain variations- subdomains. Then you need to write in the root of each subdomain in robots.txt:
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question