Answer the question
In order to leave comments, you need to log in
Answer the question
In order to leave comments, you need to log in
robots.txt is a recommendation for the PS. They can even index disallow. Usually there close doubles and snot. Better to do, work for an hour. site:domain.ru you type in a jar and look at all the rubbish in the search results.
Of course. This file sets the indexing rules, but does not prohibit it by default in any way if it is missing.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question