Answer the question
In order to leave comments, you need to log in
Why does prohibiting # cause indexing of the entire site to be banned?
If you specify a ban on all pages with the # symbol in robots.txt, then this leads to a ban on indexing the entire site, all pages that do not even contain this symbol. Why?
Answer the question
In order to leave comments, you need to log in
The # symbol in robots.txt is used only for comments that are ignored by search engine robots. The # - symbol is not used in robots.txt directives.
There is no need to close pages containing the # symbol at the end of the URL, because they do not get into the index of search engines, due to the fact that search engines regard two URLs as one:
site.ru/url
site.ru/url#abc
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question