A
A
airbor2020-08-17 11:35:48
Search Engine Optimization
airbor, 2020-08-17 11:35:48

Why does prohibiting # cause indexing of the entire site to be banned?

If you specify a ban on all pages with the # symbol in robots.txt, then this leads to a ban on indexing the entire site, all pages that do not even contain this symbol. Why?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Anton Velichko, 2020-08-17
@airbor

The # symbol in robots.txt is used only for comments that are ignored by search engine robots. The # - symbol is not used in robots.txt directives.
There is no need to close pages containing the # symbol at the end of the URL, because they do not get into the index of search engines, due to the fact that search engines regard two URLs as one:
site.ru/url
site.ru/url#abc

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question