Answer the question
In order to leave comments, you need to log in
Is it possible to write a rule in robots.txt that prohibits indexing pages whose url contains a certain fragment of the link text?
Hey! Previously, the question was asked how to hide empty pages from indexing by Yandex. I decided to try to close empty pages through robots.txt. The URL of an empty page looks like this peschanoe-letom.ru/s-20-06-2015-po-28-06-2015 . That is, it is impossible to prohibit the indexing of the directory, since it is not in the line (for example, as recommended in various articles on the Internet - Disallow: /category/otzivi/ ). How to correctly write a rule to prohibit the indexing of pages, for example, with the content of a part of the text "-po-", (since it is present in all urls of empty pages), if at all possible? Tell me please.
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question