H
H
Happymalina2016-06-19 12:59:42
robots.txt
Happymalina, 2016-06-19 12:59:42

Is it possible to write a rule in robots.txt that prohibits indexing pages whose url contains a certain fragment of the link text?

Hey! Previously, the question was asked how to hide empty pages from indexing by Yandex. I decided to try to close empty pages through robots.txt. The URL of an empty page looks like this peschanoe-letom.ru/s-20-06-2015-po-28-06-2015 . That is, it is impossible to prohibit the indexing of the directory, since it is not in the line (for example, as recommended in various articles on the Internet - Disallow: /category/otzivi/ ). How to correctly write a rule to prohibit the indexing of pages, for example, with the content of a part of the text "-po-", (since it is present in all urls of empty pages), if at all possible? Tell me please.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dimonchik, 2016-06-19
@dimonchik2013

study

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question