4
4
4upik2017-03-19 11:37:19
Search Engine Optimization
4upik, 2017-03-19 11:37:19

How to block from indexing URLs that end with numbers in robots.txt?

On the site, you can access products using different links, so you need to prohibit duplicates from indexing.
How to block URLs that end with numbers from indexing? site.ru/item/1081261/ with and without slashes (can be either 5 or 6 or 7 digits) and leave site.ru/item/1081261/dnevnik-sportivnoy-devochki-v-korobke-na-magnitah open for indexing -s-lentami

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
Dimonchik, 2017-03-19
@dimonchik2013

no way, robots is very primitive and can only nest directories
i.e. site.ru/item/* - please
put this from unwanted canonical pages to the correct ones

X
xmoonlight, 2017-03-19
@xmoonlight

301 redirect by User-Agent and that's it.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question