Answer the question
In order to leave comments, you need to log in
How to write a dynamic address in robots.txt?
How to write a dynamic address in robots.txt
. for example, I have such an address: /news/234234
where is the /234234/
dynamically changing part and there is an address for printing this same page. /news/234234/print
I do not want search engines to index duplicates, that is, to prohibit indexing /news/234234/print
and I want other pages to be indexed.
Now robots.txt
it looks something like this:
User-agent: *
Disallow: /news/*/print
Allow: /
Host: site.com
Sitemap: http://site.com.ru/sitemap.xml
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question