Answer the question
In order to leave comments, you need to log in
How to correctly specify host and sitemap in robots.txt?
Hello, on different large sites I observe different options for placing the hosts and sitemap directives.
Tell me, having settings for different robots in the form:
User-agent: Yandex
User-agent: Googlebot
User-agent: *
Where to insert Sitemap and Hosts - into each, or only one or through the line?
Answer the question
In order to leave comments, you need to log in
It doesn't matter at all. In the webmaster panels, specify which version of the site is considered the main version and where the sitemap is located.
I will reveal a couple of secrets:
1. host has been deprecated a long time ago
2. If the site is well made, then the site map is not really needed
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question