A
A
Azami2019-01-07 17:51:18
Search Engine Optimization
Azami, 2019-01-07 17:51:18

How to correctly specify host and sitemap in robots.txt?

Hello, on different large sites I observe different options for placing the hosts and sitemap directives.
Tell me, having settings for different robots in the form:
User-agent: Yandex
User-agent: Googlebot
User-agent: *
Where to insert Sitemap and Hosts - into each, or only one or through the line?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
L
La1serg, 2019-01-07
@La1serg

It doesn't matter at all. In the webmaster panels, specify which version of the site is considered the main version and where the sitemap is located.
I will reveal a couple of secrets:
1. host has been deprecated a long time ago
2. If the site is well made, then the site map is not really needed

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question