A
A
Al fox2017-08-03 08:06:04
robots.txt
Al fox, 2017-08-03 08:06:04

How to set a move in .htcaccess in order to register host in robots.txt?

The site has many 3rd level domains, as well as many mirrors. Now all domains are moving, so you need to redirect all site1.ru, site2.ru, site3.ru to site4.ru, and 3rd level domains domen1.site1.ru, domen1.site2.ru, domen1.site3.ru to domen1.site4 .ru, domen2.site1.ru, domen2.site2.ru, domen2.site3.ru to domen2.site4.ru, etc.
In .htcaccess, the relocation is now configured like this:
RewriteCond %{HTTP_HOST} ^site2\.ru
RewriteRule ^(.*)$ site4.ru/$1 [R=301,L]
RewriteCond %{HTTP_HOST} ^([^.]+ \.)site2\.ru$ [NC]
RewriteRule ^ http://%1site4\.ru%{REQUEST_URI} [R=301,L,NE]

There is an entry to different robot.txt for different 3rd level domains
RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [L]

The essence of the problem: for Yandex, you need to specify the HOST directive in robots.txt. Now the entry redirects everything, including the robot and yandex does not read because of this robots. How to redirect everything except robots.txt which are essentially located in the /robots folder and the files themselves have the format domen1.robots.txt

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question