Is it possible to give different feeds for different robots in robots.txt?

V

vox_termin2017-04-13 13:12:25

robots.txt

vox_termin, 2017-04-13 13:12:25

Good afternoon!
I'm interested in the question, is it possible to give one feed to a Yandex bot, and give the other feed to a Google or Rambler bot?

Reply

Answer the question

In order to leave comments, you need to log in

2 answer(s)

A

Alexander Pushkarev, 2017-04-13
@AXP-dev

All in one robots.txt

User-agent: Yandex
Disallow: /

User-agent: Mail.Ru
Disallow: /

User-agent: StackRambler
Disallow: /

User-agent: Googlebot
Disallow: /

User-agent: googlebot-image
Disallow: /

User-agent: googlebot-mobile
Disallow: /

S

Site Developer, 2017-04-13
@secsite

give away

Robots does not serve for permissions/prohibitions to "give away", but for asking (!) to exclude (!!) from the search results (not to be confused with indexing - they index everything anyway :) ). But whether the bots obey or not is their business.
www.robotstxt.org/robotstxt.html