V
V
vova12132018-03-12 18:05:22
Yandex
vova1213, 2018-03-12 18:05:22

How to limit site indexing by Yandex?

Hello. Yandex, like crazy, every day for a week completely indexes all the pages of my site. It's kind of good, but I have an API installed that works every time, and the number of requests for it is limited by the tariff. Before, everything was fine, I indexed a couple of hundred a day and that's it, but now it's 7,000.
How can I limit indexing? I read that you can disable indexing from the main bot, allowing only additional ones and vice versa, will this help?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Artem Spiridonov, 2018-03-12
@customtema

robots.txt

User-agent: Yandex
Crawl-delay: 2.0 # задает таймаут в 2 секунды

User-agent: *
Disallow: /search
Crawl-delay: 4.5 # задает таймаут в 4.5 секунды

https://yandex.ru/support/webmaster/controlling-ro...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question