Answer the question
In order to leave comments, you need to log in
Is robot.txt correct?
Have I configured robot.txt correctly?
I have a product page that is available at site.ru/product/product_id (there are many). Will my products be displayed in the search engine when I add the site to webmaster.yandex.ru ?
User-agent: YandexBot,Googlebot
Crawl-delay: 1
Host: http://site.ru
Disallow: /ordergo
Disallow: /admin
Disallow: /login
Disallow: /register
Answer the question
In order to leave comments, you need to log in
Use online services of search engines to check the correctness of robots.txt
https://webmaster.yandex.ru/tools/robotstxt/
https://www.google.com/webmasters/tools/robots-tes...
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question