A
A
adam_carraway2018-08-07 22:03:07
robots.txt
adam_carraway, 2018-08-07 22:03:07

Is robot.txt correct?

Have I configured robot.txt correctly?
I have a product page that is available at site.ru/product/product_id (there are many). Will my products be displayed in the search engine when I add the site to webmaster.yandex.ru ?

User-agent: YandexBot,Googlebot
Crawl-delay: 1
Host: http://site.ru
Disallow: /ordergo
Disallow: /admin
Disallow: /login
Disallow: /register

Here is an example:
There is a page with a product, this page has the title "Computer SDL-500", and the description "buy Computer SDL-500" - (the page is available at site.ru/product/5).
Will the page be displayed in a search engine if I type for example: buy Computer SDL-500 inexpensively.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
B
bkosun, 2018-08-07
@bkosun

Use online services of search engines to check the correctness of robots.txt
https://webmaster.yandex.ru/tools/robotstxt/
https://www.google.com/webmasters/tools/robots-tes...

S
SagePtr, 2018-08-07
@SagePtr

At a minimum, it should be called robot s .txt, not robot.txt

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question