Does the search robot see robot.txt?

C

Caspanch2016-02-06 15:39:21

Apache HTTP Server

Caspanch, 2016-02-06 15:39:21

At the root of the site is robot.txt, on request "site / robot.txt" of course throws a 404 error, the question of this nature is the search robot will see this file or not?

Reply

Answer the question

In order to leave comments, you need to log in

2 answer(s)

R

Rsa97, 2016-02-06
@Caspanch

Think for yourself how the robot will receive this file if the site does not give it back?

D

Dmitry, 2016-02-06
@imert

no.