Does the search robot see robot.txt?

C

Caspanch2016-02-06 15:39:21

Apache HTTP Server

Caspanch, 2016-02-06 15:39:21

At the root of the site is robot.txt, on request "site / robot.txt" of course throws a 404 error, the question of this nature is the search robot will see this file or not?

Reply

Answer the question

In order to leave comments, you need to log in

2 answer(s)

R

Rsa97, 2016-02-06
@Caspanch

Think for yourself how the robot will receive this file if the site does not give it back?

D

Dmitry, 2016-02-06
@imert

no.

Similar questions

N

ngapp2022-03-24 15:57:00

How to override one item per bottomNavView in Android? 2Reply

A

Andrnofollow2020-11-12 12:10:06

What do you think about buying links from irrelevant countries? 3Reply

A

Alexey Sklyarov2019-01-31 00:56:36

Why does a 403 error appear after re-deploying a project? 1Reply

V

Vladislav2014-08-18 11:34:55

"Eternal" sessions in PHP. Will the server survive? 3Reply

A

Andrey Plax2014-08-22 12:46:34

After transferring the site to apache php from windows to linux, ubuntu stopped sending post, where to dig? 0Reply

Didn't find what you were looking for?

Ask your question