Are prohibited files from robots.txt included in the index?

D

D

Den Belzarayn2016-07-05 16:33:02

PHP

Den Belzarayn, 2016-07-05 16:33:02

Are prohibited files from robots.txt included in the index?

Such a situation in the robots prescribed the following lines to close the folder with redirects:
Disallow: /ro/
Disallow: /ro/*.php
Yandex processed correctly, but Google did not. What could be wrong? Can something add a mention of lines in robots?

Reply

Answer the question

Answer the question

In order to leave comments, you need to log in

1 answer(s)

L

LittleFatNinja, 2016-07-05
@LittleFatNinja

MB need to specify the user agent?

Similar questions

M

Marty McFly2018-11-12 19:00:41

Where is the best place to study web design from 0 in St. Petersburg? 6Reply

M

Mikhail Potapenkov2016-06-13 18:20:30

On initial startup, the OS issues a blue screen 0x0000007B. What does it mean? 4Reply

S

Sergey Buben2016-06-16 11:37:54

How to add a second price in VirtueMart in Joomla? 3Reply

P

Pavel2014-05-18 14:34:20

How to set up Chrome DevTools Autosave on windows? 4Reply

A

Artem Gvozdev2018-11-28 11:53:31

Using the weather API? 3Reply

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question