R
R
Resize2018-04-20 15:29:52
Search Engine Optimization
Resize, 2018-04-20 15:29:52

Strange robots.txt, what to do with it?

Good day, forum users.
Today, on one site, I saw the following lines in robots.txt:
User-agent: *
Disallow: /
Allow: /$
Allow: /ua/$
Allow: /en/$

As far as I understand " User-agent: * | Disallow : / " prohibits the site from being indexed by both Google and Yandex, but the lines " Allow: /$ | Allow: /ua/$ | Allow: /en/$ " allow indexing the main page in 3 languages. How professional is this solution or is it more like nonsense?
It's just that such lines are incomprehensible to me, if someone can help and explain, I will be grateful.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Andrey Kurtuchkin, 2018-04-22
@Resize

If all useful pages are available to search engines, and all useless ones are not, then this is the correct robots.txt. You can check whether pages are available through the verification tools in Google and Yandex webmasters.
5adc683182686502438133.png5adc683c04a13718045492.png

P
Puma Thailand, 2018-04-20
@opium

Well, it really depends on what your task is.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question