A
A
Alexandra Nev2018-02-20 16:13:59
MODX
Alexandra Nev, 2018-02-20 16:13:59

What to write in the robots.txt file for Modx?

The question arose, what should the correct robots.txt file for modx look like? All resources say that these directories must be closed from indexing, that is, like this:
User-agent: *
Disallow: /manager/
Disallow: /assets/components/
Disallow: /core/
Disallow: /connectors/
Disallow: /index.php
Disallow: *?
But, if for security reasons the manager, core, connectors directories were renamed, what was the point of changing their names if these names were then written in robots.txt? What then to specify in robots.txt and how to close directories unnecessary for seo from indexing?

Answer the question

In order to leave comments, you need to log in

3 answer(s)
Y
Yana Mitrofanova, 2018-02-20
@alexanev6

In these lines:
No sense, because. if the permissions are set correctly, they will return 403.
Add:

Host: site.com
Sitemap: site.com/sitemap.xml

Z
zooks, 2018-02-20
@zooks

If renamed, they should not be specified in robots.txt.
It is better to delete this line too, because some of the styles can be read from the directory with the components.
Disallow: /assets/components/

A
Andrey Shved, 2022-01-27
@shvedan1

The file can be created through the generator capyba.ru/services/tools/robots/

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question