S
S
stichii2015-11-16 00:31:02
robots.txt
stichii, 2015-11-16 00:31:02

Why can't the robots.txt file be displayed on the WordPress site?

Good afternoon! Created a robots.txt file

User-agent: Yandex
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: */trackback
Disallow: */*/feed/*/
Disallow: */feed
Disallow: /*?*
Disallow: /tag
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /comments
Disallow: *?s=
Disallow:/comment-page-*
Disallow: /page/*
Disallow: *?all_comments

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/cache
Disallow: */trackback
Disallow: */*/feed/*/
Disallow: */feed
Disallow: /*?*
Disallow: /tag
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /comments
Disallow: *?s=
Disallow:/comment-page-*
Disallow: /page/*
Disallow: *?all_comments
Host: сайт.ру

When you click on the site.ru/robots.txt link, the above content is not displayed instead:
User-agent: *
Disallow: /wp-admin/

On one of the forums it was written that there might be a stub on WordPress.
In the WordPress settings, the checkbox "Ask search engines not to index the site" is not worth it.
Can anyone know where to look for the reason why the contents of the robots are not displayed?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
K
Ksenia Mikhailova, 2015-11-16
@arizona

Wordpress generates its own virtual robots.txt. You can turn it off with hooks: wordpress.stackexchange.com/questions/77969/how-to...
But I didn't use this filter, so I don't know how exactly it works.
Here is some more beautiful on wp-kama.ru

D
Dmitry, 2015-11-16
@seqular

Can redirect be done by means of a web server? Look in the logs, where is this file from?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question