Robots.txt for Wordpress 4.7.5?

Question

Search Engine Optimization

Good afternoon.
Can anyone share the wordpress 4.7.5 robots.txt file?
I would like to know what to cost to close from spiders.
Thank you!

Answer 1

An option that I use on all my projects. It is better to close from the index using meta robots nofollow, noindex.
User-Agent: *
Disallow: /wp-
Disallow: /?
Host: https://site.ru
Sitemap: https://site.ru/sitemap.xml

Answer 2

I would like to know what to cost to close from spiders.

Only what you yourself consider necessary when analyzing the logs of YOUR site.
For robots are not for the engine, but for the site.

Answer 3

Regarding WordPress, a lot depends on what plugins you have enabled, how comments are organized on the site, and so on and so forth, but in principle, in order to understand how to describe robots.txt for your site on WordPress, you can read this article , everything is quite intelligible in my opinion

Answer 4

I used this instruction https://dampi.ru/pravilnyiy-robots-txt-dlya-sayta-...
The only thing the author claims is that if you open uploads for all bots, then uploaded PDF and other text files appear in the index . And in Yandex webmaster, in the "Excluded pages" report, error messages appear when indexing pictures, saying that the content is not supported. So I don’t know who to believe ... But just in case, I closed it anyway, everything seems to be fine