N
N
Nostromo2017-08-10 10:36:25
Google
Nostromo, 2017-08-10 10:36:25

Optimal robot.txt for Wordpress under Google and Yandex?

Good afternoon !
We need an optimal robot.txt for Google and Yandex search engines, there is a lot of information on the network, so I can’t choose someone’s specific option, as there are a lot of disputes in the comments.
Please post a safe and valid, tested version of robot.txt
Thank you for your attention!

Answer the question

In order to leave comments, you need to log in

3 answer(s)
S
Sue Bass, 2017-08-10
@suebass

The file format of robots.txt will remain the same. You should check below resource for to create robots.txt for WordPress for Google and Yandex web crawlers:
www.robotstxt.org/robotstxt.html
https://yandex.com/support/webmaster/controlling-r...
https:/ /support.google.com/webmasters/answer/60625...

B
Barmunk, 2017-08-10
@Barmunk

We need an optimal robot.txt for Google and Yandex search engines, there is a lot of information on the network, so I can’t choose someone’s specific option, as there are a lot of disputes in the comments.

Would you like to have the same disputes here? =) I use the standard robots.txt, and I forbid it already through Ruling this all, Yoast SEO helps me a lot

B
Bender_Render, 2017-08-11
@Bender_Render

I use this one.
User-agent: *
Disallow: /wp-admin
Disallow: /wp-content
Disallow: /wp-includes
Disallow: /wp-comments
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: */trackback
Disallow: */page/*
Disallow: /xmlrpc.php
Disallow: /wp-json/
Disallow: */feed
Disallow: */author
User-agent: Googlebot-Image
Allow: /wp-content/uploads/
User-agent: YandexImages
Allow: /wp-content/uploads/
host: site.ru
Sitemap: site.ru/sitemap.xml

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question