Answer the question
In order to leave comments, you need to log in
Correct robots.txt for wordpress?
Good afternoon.
NOT when I didn’t think that finding robots for WordPress was such a problem. Rummaged through many blogs and everywhere everything is different, knowledge in this area. Who can share a robots file that actually works? I need to close everything superfluous, leave only pages, posts, and images.
Made 301 redirects.
But I also think that the robot should close everything in the file, but what exactly?
htaccess itself
RewriteCond %{QUERY_STRING} ^replytocom= [NC]
RewriteRule (.*) $1? [R=301,L]
RewriteRule (.+)/feed /$1 [R=301,L]
RewriteRule (.+)/attachment /$1 [R=301,L]
RewriteRule (.+)/comment-page /$1 [R=301,L]
RewriteRule (.+)/comments /$1 [R=301,L]
RewriteRule (.+)/trackback /$1 [R=301,L]
RewriteCond %{QUERY_STRING} ^attachment_id= [NC]
RewriteRule (.*) $1? [R=301,L]
While the robot looks like this:
User-agent: *
Disallow: /cgi-bin
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /wp-admin
Disallow: /wp- includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /wp-feed
User-agent: Yandex
Disallow: /cgi-bin
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /wp-feed
Host: site
Sitemap: site/sitemap.xml
User-agent: Googlebot-Image
Allow: /wp-content/uploads/
User-agent: YandexImages
Allow: /wp-content/uploads/
What else to add to the robot ?
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question