D
D
drveb12018-11-02 15:21:31
Search Engine Optimization
drveb1, 2018-11-02 15:21:31

Which version of robots.txt is correct?

Hello!
There is a WordPress blog.
2 freelancers made 2 different robots.txt
Help me choose the right version, I'm afraid to make a mistake.
Version 1:
User-agent: *
Disallow: /cgi-bin
Disallow: /? # all query options on the main
page Disallow: /wp- # all WP files: /wp-json/, /wp-includes, /wp-content/plugins
Disallow: *?s= # search
Disallow: *&s= # search
Disallow: /search # search
Disallow: /author/ # author's archive
Disallow: *?attachment_id= # attachment page. It's actually a redirect...
Disallow: */feed # all feeds
Disallow: */rss # rss feed
Disallow: */embed # all embeds
Disallow: */page/ # all types of pagination
Disallow: /shag-2/ # what you asked to close
Allow: */uploads # open uploads
Allow: /*/*.js # inside /wp- (/*/ - for priority)
Allow: /*/*.css # inside /wp- (/*/ - for priority)
Allow: /wp-*.png # images in plugins, cache folder, etc. d.
Allow: /wp-*.jpg # images in plugins, cache folder, etc.
Allow: /wp-*.jpeg # images in plugins, cache folder, etc.
Allow: /wp-*.gif # pictures in plugins, cache folder, etc.
Allow: /wp-*.svg # images in plugins, cache folder, etc.
Allow: /wp-*.pdf # files in plugins, cache folder, etc.
Sitemap: http://mysite.ru/sitemap-1.xml #sitemap Sitemap
: http://mysite.ru/image-sitemap-1.xml
Host: mysite.ru
Version 2:
User-agent: *
Disallow: /cgi-bin
Disallow: /?
Disallow: /wp-
Disallow: /wp/
Disallow: *?s=
Disallow: *&s=
Disallow: /search/
Disallow: /author/
Disallow: *?attachment_id=
Disallow: /users/
Disallow: */trackback
Disallow: * /feed
Disallow: */rss
Disallow: */embed
Disallow: */page/
Disallow: /shag-2/
Disallow: /xmlrpc.php
Disallow: *utm*=
Disallow: *openstat=
Allow: */uploads
User-agent: GoogleBot
Disallow: /cgi-bin
Disallow: /?
Disallow: /wp-
Disallow: /wp/
Disallow: *?s=
Disallow: *&s=
Disallow: /search/
Disallow: /author/
Disallow: *?attachment_id=
Disallow: /users/
Disallow: */trackback
Disallow: * /feed
Disallow: */rss
Disallow: */embed
Disallow: */page/
Disallow: /shag-2/
Disallow: /xmlrpc.php
Disallow: *utm*=
Disallow: *openstat=
Allow: */uploads
Allow: / */*.js
Allow: /*/*.css
Allow: /wp-*.png
Allow: /wp-*.jpg
Allow: /wp-*.jpeg
Allow: /wp-*.gif
Allow: /wp-admin/admin-ajax.php
User-agent: Yandex
Disallow: /cgi-bin
Disallow: / ?
Disallow: /wp-
Disallow: /wp/
Disallow: *?s=
Disallow: *&s=
Disallow: /search/
Disallow: /author/
Disallow: *?attachment_id=
Disallow: /users/
Disallow: */trackback
Disallow: * /feed
Disallow: */rss
Disallow: */embed
Disallow: */page/
Disallow: /shag-2/
Disallow: /xmlrpc.php
Allow: */uploads
Allow: /*/*.js
Allow: /*/* .css
Allow: /wp-*.png
Allow: /wp-*.jpg
Allow: /wp-*.jpeg
Allow: /wp-*.gif
Allow: /wp-admin/admin-ajax.php
Clean-Param: utm_source&utm_medium&utm_campaign
Clean-Param: openstat
Sitemap : http://mysite.ru/sitemap- 1.xml
Sitemap: http://mysite.ru/image-sitemap-1.xml
Host: mysite.ru

Answer the question

In order to leave comments, you need to log in

2 answer(s)
M
Malazar, 2018-11-02
@Malazar

If users from other search engines are not interested. Then you can leave the second option.

M
Mars, 2018-11-08
@Marsseo73

The second option is generally stupidly taken from the Internet. Is it really better to pay extra money for some unknown than to type in search engines "Correct robots.txt for Wordpress?".

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question