S
S
Sergey2014-07-09 08:43:11
robots.txt
Sergey, 2014-07-09 08:43:11

What is wrong with my WordPress robots.txt?

Here it is:
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /webstat/
Disallow: /feed/
Disallow: /trackback
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow : /category/*/*
Disallow: /wp-content/plugins
Disallow: /wp-content/themes
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /comments
Host and Sitemap are also written in the line below.
Why do I have 214 pages blocked in Google webmaster, and only 378 pages on the site, 197 were blocked the day before yesterday.
Which line can block pages, yet it seems to be standard?

Answer the question

In order to leave comments, you need to log in

3 answer(s)
D
Dmitry Entelis, 2014-07-09
@DmitriyEntelis

Firstly, I am confused by the line Disallow: /category/* /*
Secondly, it may be worth explicitly adding
User-agent: *
Allow: * to the end

O
Oleg Krasnov, 2014-07-09
@OKrasnov

Use a tool from google:
Webmaster Tools - Blocked URLs

T
Timur Biktashev, 2014-07-10
@tizar

I use this build. I don’t remember where I got it from - it was in 2011, when I was just mastering WordPress and making my first website on it. I added something from myself and to this day I use this "design". Perhaps someone will tell me about the jambs.

User-agent: *
Disallow: /cgi-bin 
Disallow: /xmlrpc.php 
Disallow: /wp-admin 
Disallow: /wp-includes 
Disallow: /wp-content/plugins 
Disallow: /wp-content/cache 
Disallow: /wp-content/themes 
Disallow: /trackback 
Disallow: */trackback 
Disallow: */*/trackback 
Disallow: */*/feed/*/ 
Disallow: */feed 
Disallow: /*?* 
Disallow: /*? 
Disallow: /?s= 
Disallow: *?replytocom 
Disallow: */comments 
Disallow: /comments 
Disallow: */comment-page-* 
Disallow: /tag 
Disallow: /archive 
Disallow: /attachment* 

Host: site.ru 

Sitemap: http://site.ru/sitemap.xml 
Sitemap: http://site.ru/sitemap.xml.gz

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question