X
X
xmoonlight2018-08-22 15:51:15
Google Adsense
xmoonlight, 2018-08-22 15:51:15

The rules in robots.txt affect the display of banners and external resources! Who faced and how did you solve it?

Good day everyone.
I ran into a problem (everything worked fine before)...
When checking the page from the Google webmaster console, it says that
all external links, including banners and counters, are blocked by the robots.txt file!
Banners just don't show up...
(Host: was specified before and is now specified...)
Does robots.txt work on external resources (I don't remember something like that)?!
Who has already experienced something like this?
What is this bug and how to fight?
(the reason is simple: a lot of "left" urls that don't need to be opened from Disallow)
UPD: Code from robots.txt:

User-agent: *
Host: https://domain.com
Disallow: /cache/
Disallow: /cli/
Disallow: /*.ttf
Disallow: /*.svg
Disallow: /*.pdf
Disallow: /*.txt
Disallow: /*.doc
Disallow: /*.css
Disallow: /*.js

Disallow: *?ax=boost&print=1
Disallow: */?
Disallow: *?*format=
Disallow: *?*=rss
Disallow: */index/
Disallow: */index.html$

Allow: */pagead/*
Allow: /

Allow: */pagead/* и Allow: /added recently...

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
VVB, 2018-08-23
@SEOBot

Былобы лучше если вы выложили свой файл robots.txt чтобы мы не угадывали причины. Но а так , можно предположить :
- у вас в файле robots.txt расписаны правила для разных ботов (не одно правило для ВСЕХ ботов, а отдельные правила для разных ботов)
- у вас в файле для ботов (краулеров) которые отвечают за медийку стоит (вероятно) Disallow: / - что запрещает краулеру заходить на страницу, таким образом он не может проверить, удовлетворяет ли контент вашей страницы требованиям.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question