V
V
Vadim Timoshenko2019-04-09 16:45:07
Search Engine Optimization
Vadim Timoshenko, 2019-04-09 16:45:07

How to hide regional subdomains from Google?

I promote the site on subdomains in Yandex. Everything is successful.
I got to Google, and there is a message in the Search Console:


Aggressive spam
This site has shown signs of aggressive spam: posting automatically generated nonsense or content copied from other sites, masking, and/or repeatedly or flagrantly violating Google Webmaster Guidelines. More
Affects all pages

As I understand it, this is all because of the many duplicate pages on subdomains (although there are city names everywhere).
Tell me what is the best way to do it? Correctly, I understand that one of the ways is to remove the indexing of the site by Googl on subdomains, and leave it only on one domain. Will it help? How to do it? Tell me what to write in robots.txt.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Alexander Denisov, 2019-04-09
@Grinvind

robots.txt only applies to one host, i.e. site. Subdomains are different sites for search engines (albeit belonging to the same domain). You need to write the following in each subdomain in robots.txt:

User-agent: Googlebot
Disallow: /

In this way, you will make it clear to the Google search crawler that this site does not need to be indexed.

F
Felix_vek, 2019-04-12
@Felix_vek

In addition to Alexander's comment, I want to say that Google does not always follow the robots.txt prescription and can crawl the site and pages despite the prohibitions in it - see help : "Indexed despite blocking in the robots.txt file. The page was indexed, despite being blocked by robots.txt (Google always follows this file, but if other sites link to your page, robots.txt won't help.) This status is set because we're not sure you want to remove the page from search results. If this is your intention, a robots.txt file is not the correct way to avoid indexing. To prevent a page from being indexed, you must use the noindex tag or prevent anonymous access to it. "
Therefore, in addition to the ban in robots.txt, I advise you to install a meta tag for google robots on all pages of subdomains

<meta name="googlebot" content="noindex, nofollow">

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question