M
M
master2909882019-12-11 20:01:26
Nginx
master290988, 2019-12-11 20:01:26

How to prevent address indexing by a piece of URL in NGINX?

Google webmaster constantly gets addresses that are closed in robots txt with the wording "Indexed despite blocking in the robots.txt file". However, I do not know where and where Google finds them. When you go to the site using this URL, the code 404 is issued. Therefore, the robots meta tag will not help here.

Found another way to close such pages, using the X-Robots-Tag "noindex". In the manual from Google - there is an instruction on how to close using the X-Robots-Tag "noindex".

There is the following code for NGINX:

location ~* \.pdf$ {
  add_header X-Robots-Tag "noindex, nofollow";
}


In my case, you need to close the url piece by piece, for example by /comment-subscriptions/ or srsrc=f the
url itself looks like this

instead of .pdf (as in the example)?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
dodo512, 2019-12-11
@master290988

location /comment-subscriptions/ {
    add_header X-Robots-Tag "noindex, nofollow" always;
}

location ~ \.php$ {
    if ($arg_srsrc = "f") {
        add_header X-Robots-Tag "noindex, nofollow" always;
    }

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question