I
I
Igor Ivanov2020-10-20 13:47:25
Google
Igor Ivanov, 2020-10-20 13:47:25

Googlebot indexed the URL with parameters, although there is a ban in robots.txt?

Good afternoon!

Googlebot smartphone indexed URL with request parameters, although there is a prohibit ( Disallow) in robots.txt.

Google Search Console, "Index" menu, "Coverage" submenu, "Warnings" tab. The warning is: "Indexed, though blocked by robots.txt". There are 19 URLs like this: https://domain.com/?wordfence_lh=1&hid=6E50893D6BE... (I intentionally don't specify the real domain)

Wordfence is a security plugin. The link accordingly appears from it.

Plugin support did not give a clear answer. Here:

Are you seeing these URLs in your console? If so, that's normal behavior.
I understand that this is not an error but a warning. But still, I think this affects the completeness and correctness of indexing.

SEO experts, what are your assumptions? I would be grateful for any realistic versions.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Ali Profi, 2020-10-29
@aliprofi

Robots.txt - this is just a recommendation for search engines, they often ignore it. You need to set rel=canonical for main pages.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question