Too smart GoogleBot, how to ban?

F

Fedor2011-03-13 16:45:19

Googlebot

Fedor, 2011-03-13 16:45:19

Hello!
On one of the sites there are buttons, by clicking on which Ajax automatically sends messages to the admin. Faced with the fact that GoogleBot clicks on these links when indexing the site and, accordingly, sends messages. I was especially surprised that by clicking on the link, the bot agrees to the scripted confirm. YandexBot, by the way, was not noticed in such a smart behavior :)

Actually, I would like to ask the experts how to prevent the bot from clicking on certain links? Just press. For nofollow and robots.txt doesn't help (it clicks but doesn't index).

Thanks in advance for your replies.

Reply

Answer the question

In order to leave comments, you need to log in

9 answer(s)

V

Vitaly Peretyatko, 2011-03-13
@FFF

It is necessary that your "messages to the admin" are sent not by GET, but by POST requests. Google will not send POSTs

A

Alexey, 2011-03-13
@alexxxst

Google is impudent... it even enters support chats by clicking the button in two steps... :) and support is trying to communicate with it... I had to write special rules for it...

B

bekbulatov, 2011-03-13
@bekbulatov

robots.txt?

A

AntonioK, 2011-03-13
@AntonioK

Have you tried writing to google support?

G

Gavrylyuk, 2011-03-13
@Gavrylyuk

put the link in the script

A

Andrey, 2011-03-13
@AndreyMorozov

Do a check for user-agent in the script for sending a message?

L

lashtal, 2011-03-13
@lashtal

it is possible to forbid indexing of the part of the page located between
<!––googleoff: all--> .... <!––googleon: all-->

S

Sergey Savostin, 2011-03-13
@savostin

Remove the URL from the link and proceed with Javascript after confirmation.
Goggle doesn't confirm confirm, it's just that you (probably) have it written like this:

<a href="/send.php" onclick="return confirm('Send?')">Send</a>

And Google only parses /send.php and moves on.

S

Sergey Savostin, 2011-03-13
@savostin

Yep, the "problem" seems to be over two years old and Google's algorithm is only getting better.
As I understand from the article on the link, you need to remove the explicit mention of urls, it can be encrypted somehow. But there is another option that the person working with the site has a google toolbar that hits all the urls. There is no getting away from this, except that on the server side, when requesting this url, determine the bot by user-agent (it seems to be not encrypted)