Why are Habravchan profiles closed from indexing?

V

VBart2011-01-25 16:11:51

Habr

VBart, 2011-01-25 16:11:51

Example: vbart.habrahabr.ru/robots.txt
Why is this done? What is the profit from this?

Reply

Answer the question

In order to leave comments, you need to log in

4 answer(s)

P

Petrify, 2011-01-25
@Petrify

As an option - to hide personal data from bots from search engines - collectors thereof.

S

Sergey, 2011-01-25
@bondbig

mailto:[email protected]
Let me guess, I wanted to look for my own old comment? Yes, there is such a problem.

V

Vlad Zhivotnev, 2011-01-25
@inkvizitor68sl

In general, judging by this - www.google.ru/search?sourceid=chrome&ie=UTF-8&q=inkvizitor68sl - they are not very closed. Or closed recently.
By the way, all my articles are in the Google index at site: inkvizitor68sl.habrahabr.ru

L

lam0x86, 2011-01-25
@lam0x86

As far as I remember, quite a long time ago, one of the Habr admins complained that due to the specific profile model (each profile is a third-level domain), some search engines simultaneously set hundreds and thousands of their search robots on Habr, which is tantamount to a constant DDoS attack. Maybe that's why they closed it.