How to prevent search engines from indexing private content?

Question

Search engines

I remember in July 2012 there was a boom in the discussion of the issue of content by search engines, which, in theory, should not have been issued. Even in those cases when the private sections of the site were protected by user authorization.
There are some sections on my site that I would not like to see in the search results. Actually, the subject is how to properly configure directories, robots.txt, and possibly something else to prevent the user's private content from getting into the public domain.
Authorization is used on my site, only after that the user can go to his section and see his content

Answer 1

robots.txt doesn't always work (why is an open question), but the tag meta content="noindex,nofollow" name="robots"hasn't failed yet.

Answer 2

You can, in addition to robots.txt (for example, generate it here ), wrap the content with tags (see link ):<noindex>...</noindex>

Answer 3

In robots.txt
User-agent: *
Disallow: /section_root
Well, for my own peace of mind, also in head in all personal sections:
<meta name=“robots” content=“none”>
none - replaces noindex and nofollwo

Answer 4

Well, here google-bot even scores on basic authorization, it feels like it leaks passwords and indexes closed sites with absolutely impunity, absolutely hammering on robots.txt. After that, it’s not even clear at all how to defend yourself, apparently it’s stupid not to use chrome, just kick it off on the user agent so that it is not possible to merge the basic authorization password.

Answer 5

Well, I do not have basic authorization and I really like chrome.

Answer 6

https won't help?

Answer 7

well, first you need to understand how the robot gets to pages that are private to you ... if, as it is supposed here, Google Chrome merges passwords “by itself” :) then you should still have an authorized session “merged” in the logs, if not, then you have PROBLEMS, because the robot goes where it was not called just like that :).
Well, cut it off your shoulder, in general, the robot is quite easy to detect, do not let it eat what is not supposed to - there will be nothing to index.