Answer the question
In order to leave comments, you need to log in
How to configure robots.txt so that the microsoft bot does not create a load?
Good afternoon! Please help me create the robots.txt file correctly. the fact is that the hosters write:
"Currently, php processes create the most load on your server, this load can be caused by visiting search bots on your site, for the current day the search bot from Microsoft has established a connection to your site more than 5000 times:
2188 207.46.13.106
3351 207.46.13.43
Try to restrict search bots in the robotx.txt file for your site using the crawl-delay directive. You can read the following instructions to implement this setting:
https://help.yandex.ru/webmaster/controlling -robot... "
Now the robots file is
User-agent: *
Disallow: /wp*
Can it do that?
User-agent: *
Disallow: /wp*
User-agent: Microsoft
Crawl-delay: 60
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question