Answer the question
In order to leave comments, you need to log in
What do the different user-agents in robots.txt mean?
Good afternoon!
There is robots.txt:
User-Agent: Yandex
Disallow: /doc
Disallow: /promo
Disallow: /dev
Disallow: /client
Disallow: /wiki
Disallow: /statistics
Disallow: /statistics_client
Disallow: /dima
Disallow: /testovyi
Disallow: /*search.php
Disallow: *sphrase_id=*
Disallow: *openstat=*
Disallow: *from=*
Disallow: *gclid=*
Disallow: *utm*
Disallow: *yclid=*
Disallow: /*forgot_password
Disallow: /*register
Disallow: /*change_password
Disallow: *offset=*
Disallow: *CODE=*
Disallow: /*logo.swf
Disallow: /*.js
Disallow: /new
Disallow: *bitrix_include
Disallow: *bitrix/spread.php
Disallow: *clear_cache
Disallow: /samostoyatelno/?*
Disallow: /samostoyatelno/*?*
Disallow: /samostoyatelno/*/?
Disallow: /samostoyatelno/*/*?*
Disallow: /samostoyatelno/*/*/*?*
Disallow: *SECTION_CODE=*
Disallow: *otp.php*
Disallow: *admin*
Disallow: *login=*
Disallow: *seoorder.php*
Disallow: /?%ECx
Disallow: /poiskovoe_prodvizhenie_sajtov/trafik/?*
Disallow: /studio/clients/old/
Disallow: /studio/bill/$
Disallow: /mobileform.php
Disallow: *feedback.php*
Disallow: *template=*
Disallow: /pixelplus-2015.php
Clean-param: CODE /studio/stat/
Host: www.pixelplus.ru
User-Agent: *
Disallow: /doc
Disallow: /promo
Disallow: /dev
Disallow: /client
Disallow: /wiki
Disallow: /statistics
Disallow: /statistics_client
Disallow: /dima
Disallow: /testovyi
Disallow: /*search.php
Disallow: *sphrase_id=*
Disallow: *openstat=*
Disallow: *from=*
Disallow: *gclid=*
Disallow: *utm*
Disallow: *yclid=*
Disallow: /*forgot_password
Disallow: /*register
Disallow: /*change_password
Disallow: *offset=*
Disallow: *CODE=*
Disallow: /*logo.swf
Disallow: /*.js
Disallow: /new
Disallow: *bitrix_include
Disallow: *bitrix/spread.php
Disallow: *clear_cache
Disallow: /samostoyatelno/?*
Disallow: /samostoyatelno/*?*
Disallow: /samostoyatelno/*/?
Disallow: /samostoyatelno/*/*?*
Disallow: /samostoyatelno/*/*/*?*
Disallow: *SECTION_CODE=*
Disallow: *otp.php*
Disallow: *admin*
Disallow: *login=*
Disallow: *seoorder.php*
Disallow: /?%ECx
Disallow: /poiskovoe_prodvizhenie_sajtov/trafik/?*
Disallow: /studio/clients/old/
Disallow: /studio/bill/$
Disallow: /mobileform.php
Disallow: *feedback.php*
Disallow: *template=*
Disallow: /pixelplus-2015.php
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question