R
R
raman2352015-01-15 17:17:10
Google Webmaster Tools
raman235, 2015-01-15 17:17:10

Error in Google Webmaster. Why is access to the URL blocked in the robots.txt file?

Hello. Faced such a problem. Created in CMS 1C-Bitrix, using built-in services, robot.txt and sitemap.xml files (sitemap), added robot.txt file to Google Webmasters, no errors or warnings were found by Google. By the way, this is how my robot.txt looks like:
User-Agent: *
Disallow: */index.php
Disallow: /bitrix/
Disallow: /*show_include_exec_time=
Disallow: /*show_page_exec_time=
Disallow: /*show_sql_stat=
Disallow: /*bitrix_include_areas =
Disallow: /*clear_cache=
Disallow: /*clear_cache_session=
Disallow: /*ADD_TO_COMPARE_LIST
Disallow: /*ORDER_BY
Disallow: /*PAGEN
Disallow: /*?print=
Disallow: /*&print=
Disallow: /*print_course=
Disallow: /*?action=
Disallow: /*&action=
Disallow: /*register=
Disallow: /*forgot_password=
Disallow: /*change_password=
Disallow: /*login=
Disallow: /*logout=
Disallow : /*auth=
Disallow: /*backurl=
Disallow: /*back_url=
Disallow: /*BACKURL=
Disallow: /*BACK_URL=
Disallow: /*back_url_admin=
Disallow: /*?utm_source=
Host: mysite.by
Sitemap: http ://mysite.by/sitemap.xml
The real fun started after I added the sitemap.xml to Google Webmasters. It turns out that robot.txt blocks absolutely all links that sitemap.xml contains, while at the same time I didn’t pervert over robot.txt, with the help of which manuals I wouldn’t create it again, by the way, the same applies to sitemap.xml, I do it too I tried to create not only using the built-in service, anyway robot.txt blocks all links in sitemap.xml . Has anyone experienced this and knows what needs to be done?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexey Emelyanov, 2015-01-15
@babarun

Explain what does " robot.txt blocks all links in sitemap.xml " mean?
In your example, there are no rules that would exclude all pages. Perhaps in your sitemap all sections have the /index.php ending , which will be excluded by the first rule.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question