Z
Z
ZaraMc2016-03-16 22:10:03
Search Engine Optimization
ZaraMc, 2016-03-16 22:10:03

First Disallow or Allow in robots?

Hello! Googled and everywhere opinions are divided into two camps: first we prohibit the section, then we allow what is needed and first we allow what is needed, then we prohibit

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
Vladislav Yanovsky, 2016-03-16
@kopcap_va

Yandex opinion:
The Allow and Disallow directives from the respective User-agent block are sorted by URL prefix length (lowest to longest) and applied sequentially. If several directives are suitable for a given page of the site, then the robot selects the last one in the order of appearance in the sorted list. Thus, the order of the directives in the robots.txt file does not affect how the robot uses them.
Link to Yandex Help Page
Opinion Google:
At the group level, specifically for the allow and disallow directives, the strictest rule based on the length of the [path] entry takes precedence over the less strict and shorter rule. The order of precedence for wildcard rules is undefined.

  • The order in which the directives are specified does not matter;
  • If you are worried about incorrectly written instructions, then Google and Yandex webmaster offices have tools for checking robots.txt files - you can check whether your instructions are being followed correctly.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question