R
R
ruslanbm292021-09-05 20:36:41
Google
ruslanbm29, 2021-09-05 20:36:41

How to disable indexing of navigation pages in robots.txt?

Hello, is it possible to prohibit indexing of pages with navigation that look like

For example
/ru/msk/vrachi/?page=1
/ru/msk/vrachi/?page=2
/ru/msk/clinic/?page=1
/ru/novosibirsk/clinic/?page=4
/ru/novosibirsk/ vrachi/?page=2
/ru/novosibirsk/vrachi/venerologi/?page=15
/ru/novosibirsk/vrachi/ginekology/?page=32

The problem is that there are a lot of categories and cities on the site. But all navigation pages end with "?page={Page number}"
I foresaw in advance that these pages would not get into the search results. Initially, I had a tag on pages with navigation that prohibited indexing. But somehow, the Yandex robot anyway collected navigation pages, indexed them, and even had traffic on them, and after a couple of months, it finally started spitting out navigation pages with the wording in Webmaster "prohibited from indexing" and attendance began to fall exactly the one that was on these navigation pages .. That is, the meta tag - somehow it does not work correctly. Therefore, we need solutions at the robots.txt level . Let the robot go through the pages themselves, because they have lists of doctors for indexing.

<meta name="robots" content="noindex">


But don't add navigation pages to the search results.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
A
Alexey Gnevyshev, 2021-09-05
@ruslanbm29

Disallow: /*?page=
But more often - on the contrary, robots can be ignored, but meta is more significant.

A
Alexander Denisov, 2021-09-06
@Grinvind

By closing pagination in the robots, you limit the relinking and understanding of your assortment on the listings by the search engine. So wedge couples to help you, and no robots.

E
Eugene, 2021-09-08
@Eugeny1987

Set rel="canonicle" for pagination pages
https://vc.ru/seo/72710-atribut-rel-canonical

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question