D
D
dmkaruev2020-11-17 13:21:07
robots.txt
dmkaruev, 2020-11-17 13:21:07

How to disable all parameters in robots.txt except for certain ones?

How to disable all GET parameters in robots.txt, except for certain ones. For example, such as: ?page= and ?tag=
That is, so that:
All parameters are not available for indexing Disallow
But the ?page= and ?tag= parameters are available Allow

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexander Denisov, 2020-11-18
@Grinvind

This is better to close the canonical.
If you are sure that you need to close from scanning, then try experimenting with this design:

Disallow: *?*
Allow: *?page=*
Allow: *?tag=*

And don't forget to open links to ?fbclid=, these links are often referred to by users, and if you close them, the link juice will not be transferred.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question