L
L
Lander2017-09-07 15:41:13
Laravel
Lander, 2017-09-07 15:41:13

How to properly use Allow and Disallow in robots.txt?

Good afternoon.
The site has a bunch of "garbage" links like:

http://site.com/slug_link_1/temp_name_1/
http://site.com/slug_link_2/temp_name_2/
http://site.com/slug_link_3/temp_name_3/

I'm trying to prohibit their indexing in robots.txt, but so that links like site.com/category/it are indexed. I write in the file:
...
Disallow: /*/*/
Allow: /category/*/
...

I doubt whether the rules are written in the correct order? Or is it still necessary to first allow categories, and then prohibit everything else?
Thanks in advance for your replies.

Answer the question

In order to leave comments, you need to log in

4 answer(s)
J
jazzus, 2019-05-20
@dargezrogue

With the help of with
It is necessary to make relations in the Order model for each table with which there is a connection.
Then collect them all in the controller into orders

// в зависимости от логики связи
$orders = Order::with('user.cart.lots')
               ->get();

All data will be collected in one collection

E
Eugene Wolf, 2019-05-20
@Wolfnsex

In my opinion, there are several options here:
1. Use links at the Laravel level
2. Create a view with the necessary data in the database and make a selection from it
3. Using DB::select, create a query in which you will join the tables using the query as you need

E
ettychel, 2019-06-06
@ettychel

Can be done using Eloquent
In the orders model, write 2 belongsTo
Then write a query
And now I'm looking and I don't understand why you have orders.lot_id associated with cart.id? Maybe the connection should be made from orders.lot_id to lots.id?

D
Dmitry Dart, 2017-09-07
@usdglander

If there are more junk, then disable everything and allow selected ones, for example:

Disallow: /
Allow: /category/it

If in doubt, there is a tool for checking robots.txt inside the Yandex webmaster, you can set the rules right there in the window, then enter the URL and see if it will be indexed by them, use it.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question