A
A
Artyom2022-02-28 20:17:08
robots.txt
Artyom, 2022-02-28 20:17:08

What files to exclude in robots.txt on a landing page?

Here is the file structure.
621d032c990ca669751374.jpeg
As I understand it, you need to disable phpMailer and the sendmail.php file.
Whether it is necessary to disable the script from the js folder (it contains the code for the sliders and the processing of buttons and feedback forms).
Do I need to disable the css folder? And the img folder and fonts

If I disable it based on my structure, am I doing it right?
User-agent: *
Disallow: sendmail.php
Disallow: /phpmailer
Disallow: /js
Disallow: /assets/css

Answer the question

In order to leave comments, you need to log in

1 answer(s)
T
ThunderCat, 2022-02-28
@danyvasnafig

firstly, it is desirable to move the folder with the code one directory higher, libraries should not be in the public, they must be connected from the overlying directory. According to your mind, you shouldn’t have sendmail.php there, but the index itself should be index.php, and requests from the landing page should also lead to it, and then look at the request method, variables, and so on. But this is if you really bother. In any case - take out phpmailer above.
For the rest, something like this would be enough:
Allow: index.html
Allow: /assets/img
Disallow: *
Leave the pictures so that Google can read them by name, although it seems like it will take it from the html, and font scripts / markup will not needed.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question