M
M
Maxim2014-05-05 23:04:19
PHP
Maxim, 2014-05-05 23:04:19

How to make one robots.txt for all subdomains?

The situation is this, the site has a large number of dynamic subdomains, such as username.site.com, they are all processed by the script of the main domain, the rules in .htaccess

RewriteCond %{HTTP_HOST} ^(www\.)?([^\.]+)\.site\.com$ [NC]
RewriteRule (.*) index.php?cn=%2 [NC,QSA]

Accordingly, when the crawler passes, it goes to username.site.com/robots.txt which is not there and cannot be, because it is located in the root of the site and is available at site.com/robots.txt. How to redirect the robot to the main robots.txt?
I read something about symlinks but didn’t figure out how to set it up correctly, or maybe mod_rewrite will help out again?)

Answer the question

In order to leave comments, you need to log in

3 answer(s)
V
Viktor Vsk, 2014-05-05
@viktorvsk

Do you have a username.site.com file structure that is dynamic and is set once for all? Not on each user after all the directory?
Then you just need to add a symlink to this dynamic directory. Nothing complicated. A symlink is like a shortcut in Windows.
If there is no graphic interface, then all that

[email protected]~$ cd /path/to/usersite/folder
[email protected]~$ ln -s /path/to/site.com/robots.txt robots.txt

Programming on a webserver is still not a good idea

S
svd71, 2014-05-06
@svd71

why don't you rewrite a certain script that will determine the access to robots.txt and give out the necessary content, otherwise load the requested script?

V
Vlad Zhivotnev, 2014-05-06
@inkvizitor68sl

In the apache config, this is done like this:
In the nginx config:
It should work in htaccess too, but the mb syntax is different.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question