U
U
uaf0x2019-06-17 23:13:17
Nginx
uaf0x, 2019-06-17 23:13:17

How does nginx (or not nginx) generate 301 for all pages of a wordpress site?

Hello, the problem is that the bots go to the address and write a copy of 301 in the logs.
On our site, all pages end with /, with all the Strains without a slash, the same story.
All garbage from WP removed, wp-json, feeds, closed under a reliable 410 response.
The code is all checked up and down, i.e. theoretically, the robot cannot find a link without a slash to follow it.

server{
    listen 80;
    server_name site.com www.site.com;
    return 301 https://site.com$request_uri;
}


server {
    listen 443 default ssl http2;
    #listen 159.69.20.128:80 ssl http2;
    #listen 159.69.20.128:443 ssl http2;
    server_name site.com www.site.com;

    ssl on;
    ssl_certificate /home/ssl/site.com.pem;
    ssl_certificate_key /home/ssl/site.com.key;

    access_log /var/log/nginx/site.com.log;
    error_log  /var/log/nginx/site.com.log;
    index index.php;
    gzip on;
    gzip_disable "msie6";
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;
    
    root /var/www/site;
  
    rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.xml$ "/index.php?xml_sitemap=params=$2" last;
    rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.xml\.gz$ "/index.php?xml_sitemap=params=$2;zip=true" last;
    rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.html$ "/index.php?xml_sitemap=params=$2;html=true" last;
    rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.html.gz$ "/index.php?xml_sitemap=params=$2;html=true;zip=true" last;


    location / {
        try_files $uri $uri/ /index.php?$args/;
    }

    location /.well-known {
        allow all;
    }

    location = /favicon.ico {
        log_not_found off;
        access_log off;
        expires max;
    }

    location ~ \.php$ { 
        try_files $uri =404; 
        include /etc/nginx/fastcgi.conf;
        #fastcgi_pass unix:/run/php/php7.0-fpm.sock; 
      fastcgi_pass  127.0.0.1:9000;
    }

    location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
    }

    location ~ /\. {
        deny all;
    }

    location ~* \.(css|js|gif|jpg|png|svg|ttf|txt|doc|pdf|rtf|swf|xsl|woff|eot|woff2|ttf|svg|html|ico|less) {
        expires max;
        access_log off;
        log_not_found off;
    }


}

But what it writes in the logs, though it’s not clear where it goes before, redirects are written to a separate log.
172.69.62.243 - - [17/Jun/2019:21:53:55 +0200] "GET /shop/remen-16h11-1650/ HTTP/1.1" 200 42143 "-" "Mozilla/5.0 (compatible; SemrushBot/3~bl; +http://www.semrush.com/bot.html)"

162.158.79.190 - - [17/Jun/2019:21:53:53 +0200] "GET /shop/remen-16h11-1650/ HTTP/1.1" 301 194 "-" "Mozilla/5.0 (compatible; SemrushBot/3~bl; +http://www.semrush.com/bot.html)"

The question is how to win?

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question