1
1
101-s2020-05-28 14:19:47
Django
101-s, 2020-05-28 14:19:47

What should be the server settings for large site pages?

Good {day or night} dear programmers, I have a few questions:
- what should be the server settings?
- What is the best way to implement such tasks?

Given: vds 512 MB of memory, it has django 3 + gunicorn + nginx It is necessary to display a large page in the admin panel, which is viewed by 1-2 people, which weighs 10 times more than usual, this is about 25 thousand records of product characteristics from the server settings
database

seems to be standard:

nginx
user nginx;
worker_processes auto; #колич ядер
worker_cpu_affinity auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    use epoll;
    worker_connections 1024;
    multi_accept on;
}

# директивы HTTP-сервера
http {
  server_names_hash_bucket_size 64; #увеличивает разм памяти
  
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65s;
    types_hash_max_size 2048;
    client_max_body_size 300M; #макс разм ф для загр
    fastcgi_read_timeout 3000; #врем ожид выполн пхп скрипта

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;
  
  gzip on;
    gzip_disable "msie6";
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;

    include /etc/nginx/conf.d/*.conf;
}



nginx for website
server {
    listen 80;
    server_name xxx;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static {
        root /var/www/8d/baza/;
    expires 30d;
    }

    location / {
    proxy_read_timeout 300s;
    proxy_connect_timeout 75s;
  
    #proxy_http_version 1.1; # you need to set this in order to use params below.
    #proxy_temp_file_write_size 8m;
    #proxy_max_temp_file_size 1024m;
    #proxy_connect_timeout 10080s;
    #proxy_send_timeout 10080s;
    #proxy_read_timeout 10080s;
    #proxy_buffer_size 8k; # первая часть В этой части ответа находится, как правило, небольшой заголовок ответа
    #proxy_buffers 4 4m; #Задаёт число и размер буферов для одного соединения
    #proxy_busy_buffers_size 12m; #минимум 4 4m - 4m ограничивает суммарный размер буферов, которые могут быть заняты для отправки ответа клиенту
    #proxy_redirect off;
    #proxy_request_buffering off;
    #proxy_buffering off;
    # end http_version 1.1
    
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://unix:/var/www/8d/baza/project/project.sock;
    }
}


MySQL:
[mysqld]
innodb_buffer_pool_size=32M
open_files_limit=65535


at the moment the server works without errors only with non-volume pages. After restarting the server, it works every other time. In other cases:
May 28 14:14:13 box-43747 kernel: Out of memory: Kill process 845 (gunicorn) score 319 or sacrifice child
May 28 14:14:13 box-43747 kernel: Killed process 845 (gunicorn), UID 1001, total-vm:440676kB, anon-rss:158716kB, file-rss:8kB, shmem-rss:0kB
May 28 14:14:13 box-43747 mysqld: 2020-05-28 14:14:13 27 [Warning] Aborted connection 27 to db: 'baza' user: 'baza' host: 'localhost' (Got an error reading communication packets)
May 28 14:14:13 box-43747 gunicorn: [2020-05-28 14:14:13 +0300] [1407] [INFO] Booting worker with pid: 1407

Answer the question

In order to leave comments, you need to log in

2 answer(s)
V
vreitech, 2020-05-28
@101-s

move the logic for receiving the list of records and its layout to the browser side, send records from the server to the browser in portions with such a volume that the server does not drop, via a websocket or whatever is more convenient for you.

V
Vitaly Karasik, 2020-05-28
@vitaly_il1

Given: vds 512 MB of memory, it has django 3 + gunicorn + nginx

Out of memory: Kill process 845 (gunicorn)

Only increase the server, not enough memory.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question