Uwsgi process periodically takes all the memory?

Question

Django

Good day.
I run into this problem from time to time. A bunch of VPS, Django, Nginx + Uwsgi is working, the server freezes due to the fact that one of the uwsgi processes starts eating memory and does not stop until it eats everything including swop, can someone tell me how to limit such disgrace!
here is my uwsgi.ini

[uwsgi]
master = true
processes = 5
buffer-size=65535
socket = /tmp/stp.sock
pidfile2 = /tmp/stp.pid
chmod-socket = 666
enable-threads = true
harakiri = 40
reload-on-as = 1100
reload-on-rss= 512
# idle = 10
# die-on-idle = true
wsgi-file=/home/*/stp/stp/wsgi.py

chdir=/home/al*/stp
daemonize=/var/log/uwsgi/*.log
max-requests=5000
plugin=python3
virtualenv=/home/*/envs/stp-env
venv=/home/*/envs/stp-env

what other settings can be applied in ini

Answer 1

Considering that the workers restart every 5000 requests, the memory footprint here is (I'm 99.9% sure) related to your code. By itself, uWSGI eats VERY little memory.
Is DEBUG disabled? With debug enabled, all SQL queries remain in django.db.connection.queries. If you still do not optimize N + 1 requests, then 500-2000 requests may well come to each page render. Multiplying by 5000, it turns out up to 10 million records. Each of them is a dictionary, which, let's say, occupies 200 bytes of memory. Inside contains the SQL itself, with a size of, say, 80-400 bytes and the execution time.
In total, if we take it to the maximum, each entry will take approximately 600 bytes * 10 million = 6 billion bytes, which is approximately equal to 5.6 GB. For each worker (after all, each worker you have is most likely a process, not a thread). If there are 4 workers, it turns out more than 20GB.
This is one of many optimization vectors.