How to optimize a server for 5 million hits?

Question

Python

Good afternoon!
There is a certain project written in Python using Django. The service does not have any special logic, the essence is quite simple: when you go to the appropriate URL, we write some data to the session and statistics in the database, and redirect to another URL.
PostgreSQL is used as a database, sessions are stored in Redis.
The server is quite powerful 4 cores (8 threads), 8 RAM.
During tests, Nginx logs began to appear messages like:

(11: Resource temporarily unavailable)

It can be seen that the load on the server was quite small, about 5-6% per core.
Accordingly, the question is how to optimize all this for such loads?

Answer 1

The problem is that sooner or later you will still run into the inability to increase the number of workers. Separate the collection of statistics directly and storing it in a relatively slow database. That is, going to the URL does only fast work (increase the counter in the conditional radish / memcache, write something to the session) and frees the worker to service the next request, and saving to the database is done by individual workers with some frequency.
You can look in the direction of the Nginx + Lua bundle, their capabilities may be enough for the "fast" part.

Answer 2

Perhaps the main problem lies in parallel threads. When the limit is reached, the server issues such a message. Check and increase this value in the configuration files.

Answer 3

Tune number of processes in uwsgi
Tune kernel
serverfault.com/questions/398972/need-to-increase-...
IMO not enough RAM