T
T
toddbarry2018-07-30 00:45:48
Python
toddbarry, 2018-07-30 00:45:48

Should I use asynchronous driver for postgres?

There is an application that must support a large number of websocket connections. In order to increase performance and study aiohttp in general, an asynchronous backend was launched (Later it will be proxied by nginx). After reading about the performance of a postgres driver called asyncpg, I googled an orm based on sqlalchemy that uses this driver. The ORM is called Gino.
I downloaded Gino, imported it, created a database and tables, tested queries. Everything works, and works well at first glance.
However, I'm wondering - is it worth it to run a database with an asynchronous driver? Isn't executing python code much longer than looking up information in the database? And wouldn't the standard multi-process paradigm of postgres work be more efficient, when a separate process is created for each connection.
Please share your opinion - when is the use of an asynchronous database appropriate, and when not?
P.S. The choice of an asynchronous backend server is not discussed and there are no plans to switch to a synchronous one.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
Sergey Gornostaev, 2018-07-30
@toddbarry

If we take any data of the same structure and volume, then sorting, grouping and aggregating this data in Python will be an order of magnitude slower than in a DBMS. Even if in Python this data will be located in memory. But transferring the result of data processing from the DBMS to Python code will take two orders of magnitude longer than processing them in Python. I/O is the slowest part. So slow that, against its background, everything else is insignificant costs. Asynchrony solves this problem, allowing the program to perform some useful action while waiting for I / O instead of being idle. But this only works if absolutely all code is asynchronous. If one call is blocking, the asynchrony of the rest of the call chain is useless.
Both are fine as long as you don't have more than 50 rps. First, each thread consumes server resources. Second, starting, stopping, and synchronizing threads incur additional overhead. These two points are also valid for processes, only what has been said can be safely multiplied by a thousand.

R
Roman Kitaev, 2018-07-30
@deliro

It’s worth at least because with a synchronous driver your entire event loop will wait for results from the database and there will be no sense from it
Plus, judging by the benchmarks, asyncpg is much faster than the others

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question