What kind of strange multithreading is in tarantool that it does not hold the load?

Question

Database

In general, I consider tarantool to be something between the bottom and normal technology, you can see what a mess there is in the repositories, with connectors, something is abandoned and not updated, something is updated without a changelog.

At the moment I'm interested in multithreading.

During the experiments, we noticed that if you do a lot of things in stored procedures in Lua, and call these procedures simultaneously by several clients, then it starts to slow down very much. Well, that is, the load does not hold at all.

One might think that Tarantool is generally single-threaded. But there are still some fibers there, and the study shows that each storage is spinning in its own fiber.
And besides, there are libs for the tarantula, where there are generally some "signals" (or "locks"), and there is a function that waits in the while loop (with sleep, of course!) When this signal will work, and this function must be called from Lua, that is, storage. But, as I said, if you have at least 10-20 clients, then you can consider this system as single-threaded, with the corresponding perspectives for these while.

If we take this logic out of the tarantula's storages into the service that calls them, and make the storages small and as atomic as possible, then the system will immediately hold hundreds, or even thousands of clients at the same time. And the more atomic the storage, the faster everything works, and even with 2-3 (!) Clients, you can notice the difference. (The CPU we have has 32 cores, if anything.)

What kind of "single-threaded multithreading" is this, for whom is it done, why?

Answer 1

https://www.tarantool.io/en/doc/latest/book/box/at...

Cooperative multitasking means: unless a running fiber deliberately yields control, it is not preempted by some other fiber. But a running fiber will deliberately yield when it encounters a “yield point”: a transaction commit, an operating system call, or an explicit “yield” request.

Transactions in Tarantool occur in fibers on a single thread. That is why Tarantool has a guarantee of execution atomicity.

Ever heard of cooperative multitasking ? It's just about "But there are still some fiber". I propose to study, for sure the answer lies there.

And the more atomic the storage, the faster everything works

More than a logical result, it's like doing yield in the right places in one big store.