How to distribute threads in asynchronous programming?

Question

Programming

Greetings. Imagine an application where all possible branches are performed in a separate thread (if it is safe, for example, processing an array element by element), while there are exactly as many threads allocated to the entire application as there are processor cores available. If someone does not have enough thread, the callback is added to the queue and waits for a free thread.
Question: How could such a system be designed?
Ideally, I see the system like this. When the next thread is about to branch, all callbacks are added to the application's common queue. When a thread terminates, the next callback from the message queue is passed to it for execution. The problem with this approach is that it takes time to synchronize, and with a large queue, it can be faster to process small callbacks sequentially in one thread.
Another option: a non-synchronized array of threads, while some range of free threads is transferred to the new active thread, which can be issued further along the chain (the range is split), and if someone does not have enough, they will wait in the parent thread. The problem here is that the number of cores is always limited, and with this approach, unused threads will often be idle in some long single-threaded callback, to which several threads have been allocated (even if they are useful, but not for long).
Is it possible to choose the best option?

Answer 1

What you describe is very suitable for the classic producer / consumer pattern (producer / consumer) For some reason, there are few sources in Russian, but there are plenty of English ones.
There are also ready-made implementations of this template.
Yes, and it is not at all necessary to limit the number of threads to the number of cores. You may well use more threads, I advise you to experiment with the number, measuring the speed of execution, until you achieve the optimal result.