Task service for bots?

Question

Python

Good afternoon
There are many utilities that can perform simple work and they all act as a mini web server (single thread)
Which becomes inconvenient, since there are quite a lot of such utilities and several copies on different servers (to speed up work)
I want to make a single web server service to work with them.
1. To which clients (other programs) connect and set a task for processing (json parameters up to 1kb + files 1-100mb in size)
2. Utilities that periodically connect to this server to receive tasks.
3. It is desirable to track the status of the task, new, processing, error, completed
3.5 Part result if the task is for several utilities.
4. Minimal authorization
Are there already similar libraries?
At the moment I think to implement using nginx( client_body_in_file_only on ) + tornado (I want to try long pooling to retrieve results/tasks)

Answer 1

Do you need a message broker?
https://www.rabbitmq.com/
zeromq.org
https://kafka.apache.org/
activemq.apache.org