How to quickly and safely monitor a bunch of bots?

Question

Python

There are a bunch of bots (more than 1000 standalone applications) that perform certain operations and give information about their completion and current status (50 separate fields each). How to solve the problem without using a database connection for each of them i.e. ensure independence? Information must be sent at least once per second.

Answer 1

backdoor_win well, zeromq is too raw. Fits well Gearman

Answer 2

You can implement a separate service to which all these applications will send data via the API, then only this one application will have a connection to the database.