How to teach several modules to communicate in Python?

Question

Python

There are several Python modules. Located on different servers.
It is planned to use 1 server (processor) and several clients (parsers).
Considered python sockets and kafka.
Tell us, share your experience, how to build interaction correctly? What tools to use?

Answer 1

To implement an architectural approach with data processing through workers for Python, Celery is relevant . He has a good first steps guide that will help you raise the broker and understand the basics of work. It is possible to work with various broker options (instead of Kafka).
If the presence of a broker is not necessary and you can get by with simple HTTP connections (I assume this is due to the mention of sockets), I recommend looking towards Bottle and making requests through the Requests library .