How to properly export application metrics to Prometheus in Python?

Question

Python

Hi everyone

We have

• Several servers running various python scripts (run via supervisor). For example 15 pieces per server. Suppose scripts take tasks from the queue and, for example, convert pictures, or send letters
  
• Inside each script there are various metrics (let's call them application metrics)
  

What do you want?

• Send metrics from these scripts to Prometheus and then draw them beautifully in grafana
  

Actually the question is how to do it right? From what I googled on the Internet and tried, this is connecting prometheus_client, launching a built-in server in one of the scripts and "give" a couple of my metrics. How to give metrics from several scripts is not entirely clear. So far, the solution seems to be that you need to write them somewhere in the logs (memcache, redis) and make your own "exporter", which will give them to Prometheus. But there is a feeling that this is some kind of crutch solution.

Answer 1

Prometheus is a monitoring server that regularly calls hosts that provide metrics on demand. Especially for this, libraries are needed, with the help of which they collect metrics and give them through a web service. Each host must be accessible to Prometheus on the local network.
prometheus_client has documentation and usage examples.

Answer 2

https://github.com/prometheus/client_python