What do you do: monitor, make alerts, react to alerts on your servers?

Question

Monitoring

There is one server on linux that behaves unstable. Got it by inheritance.
I want to overlay it with monitoring and make alerts based on the results of which I will locally restart the service on it and, if it is possible, remotely remove it from the cluster on HAProxy.
Consistently, it looks like this:
make monitoring and display values ​​in graphs
make alerting for values
​​make scripts (?) for alerts that restart the service and / or remove the server from HAProxy
At each stage, as I understand it, there is a tool.
Perhaps you have solved similar tasks of monitoring and instant response.
Share your experience and the solutions you use.

Answer 1

Monitoring in zabbix prometeus charts
Monitoring with nagios alerts and its derivatives - centrion icinga2
Scripts for alerts, etc., for managing all sorts of things, for example, the keepalived daemon specializes in managing VRRP, backends for all balancers, etc.