What approach to cron control do you use?

Question

Task Schedulers

There are dozens of periodic tasks on our projects, the result of which needs to be monitored.
The easiest way is to output the result of the cron execution to the console and get it all on the soap. At the same time, you have to read a ton of letters every day, and this does not guarantee that cron will be detected, which fell off and stopped running.
The current vision of the solution - placing cron in the database of the results of their work and displaying these results in munin, zabbix, nagios charts seems very laborious.
And how do you do it?

Answer 1

hmm, as a rule, I add to the script launched by the crown, a check for the success of the execution, if successful - we are silent, else - a notification to the soap. Or do you just have tons of unsuccessful ones?

Answer 2

Implemented (naturally in a team) a large social network, a bunch of task crowns. A lot of scripts should be constantly spinning (probably they can be called demons, although they are launched by cron).
I had to develop a system for deploying and managing tasks / scripts (tasks were running on different servers).
The basic principles are as follows:
All cron tasks inherit from a base class.
All cron tasks are launched from a single startup script (wrapper). Each cron task has a pid
file (in case two identical tasks do not start at the same time)
.
Each cron script in a centralized database (sharding is organized in the network, all data is divided into shards) threw off the data: launch start time, launch end time, how much was done (some measure, for example, the number of queue elements processed)
there were two analysis scripts.
the first script monitored the current data in the database, compared them with the template (how much should be) and gave the status 0 1 2
to the nagius sysadmin by nagius, if he saw that something was wrong, he launched the script monitoring table and watched the data on the scripts on it, which one, when and how did
such a multi-layer cake work

Answer 3

IMHO the script should be silent if everything is fine and swear informatively if something is wrong. Accordingly, information comes to the soap if something is needed.
Regular info is thrown into the BD for statistics, graphs and other things in the admin panel.

Answer 4

At one time, I wrote a crutch in python to send stdout via XMPP. It was done according to some blog (almost Habr), it turned out ~ fifty lines all about everything.
If there are too many messages, then only critical ones can be sent immediately, the rest can be collected in a kind of digests and sent once an hour or two.