How to correctly monitor the sequential progress of the console utilities?

Question

linux

There is data that I process sequentially with console utilities (written by me). Accordingly, they take input data in the form of files and also issue files that are processed further. The task is resource intensive and takes time.
How to correctly monitor the progress of the work and support the launch from the previous successfully completed stage, in case of incorrect completion of the current one?
Now I just look for the presence of files in the output. If there is, then this stage is completed successfully and you can skip it. But maybe there is some well-established technique for such tasks? Linux system (Ubuntu).

Answer 1

a simple version with the addition of the status function:

#!/usr/bin/bash
set -o history

status(){
  STATUS=$?; COMMAND=$(history 2 |head -n1 |awk '{$1="";print}')
  if 
    then  echo -e "[   OK  ]\t$COMMAND"
    else  echo -e "[ ERROR ]\t$COMMAND"
            exit # при ошибке останавливаем обработку скрипта
  fi
}

touch a1.txt
status

rm a1.txt
status

rm a1.txt
status

we get the following output to the console:

[   OK  ]	 touch a1.txt
[   OK  ]	 rm a1.txt
rm: невозможно удалить 'a1.txt': Нет такого файла или каталога
[ ERROR ]	 rm a1.txt

if desired, all this can be logged to a file

Answer 2

How to correctly monitor the progress of the work and support the launch from the previous successfully completed stage, in case of incorrect completion of the current one?
You should have a completion with the code right in the scripts. If there were no errors, you need to end the script through exit 0. If an error occurs, you need to end the script through exit 1.
When your scripts are done this way, you can check their exit code.
And then you can write a script with the lines

{ script1.sh && echo "script1 - ok"; } || echo "script1 - fail"
{ script2.sh && echo "script2 - ok"; } || echo "script2 - fail"
{ script3.sh && echo "script3 - ok"; } || echo "script3 - fail"

Example

{ echo x1 && echo "script1 - ok"; } || echo "script1 - fail"
{ cat x && echo "script2 - ok"; } || echo "script2 - fail"
{ echo x2 && echo "script3 - ok"; } || echo "script3 - fail"
{ touch /x && echo "script4 - ok"; } || echo "script4 - fail"

[[email protected] ~]$ { echo x1 && echo "script1 - ok"; } || echo "script1 - fail"
x1
script1 - ok
[[email protected] ~]$ { cat x && echo "script2 - ok"; } || echo "script2 - fail"
cat: x: Нет такого файла или каталога
script2 - fail
[[email protected] ~]$ { echo x2 && echo "script3 - ok"; } || echo "script3 - fail"
x2
script3 - ok
[[email protected] ~]$ { touch /x && echo "script4 - ok"; } || echo "script4 - fail"
touch: невозможно выполнить touch для «/x»: Отказано в доступе
script4 - fail
[[email protected] ~]$

Can be done like this

{ script1.sh && echo "script1 - ok"; } || { echo "script1 - fail"; exit 1; }
{ script2.sh && echo "script2 - ok"; } || { echo "script2 - fail"; exit 1; }

Then, in case of success, it will print a successful phrase and continue, and in case of an error, it will print a failed phrase and exit immediately (with an error code).