Task Queue

Question

linux

Hello.

There was a need for software for linux that implements a task queue. Is there anything not too heavy?

Here's how I imagine it. Several processes add tasks to the queue. These tasks are performed in the order they were added. Until the previous task is completed, the next one is not started.

If it suddenly turns out that this type of software is in bulk, tell me something that is in the CentOS packages.

Thanks in advance.

Answer 1

A Google search for "linux simple jobs queue" finds the Task Spooler:
www.ubuntugeek.com/task-spooler-personal-job-scheduler.html

-S [num] set the number of max simultanious jobs of the server.

Answer 2

habrahabr.ru/company/biggo/blog/102742/

Answer 3

www.celeryproject.org/
www.gearman.org/

Answer 4

There is a very simple task queue for python - python-rq.org/
And with python you can run console commands, programs, scripts, etc.
If you want even easier - write your own on the crown, files and bash.

Answer 5

How heavy is it? Jenkins?

Answer 6

Bicycle:
Preparation:

groupadd jobadd
mkdir -m770 /var/lib/jobqueue
chown root:jobadd /var/lib/jobqueue

/usr/local/bin/jobadd:

#!/bin/sh
f=`/var/lib/jobqueue/date +%Y.%m.%d-%H%M%S-$$`
echo "[email protected]" > "$f"

/usr/local/sbin/jobexec:

#!/bin/sh
cd /var/lib/jobqueue
while : ; do
    for f in *-*-*; do
        test "$f" = "*-*-*" && break
        logger "jobexec: $f `cat $f`"
        sh "$f"
        rm "$f"
    done
    sleep 30
done