G
G
Glory2022-02-14 18:13:45
Laravel
Glory, 2022-02-14 18:13:45

Is there a separate worker for each queue, or are there several queues in one worker?

I am using Laravel. And therefore, to start the queues, I use the command

php artisan  queue:work rabbitmq --queue=jobs, emails,  imports --tries=3


and of course I use supervisor .

And here's the question. In what cases to start several workers in the supervisor for one project and specify its turn in each worker. For example, emails

Or in which case to start one worker per project and just list the queues after queue=

separated by commas. Do you need to start separate workers if you want to start queues with different priorities and number of processes?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
P
pLavrenov, 2022-02-14
@pLavrenov

One for everything. Via Horizon. There you can balance processes in queues.
horizon

V
vism, 2022-02-15
@vism

Separately, it is necessary when independence and reliability are needed.
For example, queue can freeze, hang the entire worker and all other queues in the worker.
Or do you need separate settings for some queue.
I usually create one worker for high, medium, low
AND separate workers for important or heavy queue
example

'defaults' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['high','medium', 'low'],
            'balance' => 'simple',
            'minProcesses' => 1,
            'maxProcesses' => 5,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            'tries' => 3,
            'timeout' => 600,
        ],
        'supervisor-2' => [
            'connection' => 'redis',
            'queue' => [
                \App\Services\QueueService::QUEUE_NAME__MEDIA_LIBRARY,
            ],
            'balance' => 'simple',
            'minProcesses' => 1,
            'maxProcesses' => 2,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            'tries' => 3,
            'timeout' => 60,
        ],
        'supervisor-3' => [
            'connection' => 'redis',
            'queue' => [
                \App\Services\QueueService::QUEUE_NAME__IMPORT_WEBHOOK,
            ],
            'balance' => 'simple',
            'minProcesses' => 1,
            'maxProcesses' => 2,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            'tries' => 3,
            'timeout' => 1200,
        ],
    ],
    'environments' => [
        'production' => [
            'supervisor-1' => [
                'minProcesses' => 1,
                'maxProcesses' => 5,
            ],
            'supervisor-2' => [
                'minProcesses' => 1,
                'maxProcesses' => 2,
            ],
            'supervisor-3' => [
                'minProcesses' => 1,
                'maxProcesses' => 2,
            ],
        ],

        'staging' => [
            'supervisor-1' => [
                'minProcesses' => 1,
                'maxProcesses' => 2,
            ],
            'supervisor-2' => [
                'minProcesses' => 1,
                'maxProcesses' => 2,
            ],
            'supervisor-3' => [
                'minProcesses' => 1,
                'maxProcesses' => 2,
            ],
        ],

        'local' => [
            'supervisor-1' => [
                'minProcesses' => 1,
                'maxProcesses' => 4,
            ],
            'supervisor-2' => [
                'minProcesses' => 1,
                'maxProcesses' => 2,
            ],
            'supervisor-3' => [
                'minProcesses' => 1,
                'maxProcesses' => 2,
            ],
        ],
    ],

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question