Answer the question
In order to leave comments, you need to log in
How to set celery to different queues for different Django projects on the same server?
So there is VDS.
It has two different Djangos with their own virtual environment. In different places.
Each has its own Celery.
RabbitMQ is also installed in the system (so to speak, common to all) and the whole thing is launched by SuperVisor.
Each Django has its own different tasks (task), but one of them in each Django has the same functionality. Namely, the creation of a PDF file. Naturally, when setting up a task, each of these "pdf-tasks" has its own way to save the file.
So all the tasks are lined up in one queue and the following is obtained. I request to do PDF for one project (well, invoice) all the time. And once the pdf-file appears in the folder of the desired project, and the second time in the folder of another. And so in succession.
one-celery.conf (for supervisor)
[program:one-celery]
command=/home/iuser/djangoprojects/one/v_one/bin/celery worker -A one --loglevel=INFO
directory=/home/iuser/djangoprojects/one/
user=iuser
numprocs=1
stdout_logfile=/home/iuser/djangoprojects/logs/celery.log
stderr_logfile=/home/iuser/djangoprojects/logs/celery-err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
stopasgroup=true
priority=1000
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'one.settings.prod')
app = Celery('one')
app.config_from_object('django.conf:settings', namespace='CELERY_ONE')
app.autodiscover_tasks()
CELERY_BROKER_URL = 'amqp://localhost'
Answer the question
In order to leave comments, you need to log in
So, having tried a bunch of options, I got only one. It’s hard for me to say why it is, but maybe it has something to do with the versions of Celery itself, because all the solutions older than 2018 just didn’t work for me. In the end, I combined a couple of solutions (found on stackoverflow) that were proposed in 2018 and 2019 and finally I got it.
So the settings.py file. In general, he was not involved in any way. In it (if you look above) there was only a broker setting. It migrated to the celery.py file
celery.py now looks like this:
from kombu import Queue, Exchange, binding
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'one.settings')
# default_exchange = Exchange('default', type='direct')
one_exchange = Exchange('one_go', type='direct')
app = Celery('one', broker='amqp://localhost')
app.conf.update(
task_queues= (
Queue('one_go', [binding(one_exchange, routing_key='one_go')]),
)
)
app.conf.update(
task_routes= {
'plugin.email.tasks.send_email': {'queue': 'one_go'},
'plugin.print.tasks.create_pdf_task': {'queue': 'one_go'},
'plugin.print.tasks.create_pdf_from_site_task': {'queue': 'one_go'},
}
)
app.autodiscover_tasks()
[program:one-celery]
command=/home/iuser/djangoprojects/one/v_one/bin/celery worker -A one -l info -Q one_go
RabbitMQ has the ability to set up virtual hosts.
https://www.rabbitmq.com/vhosts.html
Accordingly, in your projects you raise a connection to one rabbitmq instance, but to different virtual hosts.
# settings.py
CELERY_BROKER_URL = 'amqp://user:[email protected]:5672/my_first_host'
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question