D
D
domanskiy2021-12-07 11:39:33
Python
domanskiy, 2021-12-07 11:39:33

Celery does not see the module. How to connect celery to FastApi?

There is a project on FastAPI I'm trying to
fasten celery. The problem is that image does not start on docker.

Error: 
Unable to load celery application.
The module celery_worker was not found.


Project structure
docker-compose.yaml
Dockerfile
.env
requirements.txt
Located at the root.

\api\celery_worker.py
spoiler
"""celery module"""

import os

import time

#celery
from celery import Celery

celery = Celery(__name__)
celery.conf.broker_url = os.environ.get("CELERY_BROKER_URL", "redis://localhost:6379")
celery.conf.result_backend = os.environ.get("CELERY_RESULT_BACKEND", "redis://localhost:6379")


@celery.task(name="create_task")
def create_task1(arg1, arg2):
    time.sleep(int(arg1+arg2) * 2)
    return True


docker-compose.yaml
spoiler

version: "3.9"
services:
  db:
    image: registry.all-code.com:5000/custom-postgres
    build: ./custom-postgres
    restart: unless-stopped
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./custom-postgres/custom_postgresql.conf:/etc/postgresql.conf
    command: postgres -c config_file=/etc/postgresql.conf
    environment:
      - POSTGRES_DB=b2market_marketplace_development
      - POSTGRES_USER=b2market
      - POSTGRES_PASSWORD=postgres

  api_py:
    build: .
    restart: unless-stopped
    ports:
      - "5000:5000"
    depends_on:
      - db
    command: bash -c "cd b2market_marketplace && alembic upgrade head && uvicorn api.main:app --host 0.0.0.0 --port=5000 --reload"
    volumes:
      - /b2market_marketplace
      - ./alembic:/b2market_marketplace/alembic
      - ./api:/b2market_marketplace/api
    env_file:
      - .env

  worker:
    container_name: worker
    build: .
    restart: always
    command: celery -A celery_worker.celery worker --loglevel=INFO
    volumes:
      - /b2market_marketplace
      - ./api:/b2market_marketplace/api
    environment:
      - CELERY_BROKER_URL=redis://redis:6379/0
      - CELERY_RESULT_BACKEND=redis://redis:6379/0
    depends_on:
      - api_py
      - redis

  redis:
    container_name: redis
    image: redis:6.2-alpine



I don’t understand why it doesn’t find the celery_worker module stubbornly?
I tried to specify api.celery_worker

In requirements.txt
celery==4.4.7
flower==0.9.7
redis==3.5.3

I tried instead of celery = Celery(__name__) to write celery = Celery("worker")
Does not work
Who knows in what's the problem? Help me please!

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
domanskiy, 2021-12-08
@domanskiy

As a result, a working solution:
61b0510541964306152675.png
It's not entirely clear to me how celery finds this particular task.py
But everything works.
!!!Attention
celery==4.4.7
flower==0.9.7
For the 5th version of celery, the launch code is different. See documentation

R
Rodion, 2021-12-07
@rodion4dev

Incorrect path to Celery application in docker-compose.yaml in worker.command. Try celery_worker:celery if the api folder is just a folder (not a python package); otherwise - api.celery_worker:celery

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question