S
S
Sergey Alekseev2019-02-13 11:31:34
Django
Sergey Alekseev, 2019-02-13 11:31:34

Celery, how to fix the error: Error daemonic processes are not allowed to have children?

Good afternoon, I understand the mistake that other processes cannot be created from the daemon process, but where this process is created is also not clear, in the seler? How to avoid it?

Таска
@app.task
def periodic_download_ostrovok():
#         Сlass.download()
Вызываемый метод
Ругается на multiprocessing.Manager()
def download():
    with multiprocessing.Manager() as manager:
          for i in range(n_proc):
                    p = multiprocessing.Process(
                        target=cls._add_hotel,
                        args=(n_proc,
                              i,
                              filename,
                              db_name,
                              regions,
                              result,
                              cut))
                    process_pool.append(p)
                for p in process_pool:
                    p.start()
                for p in process_pool:
                    p.join()
                if result.get('error'):
                    raise Exception(result.get('error'))

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dmitry Shitskov, 2019-02-13
@Zarom

Your task is the very "daemonic process" launched by Celery.
Multithreading must be implemented by Celery - this is a built-in functionality, you should refer to the documentation for examples of parallelizing the execution of jobs.
Superficially looking through the dock, I understand that if a certain list comes to the input of a job, Celery can cut it into chunks and start processing them asynchronously.
PS You have already been answered where these Celery processes are created, how does it work?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question