M
M
m0ody2014-08-16 15:34:06
Python
m0ody, 2014-08-16 15:34:06

Celery: how to make chord execute callback when Exception occurs in tasks?

@shared_task(time_limit=10)
def task_err():
    ...
    raise Exception()

@shared_task
def task_success():
    ...

@shared_task
def callback(result):
     ...

chord([task_success.s(), task_err.s()], callback.s()).apply_async()

We get an error:
Chord callback for 'dad01f6b-bfdd-48a1-aacb-c4d713de0988' raised: ChordError('Dependency e7309f65-a741-4027-8a52-7d5e230f98c5 raised <ExceptionInfo: Exception()>',)
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\celery\backends\redis.py", line 221, in _new_chord_return
    callback.delay([unpack(tup, decode) for tup in resl])
  File "C:\Python27\lib\site-packages\celery\backends\redis.py", line 184, in _unpack_chord_result
    raise ChordError('Dependency {0} raised {1!r}'.format(tid, retval))
ChordError: Dependency e7309f65-a741-4027-8a52-7d5e230f98c5 raised <ExceptionInfo: Exception()>

How to overcome this? Of course, you can process errors in tasks and return the error code or the exception itself, but what if there is a time_limit? Such a task cannot be processed.

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question