Celery - how to complete nested tasks?

Question

Django

There are tasks (task) of this type - for a given set of sentences, I get a list of links. Then I get the content of the pages on these links and analyze them (the analysis itself can take a long time, from 1 second to 5 minutes). Of course, in the process of analysis, exceptions arise that need to be caught and interrupt the execution of the entire task as a whole. It follows that you need to cancel the execution of other tasks that are already running.
Now the following code works

@celery_app.task(
    soft_time_limit=180,   
    ignore_result=False
    )
def search_job(...)
    try:
        markup, error = get_content(url)

        # код анализа страницы здесь
    except SoftTimeLimitExceeded:
  
        log += url + " - Soft Time Limit Exceeded Error Catch!\n"
    result['log'] = log
    return result

@celery_app.task(
    queue="ta",
    soft_time_limit=600,
    base=BaseCeleryTask
    )
def search(...):

    try:
       # сдесь получение списка list_urls
        group_work = group(search_job.s(..., url) for url in list_urls)()

        while not group_work.ready():
            time.sleep(1)

    except SoftTimeLimitExceeded:
        # Если это задание ждет на результат выполнения всех заданий в группе более 600 секунд 
        # я хочу отменить все "мелкие" задания и считать что задание в целом завершилось с ошибкою    
        group_work.revoke()
        log += "Big task time limit exceeded!"

   # Если все норм - получаем результаты. Оказывается, так делать нельзя(((  
    with allow_join_result:
         results = group_work.get()
    # Далее их сохраняем
    save_results(results)

So, in practice, small tasks freeze, SoftTimeLimitExceeded is not caught. And they just hang and eat memory. This rarely happens, but still.
What could be the best solution for this task?

Answer 1

In general, this article helped to solve my problem.
https://www.azavea.com/blog/2016/10/20/how-to-buil...
Do you think it would be useful to translate it and post it here?

Answer 2

faster - kill
stackoverflow.com/questions/8920643/cancel-an-alre...
stackoverflow.com/questions/29306337/how-to-stop-c...
safer - write code with more hooks - I think you have everything depends on the work of the network