Why do you need celery?

Question

Python

So, if we take dry information from the manuals, we get the definition:

Celery is nothing more than a distributed job queue implemented in Python.

So, what I understood here is that this is a queue.
But the queue is a very simple data structure, why celery for it, why not create it yourself?
Why is it only for assignments? Where does this specialization come from?
What real problems did celery solve in your practice and why exactly? (It is interesting that you have a real application)
And excuse the newbie if the question is completely stupid, but I wanted to sort out this issue.
------------
Thank you!

Answer 1

For example, a video conversion job queue for a Django web application.
, while the first video file will be converted for 5 minutes, after that it will be marked in the database as is_converted (converted) and the command will end there. After a minute, cron will run the command again to convert the next file.
But in 5 minutes the file will not be marked as converted yet, so cron will create 4-5 processes in 5 minutes that will convert the same file.
Okay, then at the start of the conversion, we will immediately mark the file as convertible so that other processes take other files.
But what happens if the process crashes? Mark the start time of the conversion in the database and if more than n hours have passed, convert again - so-so solution.
And what if users upload 10 files, each of which takes 30 minutes to convert? In 10-11 minutes, 10 converting video processes will be running on the server. Heavy load on the server. You can create a table in the database that notes how many files are being converted at the moment.
When we decide to use cron for long running tasks, we have to deal with a number of problems.

Answer 2

Frameworks are worth using due to the fact that they create a unified environment that everyone who reads the documentation can understand. You don't write documentation for your decision. Frameworks provide basic and boring code that no one likes to write.