S
S
Swasher2014-08-26 17:31:20
Django
Swasher, 2014-08-26 17:31:20

Incron within Django: Pynotify + Celery or something else?

There is a script running in production, written in python, that processes files. How it works: a file falls into the hot folder, incron triggers an event and starts the script. The file is processed and the results are written directly to the html file. Files can be large (half a hectare or more) and fall several at a time, but during the day the load is not high, 1-2 dozen files.
I want to translate this case to Django.
While the idea is this - there is a PyNotify library ( https://github.com/seb-m/pyinotify), it can create an event and queue a file. The queue is processed in Celery, the convenience of parallel processing is obvious if several files have fallen at once.
Or you can do without PyNotify, using incron as before to run a python file that will queue a new file.
Have I chosen the right way to solve the problem, or is there some more elegant / lightweight / simple solution? Will not Celery in this case from a cannon on sparrows?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
Swasher, 2014-09-10
@Swasher

I answer myself - the choice fell on Python-RQ and, accordingly, django-rq as a lightweight replacement for Celery. PyNotify did not use.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question