R
R
Roman Sarvarov2021-08-26 14:15:49
Laravel
Roman Sarvarov, 2021-08-26 14:15:49

Does it make sense to use a queue in Cron jobs?

I have a cron job that parses ~50 pages of some site every hour.
There are 100 product blocks on each page (let's say), and by identifier, data is created / updated for each product in the database.

Is it worth doing some Job classes for this? Or does this make no sense in the crown - there is no need for such asynchrony?
The only thing I still need is to limit the number of requests, i.e. for example no more than 1 per second.
At the moment, in the crown I do simply . If you do Job classes, do you think you need to do tasks only for a request to the site (~50 tasks) or, in addition, one more Job for searching and creating / updating a specific product (~500 tasks)? sleep(1)

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
Ilya, 2021-09-02
@megakor

I think it makes no sense to use jobs. Do everything within the console cron task, add sleep where necessary. Since the task is running for a long time, it is better to pull runInBackground from it so that it does not stop other tasks (if there are any, of course).

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question