What is the best way to implement an infinite job queue?

Question

PHP

The bottom line is this - I need to check certain pages on different sites. Continuously and sequentially. When the queue reaches the end, it starts again from the first task. It should be possible to add new tasks, swap tasks and delete them. Accordingly, the question is, how best to implement it? I know PHP and ActionScript (more or less close to Java), but as far as I understand, they are not suitable for such purposes. The tasks themselves are quite simple, I think that it will not be a big problem to implement them in any language, so you need to choose the most suitable one. Should work on Debian. Please advise any specific bundles that you consider the most suitable for such a task.

Answer 1

you can use Node.JS and check not only sequentially, but also in parallel, that is, asynchronously :) If you know PHP, then I can assume that you are familiar with Javascript :)
https://github.com/request/request - here it is you can make a request to the page
https://github.com/cheeriojs/cheerio - this is how to parse it

Answer 2

"Bundle": crontab, php
And a database with tasks, then you yourself will figure out how it should be.

Answer 3

Start +1 php process in infinite loop with include. Further, at any time, edit the file-include. Include multi-threaded curl in it. (see documentation), etc.

Answer 4

e.g.
$currentTask = 0;
$allTasks = 1000;
$currentTask++;
if($currentTask == $allTasks) {
$currentTask = 0;
}