How to implement repetitive tasks in php (cron)?

Question

PHP

Hello! Please point me in the right direction.
There are 2 situations that need to be solved using php.
1. Playback function at a well-defined time. An illustrative example - the user creates a reminder for a specific date and time (for example, today, 22:35) and exactly at this time the script must work out (in this case, for example, send a notification to the mail).
How to implement it most competently? Set up a cron script with an interval of 1 minute that will check for any jobs for the current minute? Will this be the most correct solution? What if the execution of a function takes time (for example, we are talking about parsing), and at a certain time you need to perform several tasks at the same time? For example, it is critical to parse data from three sources minute by minute.
I'm talking about the fact that if you implement this in order of priority, then the next iteration of the function will occur only after the end of the previous one. Those. until the script parses the first data, it will not proceed to parsing the second.
Read about multithreading in curl, i.e. there is a solution in case of parsing, but what about in other situations?
2. Implementation of the function repetition. Example - the user starts a task, within the framework of which the same function must be executed on the server, for example, 100 times with an interval of 40 seconds. How to implement it most competently?
The main problem here is the script execution time. I know about set_time_limit / max_execution_time, but is this the only solution? And in general, how correct is it, for example, to disable the maximum script execution time if we do not know how long it can take?
The questions are rather not about specific examples, i.e. the function can be any - sending mail, accessing api, parsing data, and so on. What is interesting is how to implement it correctly. Tell me where to dig, what to read.
Thank you.

Answer 1

There are cron implementations (not crontab itself, but on the same bash) and once a second, you can google options.
You can use a queue server (eg Gearman). The server performs the task and has all the data about it. The task can be reassigned to itself automatically after execution, along the way changing certain data in it.
Parsing 3 tasks is also easy. 3 workers are made for the same server, which work in parallel. As soon as the Girman receives a task, the free worker immediately grabs it. 3 tasks will arrive - 3 workers will grab them and will perform them simultaneously. Moreover, tasks can be prioritized. Optional start with low, mandatory with high priority. A free worker will decide which task to take on himself.
In general, intermediate data when starting a script (timeouts, etc.) can be stored by any nosql solution. At least write to a file, at least in redis, at least in memcache, if the access to the database is not critical. Task completion statuses, task types, tasks that are being executed - all in the same place. Well, or again, in the database. It's more convenient here.
Previously, for example, we implemented page parsing through redis. Cron ran once every 10 minutes (it took a long time to parse), and the value of the current page was put in the radish. The next time the script was run, the value was retrieved and the next page was paged on it when requesting the API. Then this key was banged (I don’t remember anymore, either manually or by expire) and everything started anew.

Answer 2

I would make a small separate daemon/script/whatever that starts from under the crown with some frequency, checks the lock (flock, to the extreme), climbs into the database, understands what tasks it needs to complete now, launches child- processes for their processing (here you can limit appetites, if necessary), notes in the database that the processes are running, releases the lock and dies. It if an error in one minute - does not confuse.
If it confuses - a separate daemon that listens, for example, to OS signals, and upon arrival of a signal from a php application or its own alarm, does all this from the first paragraph, only first turning off signal processing;). With startack, the daemon writes its pid somewhere so that your application knows who to signal. ;)