What is the best way to break the script?

Question

PHP

Hello,
There is a php script that runs phantomjs through the console, you need to parse pages every few minutes. ~500 pages, 2 seconds per page, what's the best way to split the script?
There is an idea:
1) Divide into several batches and run sequentially, the same time
2) Divide into batches, but run several tasks at once.
3) Use some handler, put the Url in the database/memory/file, and process 1 by one

Answer 1

Without specifics about everything, you can only guess what you want. Do you want to do one task and do it, or do several, depends on the implementation of the script, the capabilities of the scheduler and your capabilities.

Answer 2

Run 20 handlers through the supervisor (it won't be enough - add more), for each, allocate your own list of urls, that's all. P - Parallelization

Answer 3

Is a queue server suitable? https://yandex.ru/search/?text=server%20queues%2...