Am I planning to deploy a network of servers correctly?

Question

PHP

Hello.
There is a PHP script that parses data from 9 different sites. Next, it enters the received data into the SQLite database. And then the user opens the site and looks at the data from the database.
It is possible to split the script into parts, say into 3 scripts, so that it parses 3 sites each. The average execution of a script on one server when it parses 9 sites at once is ~4.1 seconds. For me it's a long time. For reference: data that is shared from sites no more than 1mb. In total, 9mb maximum.
So.
There are 5 servers running Windows Server. The names are conditional.
s1
s2
s3
main
database

They are connected to each other in a local network and each has a public IP.
My idea is the following:
Break the parsing script into parts. That is, s1 will receive data from three sites, s2 from three sites and s3, respectively, from three sites.
They will enter the received data into the SQLite database on the database server.
And the main server, where the site is actually hosted, is already receiving data from the database server and issuing it to the user.

Now a question.
What is your opinion on this idea? Rave?
Maybe it makes no sense to bother because of 9mb, and use some kind of streaming download on one server? If yes, what would you recommend?

Answer 1

Most likely (without seeing the code it's hard to say for sure) the root of your problem is that the data from the sites is obtained sequentially. In this case, it will be enough for you to execute requests in parallel using https://docs.guzzlephp.org/en/stable/quickstart.ht... or https://www.php.net/manual/ru/function.curl- multi-...

Answer 2

It is obvious that the use of SQLite is not intended, this DBMS is not designed for concurrent and network access.

Answer 3

It is not entirely clear why 4.1 seconds is a long time for you, what are the general requirements? As an option, I propose to consider the following. configuration:
Split the script into 2: The first one will make a request to your sites, receive a response and save it in special. table. The second one will take new data from this table, run the desired handler, parse the saved response and update the data you need in the database. In general, classic queues (google for php queue, php jobs)
Advantages:
1. Separate receiving and processing data
2. Each part of the script can be modified separately (for example, receiving through parallel requests)
3. You can do as many queue handlers as you need
4. With an increase in the number of sites, the execution time of one request will not change.