How to organize many parallel requests to different servers/APIs?

Question

PHP

Dear colleagues.
There is a website - an online store with the ability to search for goods in the catalogs of suppliers and order through our website. Directories, of course, dofiga and more, each with its own API, which returns a response in its own format.
What I would like to receive : Search is similar to skyscanner, when one request receives several answers from different providers. Those. something like a specialized search engine that looks for a specific product and its current availability.
At the moment it works like this:
1. A request is made for the backend
2. Curl makes a request to one of the providers in turn in the list
3. Data from all providers is sorted, filtered and displayed in the required form.
Problems:
1. The overall performance is not very good
2. If one API starts to lag, then it only remains to wait for the timeout and only then move on to the next one.
How to organize many parallel requests to different servers/APIs? Caching, in this case, does not help, because. you need to check the availability of rare goods from a specific supplier
. 2 ideas come to mind:
1. Organize n number of requests from the front to my back with the supplier indicated, respectively, n * number of search visitors requests will be launched on the back. In addition to a certain crutch of this decision, the question arises, will it not gobble up all the memory?
2. Somehow rewrite the code with phtreads so that it all runs in parallel, but the question arises, how to make sure that the results are returned to the front / updated with a response from each source?
What else can be offered?

Answer 1

On a small scale, you can use parallel curl requests via curl_multi_init() - their execution will take as long as the slowest of the requests.
On a larger scale, you need to raise the queue of tasks, parallel machines / processes "workers".

Answer 2

RabbitMQ on the back and long-polling/WebSocket on the client.
In short, the scheme is as follows: the client sends a search request to the backend (to a certain task manager), subscribes to receive results (WebSockets are used here) and actually waits for them. The dispatcher scatters tasks in a queue (RabbitMQ) and dies. Handlers (can be scaled horizontally to n-machines) make requests to third-party APIs (i.e. useful work), get the result and return it to the client.
Caching and other goodies are added to taste.

Answer 3

https://www.youtube.com/watch?v=uO268voCGwA - the situation is similar, although they had to send several requests to the same API server.