How to make a queue in PHP for working with a CRM system?

Question

PHP

Hello!
In one of the popular CRM systems, there is a public application that is used by 400+ accounts. When changing entities in CRM (deal, contact, etc.), webhooks come to our server, during the processing of which we need to query the client's CRM account via the API. The API limit is 7 requests per second. If this limit is exceeded, the account is blocked from receiving data via the API. Before blocking CRM, the system, of course, sends in the response header that the request limit is exceeded, but this answer does not help, since, firstly, the request has already been made, and secondly, requests from the CRM system are still processed in parallel.

Problem: if a client imports 10,000 contacts into CRM at once, then CRM will send ~ 10-20 requests per second to our server about creating a contact. Since we need to process each request, our server also makes ~ 10-20 requests per second via API already in CRM. Since the request limit is 7 pcs, our application is blocked.

Question: How can I solve the problem? The obvious solution suggests itself - queues, but I have little knowledge of them. As far as I know, there can be a fixed number of workers, for example 10. What if 30 accounts of the CRM system decide to import 10,000 contacts at the same time? And there are only 10 workers. Will the other 20 be waiting?

And is there a dynamic creation of workers in ready-made solutions for working with queues? For example, when there is a process that scans queues. If there are messages in the queues, launches a worker on this queue. Once the worker has completed the queue, the worker process is killed. A new worker will be launched if there are new messages in the queue.

Answer 1

Yii2 has Yii-quque