W
W
Webber2020-05-27 18:00:48
Laravel
Webber, 2020-05-27 18:00:48

How to send several million messages in parallel?

Project on laravel, postgres, redis queue.

The database contains 10 million rows. You need to filter them by conditions and send them messages.
Current implementation:

function handle() {
Users::query()->chunk(100, function($users) {
  $filterUsers = $this->filter($users);
  
  foreach ($users as $user) {
     $this->sendToUser($user, 'message');
     // помечаем в бд что отправили
  }
});
}

function filter($users) {
 // опращаемся на врешний апи за инфой по юзерам
 // фильтруем их по условиям из нашей бд
 return $filterUsers;
}

Roughly such logic now.

Processing one worker is all - which is very long. How to make, for example, 50 workers work simultaneously? How will they take records from the database and not conflict?

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question