L
L
LayneBuchyn2013-07-27 17:58:21
Python
LayneBuchyn, 2013-07-27 17:58:21

Load balancing algorithm between SMTP servers

At the moment, I am solving the problem of creating a system of mass mailings of mail - not spam, letters will be received by registered users who have subscribed to a mailing list for certain events. The solution must meet the following requirements:
1. Estimated load - about 500 thousand letters per day.
2. You need to be able to add new SMTP servers on the go, and you should see an increase in mailing performance.

In short, the solution I chose is this:
Two daemons work (written in Python3):
1. The first daemon retrieves a list of recipient addresses and a letter template from the database (MongoDB), and forms personal letters (“Dear ...”) according to the template. He adds the generated letters to the queue organized on Redis.
2. The second daemon listens to the Redis queue, takes letters from it, and gives them to SMTP servers.
It is possible to finish both demons - so that they work in several processes.
I have 2 such questions:
1. I would like to know in advance which link in this system will be the weakest: I assume that the bandwidth of the SMTP server - you need to send letters to SMTP servers no faster than they can send them out.
2. How to properly distribute the load between SMTP servers so that adding new servers brings a real performance boost.

I would be grateful for any advice from a competent person in this matter.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
M
moonsly, 2013-07-27
@LayneBuchyn

1. In addition to the SMTP bandwidth, you need to separately control the limits for each service (mail.ru, rambler.ru, yandex.ru etc) - if you send too often to one service (several letters per minute), you will quickly get into filters and blacklists, and no one will see if your site/mailing list is white or not. You can google specific numbers with letter limits for each service, I don’t know them, but they exist.
2. Ordinary round-robin, i.e. each new added SMTP participates in the ring queue for sending the next letter.
If you control the delivery/reading, then you can separately keep its rating for each SMTP (for the delivery of letters), and take this rating into account when choosing a server (the higher the delivery, the more you can send for now, but don’t overdo it).

P
Puma Thailand, 2013-07-28
@opium

round robin i think will suit everyone

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question