S
S
Sergey Sokolov2012-06-09 13:12:48
Exim
Sergey Sokolov, 2012-06-09 13:12:48

How to send a million emails in one hour?

I am planning a solution for clients with the need to send out about a million emails once a day. Clients are not spammers. Coupons sending out to subscribers hungry for discounts. "Unsubscribe" link in every email. Letters without attachments, up to 100k in weight.
There is a dedicated server on Hetzner, there is Amazon SES.
Question: is it worth relying entirely on the strength of your server, on which there is also a database and a web, and send through your own Exim4,
or drop all letters on Amazon, and let it steam further, delivering them?
Priorities:
1. deliverability;
2. the speed of mailing, so that this million does not stretch for half a day;
3. expenses. Amazon will fly into a dollar if you send a million every day.
Additional whys:
How does the Exim server load when sending mail?
How parallelized is it, or does it send strictly one letter at a time?
Does anyone have experience mailing through both Exim and Amazon SES to compare?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
Y
Yaroslav, 2012-06-09
@sergiks

One hundred thousand million years ago , a system for mass mailing was written for a customer. Now I don’t remember exactly, but it seems that they sent out 3000 simultaneous connections. I won’t lie about the speed - I don’t remember, but less than a million per hour. Maybe now (computers are much more powerful than steel, channels are wider) and it is possible, but I'm not sure.
Almost 100% sure that exim will NOT work.
There will be a lot of problems, starting with the fact that it is not a fact that the OS will thrash around when a million files appear on the disk in the mail spool.
The second problem is that no matter how clean the maillist is, with such volumes there will be dead emails, already dead domains, or domains that will have name servers, and then every attempt of a regular MTA to connect to the mail server will be delayed for a very long time (at the same time, this MTA connection does nothing, tries to send this letter, cannot, but spends a lot of time, for which one could send a hundred letters).
Another problem is that with such a powerful mailing, the channel will be clogged (of course, if the mailer manages to download it). That is, there will be packet loss, loss of DNS requests and responses, and so on. (in the meantime - Redhat's bind from old distributions - generally fell under such a load)
In any case, it seems to me that with a regular MTA (without a very hard file refinement) it is unlikely to succeed, especially in an hour. They are still designed for "expensive but reliable" delivery - that is, a letter can be in the queue for at least a week if the recipient's server is down. If the base is really clean and legal, there will be relatively few such problematic letters, but each of them greatly slows down the system. And you need “cheap and fast” delivery, when the problematic letter is more likely to be thrown out (and to hell with it), but will not devour resources for attempts to deliver. And besides, do not forget about the now fashionable greylisting, which is almost everywhere. So each letter may well stand in line for at least 15 minutes.

E
eaa, 2012-06-09
@eaa

It feels like it won't work in principle on existing non-specialized clients.
They try their best to provide delivery, but not speed. Accordingly, the letter will lie for a day or two if it cannot be delivered immediately. If there are many of them, then in a week with such volumes of garbage it will become just a bulk. And even if your service copes, there is no guarantee that those smart hosts and other external mail servers will pull the load, and we cannot influence their configuration. For example, if there are many letters for one, let's say yandex.ru - how will he react to such a flow? Here you need a specific agent that will take all this into account ...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question