How to reserve and balance balancers?

Question

Nginx

hypothetical situation. There is a frontend to which the domain points. It only has nginx, which proxies requests to internal worker servers.
Here a problem arises - there are a lot of workers and their number continues to grow. Still far from resting on the performance of nginxa, but one or two more large partners - and one machine will not digest everything. How to balance the load in front of the balancers?
More. Fortunately, the hoster is reliable and the balancer has never fallen. But how to reserve it? After all, changing the A-record is not a quick matter, and some caches may not be updated at all for weeks?

Answer 1

Regarding the primary balancing:
I personally think that on large flows, the load on the front can be balanced with a simple round robin.
We use amazon route53 + their own health check to rule out downed servers.
Regarding reliability, if there are few fronts:
hetzner has such a thing - failover ip wiki.hetzner.de/index.php/Failover/ru (other hosters also have it, it can be called differently)
This is an ip that can be very quickly switched between physical servers.
You lift the 2nd server, in case of falling of the main - ip automatically switches to the 2nd server.

Answer 2

Hetzner and OVH have failover-ip addresses.
They switch between machines in 10 seconds.
Well, then - heartbeat and let's go.