How to minimize points of failure?

Question

System administration

Hello.
Now I'm going through the "course of a young fighter", or rather, I'm studying to be an accessibility engineer.
I am just getting to know this world, so I can ask such, probably simple questions.
But still, tell me, please, how to minimize the points of failure?
Let's take the example of mail. How does mail work for them even if the DC is down?
We were given the following userkeys: 3 servers, we need to configure the service so that the uptime is close to 100%
A thought flashed through my head: 1 - balancer, 2.3 - web, and so on.
But in this case, just the balancer is the point of failure.
Therefore, the question arose - how do "big" companies "fight" this? or how to make sure that this balancer does not become a point of failure?
Thanks

Answer 1

Two balancers with the same addresses (a pair in a cluster, four in total), each of which is located in different regions of the country, each balancer is connected with two legs to different routers via EBGP. Backs of balancers in constant synchronization (heartbeat). We add a reserve for virtualization (vmotion + fault tolerance), for power supply (two independent inputs + UPS + DGU). We statistically measure unavailability + recovery time + duration of preventive maintenance and scale the solution according to the above scheme, reach the required level of availability)
We have some nodes where there is a risk of political unrest in transit countries, via two more satellite channels.

Answer 2

Therefore, the question arose - how do "big" companies "fight" this?

Usually nothing or something.
If according to engineering, then by duplication, sometimes multiple (on aircraft, the control hydraulics can have fourfold duplication).
In large and adult companies, the input is usually a hardware stacked network balancer that spreads the load across clusters.

Answer 3

You can make a balancer at the DNS level.
And you, as I understand it, mean a tunnel (proxy).
PS
As a variant of Failover IP and a backup balancer.

Answer 4

There are also several balancers. on top of them DNS servers)

Answer 5

In other words, raise the service, and then multiply it by 2. (two servers, two switches, two blocks in servers, two UPSs, etc.)