Do you need a solution that, if one server fails, will redirect incoming traffic to another server?

Question

Network administration

There are 4 real IP addresses from one provider: 3 IP addresses are already in use (1.1.1.1, 2.2.2.2, 3.3.3.3) on 3 adjacent servers in his data center and one IP is free (4.4.4.4)
There is 1 more IP ( 5.5.5.5) from another provider, which is used by the server located in its data center.

Let's admit clients are rigidly attached to IP 1.1.1.1. A solution is needed that, in case of failure of 1.1.1.1, will redirect all traffic to 5.5.5.5 (another data center).
I know itpables does it, but the catch is that it will do it when set to 1.1.1.1 and when that server is up. And it is necessary that it be possible to make a redirect even if the server 1.1.1.1 is turned off. At the same time, it is undesirable to put a separate server in front of it that will steer this, because if it falls, there will be no redirection at all. That is, it is impossible to make the MAIN link in this chain, it is necessary to be interchangeable. For example, temporarily assign IP 1.1.1.1 next to 2.2.2.2, which will redirect to 5.5.5.5, and when 1.1.1.1 returns to normal operation, then return IP and cancel the firewall policy on 2.2.2.2. But in this scheme, you need to install iptables on each machine, and monitor them in order to turn on and off the necessary / unnecessary firewalls. This is not entirely correct, as I understand it.

Please suggest a more elegant solution to this problem.

Answer 1

In your particular case, only configure via DNS, because:
1. You do not want to install additional equipment.
2. Servers in different networks.
Make short TTL for A record, say 3-5 minutes. On the DNS server, hang a script in cron, or a daemon that will check the availability of the first server and, if unavailable, change the A record in the domain zone to the IP of the second server and reread the zone, when the first server comes back to life, change it back accordingly.
maximum delays = TTL*2

Answer 2

Maybe I'll say something stupid, but don't records for this make different ones per domain?

Answer 3

read about clusters

Answer 4

Try to look in the direction of CARP , the only thing is that it needs 3 addresses located on the same network for it to work.
The scheme is as follows, we make 1.1.1.1 CARP the address > If the server is available, everything works as it is, in case of unavailability, another server answers, and then redirects traffic to another data center.
In fact, the allowable Downtime is very critical, in the case of CARP it is a couple of seconds. And yes, DNS

Answer 5

CARP, in the worst case HA Proxy or nginx.
Nothing is hung on the input address, except for the balancer.

Answer 6

1) Look at the radware company's products
2) Did a heart-beat between servers. And after a series of checks (is the application alive? is the server alive?), the IP address was spoofed.

Answer 7

it is necessary to put a server that already redirects people to other servers behind it. Because he only redirects himself, then he does not fall. The rest are already as lucky. Those. when one server behind it falls, it must stop directing traffic to the one that fell, and distribute it to others. Mustache. No other way. All other methods are tied to changing the IP.
Alternative:
In your application, when connecting to the main server, you can get from it the address of the server to which to send a particular user. At the same time, it will be possible to smear the load. But there is no hard binding to IP, which is good.