IKEv2 and two WAN interfaces?

Question

network hardware

Hello, please tell me about the correct IPSec routing on Mikrotik or in general.
We have:
- Office (Mikrotik, 192.168.55.0/24)
- Two WAN interfaces in the Office (ISP1 - 78.11.33.69, ISP2 - 53.16.99.7)
- Remote machine (Win Server)
On Mikrotik in the office, two channels were raised from default route to them with different metrics.
No complex failover configurations with netwatch scripts or external monitoring have been done yet,
just stupidly two default routes with different metrics.
The connection mangle is configured so that the packets go to the same ISP from which they came.
And everything works fine, within the framework of the needs.
But suddenly we wanted to tie remote servers to the domain and not just tie them, but using fashionable IKEv2.
And everything seems to be fine, set up - it works.
But then I wanted to dispose of the backup channel, but the fact that it "lies" does nothing for itself.
I decided to try to make IKEv2 tunnels go up through ISP2, and they went up. And they even seem to work for a while. But then it's hard to go to bed. Moreover, from the side of Mikrotik, this obviously peer takes and disappears. But on the Win server side, everything looks as if the connection has been established. That's just the packets do not pass anymore. Reconnecting again works on the strength of about five minutes and hello again.
Moreover, through ISP2 (backup channel), the viewer for cameras and RDP works quietly, and by and large everything that they could try, but there is no IPSec.
Question where to dig? If I understand everything correctly in the direction of routing, in the sense that the viewer for cameras and RDP work within the same connection session and they are correctly processed in the mangle and, accordingly, are also correctly routed.

Answer 1

I will share my personal experience:
There was a case of two Mikrotiks in different offices, each had a white IP, between them there was an l2tp tunnel (the type of tunnel is not important)
The tunnel also fell away like this.
network of the 1st mikrotik 192.168.0.0/24
network of the 2nd mikrotik 192.168.99.0/24
addresses inside the tunnel mirkotik1 10.10.10.1 <---> 10.10.10.2 mikrotik2
and the routes were registered, everything worked but sometimes fell.
As it turned out, with the wan of the Mikrotik2 interface (which had a white ip), arping showed the ip 10.10.10.1 (it was used by someone in the provider's network) if this address fell into the Mikrotik2 arp table with the wan interface, then everything broke because the packets did not go into the tunnel, but in wan interface. The problem was solved by changing the IP addresses inside the tunnel.