How to configure policy routing with src ip and dst interface substitution?

Question

linux

Good afternoon!
The situation is this:
2 providers, you need to configure routing in such a way that output icmp goes through the second, and everything else through the first.
Added route table, route list. Iptables replaces src
settings:

[[email protected] ~]# ip ru list
0:	from all lookup local 
32757:	from all fwmark 0x1 lookup gw2 
32758:	from 192.168.110.135 lookup gw2 
32759:	from 192.168.21.6 lookup gw1

[[email protected] ~]# ip ro sh ta gw1
default via 192.168.21.1 dev eth0 
[[email protected] ~]# ip ro sh ta gw2
default via 192.168.110.1 dev eth1

iptables -A POSTROUTING -p icmp -j SNAT --to-source 192.168.110.1 -t nat
iptables -A OUTPUT -p icmp -j MARK --set-xmark 0x1/0xffffffff

icmp traffic goes through the gateway of the first provider, but with the second one. if you run ping -I eth1 (port of the second provider), then the pings do not go away at all until I explicitly register the route through gw of the second dip. How to be?

Answer 1

FORWARD packets are normally tagged and sent to the right place. the problem was with OUTPUT from the server. solved like this:

iptables -A OUTPUT -t mangle -p icmp -j MARK --set-mark 100
iptables -A POSTROUTING -t nat -p icmp -j SNAT --to-source 192.168.110.135

from all fwmark 0x64 lookup gw-a 
from 192.168.210.6 lookup gw-d
from 192.168.110.135 lookup gw-a

# ip ro
default via 192.168.210.1 dev eth0 
192.168.210.0/24 dev eth0  scope link  src 192.168.210.6 
192.168.110.128/26 dev eth1  scope link  src 192.168.110.135

# ip ro sh table gw-a
default via 192.168.110.129 dev eth1 
192.168.110.128/26 dev eth1  scope link  src 192.168.110.135 

# ip ro sh table gw-d
default via 192.168.210.1 dev eth0 
192.168.210.0/24 dev eth0  scope link  src 192.168.210.6

Answer 2

perhaps you can adapt my configs for your tasks
I have a gateway - it divides traffic into two providers,
interfaces:
enp2s0 local network
enp2s1 prov1
enp2s3 prov2
some clients go only through 1st provider (someone through prov1 and someone through prov2) the rest through both at once
all commands below from iptables-save mangle table:
We catch all new connections and hang labels
-A PREROUTING -i enp2s0 -m state --state NEW -j MULTIWAN_MARK We
catch traffic related to already stopped connections and cling the marking to packets
-A PREROUTING -i enp2s0 -m state --state RELATED,ESTABLISHED -j MULTIWAN_RESTORE
-A MULTIWAN_RESTORE -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
on external interfaces mark 0x8000 and 0x8001
-A FORWARD -i enp2s1 -j CONNMARK --set-xmark 0x8000/0xffffffff
-A FORWARD -i enp2s3 - j CONNMARK --set-xmark 0x8001/0xffffffff
-A POSTROUTING -o enp2s1 -j CONNMARK --set-xmark 0x8000/0xffffffff
-A POSTROUTING -o enp2s3 -j CONNMARK --set-xmark 0x8001/0xffffffff
here some clients only need to walk through 1n external interface, specify which external interface
-A MULTIWAN_MARK -s 10.10.0.3/32 -j MULTIWAN_enp2s3
-A MULTIWAN_MARK -s 10.10.0.200/32 -j MULTIWAN_enp2s1
-A MULTIWAN_MARK -s 10.10.0.4/32 -j
MULTIWAN_ENP2s1 -s 10.10.0.5/32 -j MULTIWAN_enp2s1
-A MULTIWAN_MARK -s 10.10.0.2/32 -j MULTIWAN_enp2s3 marking of types from the
list above two providers 1n 1m default proto static nexthop via 192.168.10.254 dev enp2s1 weight 1 nexthop via 172.16.0.1 dev enp2s3 weight 1