How to send different types of traffic to different providers?

Question

linux

It is required to mark the necessary traffic in iptables and send it to the required gateway.
now everything works through the first uplink:

# traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  192.168.210.1  0.220 ms  0.243 ms  0.234 ms
 2  192.168.105.101  20.163 ms  20.188 ms  20.224 ms
 3  192.168.66.131  1.653 ms  1.718 ms  2.050 ms
 4  193.106.112.21  8.025 ms  7.841 ms  8.005 ms
 5  216.239.47.145  8.984 ms 216.239.47.149  8.520 ms 216.239.47.151  8.927 ms
 6  8.8.8.8  9.259 ms  8.713 ms  9.580 ms

add rules:

# ip ru a fwmark 1 tab 1077
# ip ro sh tab 1077
default via 192.168.110.129 dev eth1
# ip ru l
0:	from all lookup local 
32764:	from all fwmark 0x1 lookup 1077 
32765:	from 192.168.110.135 lookup 1077
32766:	from all lookup main 
32767:	from all lookup default
# iptables -A OUTPUT -t mangle -d 8.8.8.8 -j MARK --set-mark 0x1
# iptables -A POSTROUTING -t nat -d 8.8.8.8 -j SNAT --to-source 192.168.110.135
# traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  192.168.110.129 (192.168.110.129)  1.771 ms  2.202 ms  4.950 ms
 2  * * *
 3  * * *
 4  * * *

and doesn't work. and tcpdump catches packets in both directions

# tcpdump -i eth1 icmp and host 8.8.8.8 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
17:18:29.578352 IP 192.168.110.135 > 8.8.8.8: ICMP echo request, id 4077, seq 1, length 64
17:18:29.586762 IP 8.8.8.8 > 192.168.110.135: ICMP echo reply, id 4077, seq 1, length 64

the addresses are real, just conspiratorial. there are no other rules in iptables
where to look to look?

Answer 1

The rules look correct... But this behavior is very similar to rp_filter.
Try turning it off:

sysctl net.ipv4.conf.default.rp_filter=0
sysctl net.ipv4.conf.all.rp_filter=0

A little immersed in this situation, there is a feature. If you remove SNAT, but at the same time do it ping 8.8.8.8 -I eth1, then with rp_filter enabled, it works. The fact is that in the routing tables you have an outgoing address for 8.8.8.8 from another network / interface (because there is a default or another route suitable for 8.8.8.8), so any program (socket), be it ping or a browser , will select the default desired outgoing address according to the routing tables (but not according to the labels there). And when iptables marks, and the packet gets to another gateway / interface according to the mark, rp_filter is triggered, because the source address does not belong to the network on the outgoing interface. Therefore, if the task is to send specific addresses to a specific gateway, then it is better to create many ip rules and not mark traffic, get rid of SNAT. If this is not possible, then yes, disable rp_filter.