How to configure the routing of clients from the LAN through VPN, and the device itself through the provider?

Question

linux

Good afternoon!

I have a Raspberry Pi on my home network, which I want to use as both a dns ad cutter and a vpn gateway.

For these purposes, I installed AdGuard home and configured dns-over-tls on it, releasing it to the Internet. Now ads on mobile phones are cut not only when connected to home wifi, but also when using a mobile network (Android has standard settings for this: Settings -> Network and Internet -> Personal DNS server). For everything to work, for a certain subdomain, I configured a DNS record of type A, pointing to the IP received from the provider.

But with the simultaneous use of Malinka as a vpn gateway, problems arise. To connect to the VPN, I use the standard NordVPN application for Linux and the Nordlynx protocol (a proprietary implementation of Wireguard). As soon as the connection is up, all traffic goes into the tunnel and I lose access to the AdGuard dns server from the Internet. I must say right away that setting up a DNS record of type A on the IP VPN provider is not an option - all ports are closed there for incoming connections.

Thus, I need to set up the following scheme: the traffic of any client from the LAN (10.10.10.0/24) using Malinka as a gateway must be routed through a tunnel named nordlynx, and the traffic of Malinka itself must go through the provider.
I understand that I need policy routing ( https://habr.com/ru/post/108690/), but how exactly to configure it in such a configuration - I can’t guess yet. This is the part where I ask for help.

My current config.
nat rule for nordlynx interface:
-A POSTROUTING -o nordlynx -j MASQUERADE
Rules for FORWARD chain:

-A FORWARD -i eth0 -o nordlynx -j ACCEPT
-A FORWARD -i nordlynx -o eth0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

This is what the routing tables look like without VPN connected:

$ ip rule
0:      from all lookup local
220:    from all lookup 220
32766:  from all lookup main
32767:  from all lookup default

Here it is, with it enabled:

$ ip rule
0:      from all lookup local
217:    from all lookup main suppress_prefixlength 0
218:    not from all fwmark 0xca6c lookup 51820
219:    from all to <IP VPN провайдера> lookup main
220:    from all lookup 220
32766:  from all lookup main
32767:  from all lookup default

This is how routes without a VPN connection look like:

$ ip route
default via 10.10.10.1 dev eth0 onlink
10.10.10.0/24 dev eth0 proto kernel scope link src 10.10.10.100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1

And so - with a connected VPN:

$ ip route
default via 10.10.10.1 dev eth0 onlink
10.5.0.0/16 dev nordlynx proto kernel scope link src 10.5.0.2
10.10.10.0/24 dev eth0 proto kernel scope link src 10.10.10.100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1

Thanks in advance for your advice!

Answer 1

I managed to solve the problem in 2 ways. Duplicate my own comment to mark it as an answer - maybe it will help someone. The first one was suggested on the forum and it does exactly what I described in the initial message with the question: it passes client traffic from the LAN through the VPN tunnel, and the traffic of Malinka itself through the provider. To do this, after connecting the VPN, add a routing table with a default rule to let traffic through the router and 2 rules for the lo interface:

ip route add to default via 10.10.10.1 table 100
ip rule add iif lo to 10.10.10.0/24 lookup main prio 16000
ip rule add iif lo to default lookup 100 prio 16010

As it turned out, a significant disadvantage of this solution is an increase in ping from 4 to 120 ms. and a speed drop of more than 10 times to the provider. This is very sad.
I found the second solution to the problem when I started to google for the sake of interest about

not from all fwmark 0xca6c lookup 51820

The solution is described here: https://unix.stackexchange.com/questions/607004/ca ...
You need to add one route:
ip rule add from 10.10.10.100 lookup main
Then the traffic coming from the provider's IP will be able to reach Malinka.
You need to add rules after connecting because otherwise the cunning NordVPN will look at their order and add their own rules before them.

Answer 2

You just need to add a static route to your DNS provider in the raspberry, where AdGuard goes. This route should be through the DNS of your telecom operator (or whatever you have now as a default gateway without a raised VPN).
No complicated gestures with setting up a firewall are required.

Answer 3

Something is not visible how all the traffic of clients using Malinka as a gateway is sent to the VPN.
Even stranger is ip rule 217: from all lookup main suppress_prefixlength 0, which simply repeats the standard rule 32766: from all lookup main.
Either routes look completely different in the case of a VPN connection, or something else. Because with this configuration, clients from the Internet should easily access DNS. Maybe there is something in iptables?
And what does it give ip route show table 220?