How to fix the problem with the unavailability of NLB Unicast, which has a Dlink switch with a default gateway?

Question

D-Link

Who faced the problem with the availability of the NLB Unicast address, forwarding is enabled on the NLB interface, and the Dlink switch, followed by the router, acts as the default gateway, the second interface looking into the local network. These 2 servers are located behind different Dlink ports, the servers are located in their own vlan, the ports are configured as tagged. Dlink has only one entry pointing to ip / mac on one port and some of the clients can connect, but some can’t, apparently due to the fact that NLB throws them on the second server, there is no record about it on Dlink, how can I fix this problem and so that both servers behind NLB are available?