How to properly network in Proxmox using pfSense?

Question

Network administration

There is a dedicated Hetzner server with one network interface (for example 42.88.80.15 (GT 42.88.80.1)) and 3 additional IP addresses (for example 42.88.80.27, 42.88.80.30, 42.88.79.152 (GT 42.88.79.225)). The server has Proxmox installed with virtual machines and containers. Two web servers use IP 42.88.80.27, 42.88.80.30.

One virtual machine has pfSense (Virtual Machines) installed, which acts as a gateway. Through it passes traffic to other virtual machines. Two virtual machines (on a Linux Container) use dedicated IPs, the others use the IP address of the machine where pfSense is installed.
Now the Proxmox server is configured with:

1. bridge vmbr0 (with external IP) to enp4s0;
   
2. vmbr1 (LAN 10.20.30.1)
   
3. other vmbr.
   

On the pfSense virtual machine, interfaces net0, net1, net2 (Intel E1000) are created, which look in vmbr0, but have their own MAC addresses that are bound to a specific IP. On pfSense itself, an interface is created for each interface, respectively.
Servers using IP 42.88.80.27, 42.88.80.30 are configured in NAT 1:1 mode, the rest via Port Forward.
I recently noticed that the server with IP 42.88.80.30 uses 42.88.80.27 for outgoing connections. In addition, everything is very slow, the servers respond for a long time. Apparently something is misconfigured. I met articles that said that such a network needs to be configured using VLANs.
As a result, I wanted to get the following:

1. two virtual machines used their own IPs (for outgoing and incoming traffic). These are web servers that host websites;
   
2. the other VMs shared the same IP address with the pfSense server for outbound traffic. For incoming traffic, only certain ports were forwarded to these machines (for example, ports 5060, 5061, 10000-20000, etc. wake up on a server with Asterisk);
   
3. all virtual machines are on a common pfSense local network (with the ability to limit traffic between individual machines);
   
4. the main IP address of the Hetzner server was only used to access Proxmox
   

Please tell me which model of organizing such a network will be the most correct and productive?

Answer 1

This scheme works for me, but with MIkrotik.
I added 6 interfaces (according to the number of additional IPs) looking in vmbr0 (the external interface of the server is also included there. MAC and naturally taken from the Robot.
Next, I made a LAN interface looking in vmbr1 (where all virtual machines are connected) and cut vlan- s. Accordingly, in the network settings of KVM machines and LXC containers, I specify not only the vmbr1 interface, but also the required vlan.
Mikrotik is configured with Source policy routing, a rake - you need to explicitly specify gateway interfaces, because Hetzner most often additional IPs come from the same network and with one GW.

Answer 2

The question is not very clear, as the answer is given in the picture.
Just do it on her

Answer 3

There is some useful information here:
https://docs.netgate.com/pfsense/en/latest/virtual...
Because the hardware checksum offload is not yet disabled, accessing pfSense webGUI might be sluggish. This is NORMAL and is fixed in the following step.
To disable hardware checksum offload, navigate under System > Advanced and select Networking tab. Under Networking Interfaces section check the Disable hardware checksum offload and click save. Reboot will be required after this step.
It is possible that the checksum verification settings on ProxMox slow down network packets.