Several web resources on the same machine cannot open simultaneously on the internal network. Nginx and Mikrotik - who is to blame?

Question

Nginx

I have a picture (although it already seems to me that she is me):
Mikrotik router with external statics 1.1.1.1 and internal address 88.1, server 88.22 with app application web face, server 88.33 with nginx server configured with 2 sites and proxying to the application on server 88.22. I received a wildcard ssl certificate for the domain, so I specify the same certificate in the settings of all subdomains. Harpin NAT is configured on Mikrotik to provide access from the internal network to resources thrown outside (ports 443, 80, RDP, and so on ...), absolutely standard setup according to guides from the network, nothing unusual. During checks of the availability of sites and services, I did not find any problems, as I checked one at a time. But when I opened both sites at the same time, I saw that one was unavailable. Moreover, if you do nothing for 5 minutes, then the second site opens, while the first one becomes unavailable. I was afraid that it was that the same CMS blocks php on itself (I understand that it's nonsense, but I'm absolutely not familiar with the web, so I can't deny this possibility). But to check, I opened the proxied application and saw that it also did not open. That is, for 3 requests, the answer comes only from the first requester, and the rest go into timeout. I smoked nginx setup guides, which eventually resulted in its current configuration, but this did not help with the problem. Having smoked a second pack of cigarettes, inserted an LTE modem into the laptop and turned off the home network, and was surprised to find that all three sites can be easily opened from the outside and work flawlessly simultaneously from one browser. Now I sin on Mikrotik. Added static web names to dns, but it didn't work. It has also been noticed for a long time that outside it is much faster to establish an RDP connection to a machine with Windows,Actually, I ask for help from more advanced colleagues, who is to blame for my misadventures - Nginx or Mikrotik, and how to deal with it?

Mikrotik config excerpt:

/ip dhcp-server network
add address=192.168.88.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.88.1 \
    netmask=24
/ip dns
set servers=192.168.88.1
/ip dns static
add address=192.168.88.33 name=site1.example.com
add address=192.168.88.33 name=site2.example.com
add address=192.168.88.33 name=app.example.com
/ip firewall filter
add action=fasttrack-connection chain=forward comment="Fasttrack TCP" \
    connection-state=established,related protocol=tcp
add action=fasttrack-connection chain=forward comment="Fasttrack UDP" \
    connection-state=established,related protocol=udp
add action=accept chain=forward comment=\
    "1.1. Forward and Input Established and Related connections" \
    connection-state=established,related
add action=accept chain=input connection-state=established,related
add action=add-src-to-address-list address-list=ddos-blacklist \
    address-list-timeout=1d chain=input comment="Stop ddos" connection-limit=\
    100,32 in-interface-list=WAN protocol=tcp
add action=tarpit chain=input connection-limit=3,32 protocol=tcp \
    src-address-list=ddos-blacklist
add action=drop chain=input connection-state=invalid
add action=drop chain=forward connection-state=invalid
add action=drop chain=forward connection-nat-state=!dstnat connection-state=\
    new in-interface-list=WAN
add action=jump chain=forward comment=SYN-Protect connection-state=new \
    jump-target=SYN-Protect protocol=tcp tcp-flags=syn
add action=jump chain=input connection-state=new in-interface-list=WAN \
    jump-target=SYN-Protect protocol=tcp tcp-flags=syn
add action=return chain=SYN-Protect connection-state=new limit=200,5:packet \
    protocol=tcp tcp-flags=syn
add action=drop chain=SYN-Protect connection-state=new protocol=tcp \
    tcp-flags=syn
add action=drop chain=input comment="Port Scanners Protection" \
    src-address-list=Port-Scanners
add action=add-src-to-address-list address-list=Port-Scanners \
    address-list-timeout=none-dynamic chain=input in-interface-list=WAN \
    protocol=tcp psd=21,3s,3,1
add action=accept chain=input comment="Limit on ping" in-interface-list=WAN \
    limit=4,2:packet protocol=icmp
add action=drop chain=input comment="Drop All Other" in-interface-list=!LAN
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
add action=masquerade chain=srcnat dst-address=192.168.88.33 dst-port=80 \
    protocol=tcp src-address=192.168.88.0/24 to-ports=80
add action=masquerade chain=srcnat dst-address=192.168.88.33 dst-port=443 \
    protocol=tcp src-address=192.168.88.0/24 to-ports=443
add action=dst-nat chain=dstnat dst-address=1.1.1.1 dst-port=80 protocol=\
    tcp to-addresses=192.168.88.33
add action=dst-nat chain=dstnat dst-address=1.1.1.1 dst-port=443 \
    protocol=tcp to-addresses=192.168.88.33
add action=netmap chain=dstnat comment="RDP" dst-port=3389 \
    in-interface=ether1 protocol=tcp to-addresses=192.168.88.22 to-ports=\
    3389
add action=netmap chain=dstnat dst-address=1.1.1.1 dst-port=3389 \
    protocol=tcp src-address=192.168.88.0/24 to-addresses=192.168.88.22 \
    to-ports=3389
add action=masquerade chain=srcnat dst-address=192.168.88.22 dst-port=3389 \
    protocol=tcp src-address=192.168.88.0/24 to-ports=3389
/ip firewall raw
add action=drop chain=prerouting dst-port=53,137,138,139 in-interface-list=\
    WAN protocol=udp
/ip upnp interfaces
add interface=bridge1 type=internal
add interface=ether1 type=external
/routing igmp-proxy
set quick-leave=yes
/routing igmp-proxy interface
add alternative-subnets=0.0.0.0/0 disabled=yes interface=ether1 upstream=yes
add disabled=yes interface=bridge1

nginx config:

user www-data;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
worker_processes 4;
worker_rlimit_nofile 20000;

events {
  use epoll;
  worker_connections 800;
  multi_accept on;
}

http {
  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 30;
  keepalive_requests 100;
  types_hash_max_size 2048;	
  reset_timedout_connection on;
  client_body_timeout 10;
  send_timeout 2;
  client_max_body_size 20m;
  server_names_hash_bucket_size 64;
  include /etc/nginx/mime.types;
  default_type application/octet-stream;
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
  ssl_prefer_server_ciphers on;
  access_log off;
  error_log /var/log/nginx/error.log crit;
  gzip on;
  gzip_disable "msie6";
  gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
  open_file_cache max=200000 inactive=20s;
  open_file_cache_valid 30s;
  open_file_cache_min_uses 2;
  open_file_cache_errors on;
  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
}

Config of one site (the second one is absolutely similar, two wikis for different purposes):

server {
    listen               80;
    listen                   [::]:80;
    server_name          site1.example.com;
    return 301 https://$server_name$request_uri;
}
server {
    listen [::]:443 ssl;
    listen 443 ssl;
    server_name site1.example.com;
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_session_timeout 5m;
    ssl_ciphers "HIGH:!aNULL:!MD5 or HIGH:!aNULL:!MD5:!3DES";
    client_max_body_size 20M;
    client_body_buffer_size 128k;
    root /var/www/site1.example.com;
    index doku.php;
    #Remember to comment the below out when you're installing, and uncomment it when done.
    location ~ /(conf/|bin/|inc/|install.php) { deny all; }
    #Support for X-Accel-Redirect
    location ~ ^/data/ { internal ; }
    location ~ ^/lib.*\.(js|css|gif|png|ico|jpg|jpeg)$ {
        expires 365d;
    }
    location / { try_files $uri $uri/ @dokuwiki; }
    location @dokuwiki {
        # rewrites "doku.php/" out of the URLs if you set the userwrite setting to .htaccess in dokuwiki config page
        rewrite ^/_media/(.*) /lib/exe/fetch.php?media=$1 last;
        rewrite ^/_detail/(.*) /lib/exe/detail.php?media=$1 last;
        rewrite ^/_export/([^/]+)/(.*) /doku.php?do=export_$1&id=$2 last;
        rewrite ^/(.*) /doku.php?id=$1&$args last;
    }
    location ~ \.php$ {
        try_files $uri $uri/ /doku.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param REDIRECT_STATUS 200;
        fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
    }
}

Proxying:

server {
    listen 80;
    listen [::]:80;
    server_name app.example.com;
    return 301 https://$server_name$request_uri;
}
server {
    listen [::]:443 ssl;
    listen 443 ssl;
    server_name app.example.com;
    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers         HIGH:!aNULL:!MD5;
location / {
    proxy_pass http://192.168.88.22:80;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    }
}

Answer 1

The problem was solved by the following steps:
1) In the DHCP server, register the DNS address of the router (internal)
2) In IP / DNS, after registering static paths to the web server and resetting the cache, check the box Allow Remote Requests
3) do not forget to remove harpin NAT for 80 and 443 ports (only if there is no proxying on the Mikrotik itself, in my case the server with nginx acted as a proxy, that is, there was one entry point)
At the moment, a request for a domain name from inside the network immediately goes to the local ip and processes all available names at once, and not just one.
I'll try the same with other services (as I already said, there is a slight dissatisfaction with the connection speed via RDP).
The problem with this solution is that all domain names must be registered in Mikrotik, that is, when adding in bulk, there is a lot of manual work, although I have not yet looked at the possibilities of automating this process (I'm going to add about two dozen services). Thank you all for participating! I hope I helped not only myself!

Answer 2

look, for something (X) to work, two conditions must be met:
1) there must be a route from the tested address to address X
2) there must be a service on X that sends a certain service to the target port
about routes, you can ask here , a friend will figure it out and will tell everyone, to check the route, they use:
ping (better in the form of ping -t), traceroute (in Linux, in Windows tracert)
to check the port, telnet
is usually used with the help of the above, after performing the tests, you can find problem areas

Answer 3

Try clearing the DNS cache on Mikrotik. It helped me. Then I put a smaller ttl and it started working fine

Answer 4

Strange, I have everything tied to Mikrotiks. Even through vpn channels, forwarding for web servers works.
There are servers of different classes, cloud storages, web servers for many sites, ftp, etc.
Yes, dns statics are prescribed in mikrotiks, but you can do: 1) type api so that dns statics are automatically created, 2) raise the dns server and deploy it what you need.

Answer 5

We register the domain name of the site in the dns registrar. On Mikrotik, we set up port forwarding to the site from the outside, we set up a hairpin nat, that's all, nothing else is required. From the local dns, it is only necessary that it make recursive requests to external servers, and whether it will be on Mikrotik or somewhere else it is configured, it does not matter at all.
But it's best when your dns server is able to give different zone data for different subnets. Those. for the Internet, this is your external ip, and for the internal subnet, the internal ip of the web server, but users only know the site address.