D
D
Dmitry2016-06-21 10:46:15
Nginx
Dmitry, 2016-06-21 10:46:15

Why is nginx unavailable during reload?

During the reload of the nginx config (4 cores, 4Gb, acts as a reverse proxy) temporarily stops responding to requests, including those that come to localhost (zabbix reports unavailability). RPS at the same time is at the level of 1600-1800, netstat does not show anything, in my opinion, unusual.

$ netstat -tpn |awk '/:80/||/:443/{print $6}' |sort |uniq -c                                                                    3 CLOSE_WAIT
     13 CLOSING
  17568 ESTABLISHED
    292 FIN_WAIT1
    962 FIN_WAIT2
    171 LAST_ACK
    309 SYN_RECV
  54314 TIME_WAIT

$ netstat -tpn |awk '/nginx/{print $6,$7}' |sort |uniq -c
   3124 ESTABLISHED 10005/nginx
   3073 ESTABLISHED 10006/nginx
   2965 ESTABLISHED 10007/nginx
   2980 ESTABLISHED 10008/nginx

some nginx.conf:
worker_processes  4;
timer_resolution 100ms;
worker_priority -15;
worker_rlimit_nofile 200000;

events {
  worker_connections  65536;
  multi_accept on;
  use epoll;
}

http {

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;

  keepalive_requests 100;
  keepalive_timeout  65;

  server {
    listen  80 default_server backlog=1000;
    listen  443 ssl backlog=1000;
    ....
    }
}

custom sysctl.conf
net.ipv4.ip_local_port_range=1024 65535
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.all.secure_redirects=0
net.ipv4.conf.all.send_redirects=0
net.core.netdev_max_backlog=10000
net.ipv4.tcp_syncookies=0
net.ipv4.tcp_max_syn_backlog=20480
net.ipv4.tcp_synack_retries=2
net.ipv4.tcp_syn_retries=2
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.netfilter.nf_conntrack_max=1048576
net.ipv4.tcp_congestion_control=htcp
net.ipv4.tcp_timestamps=1
net.ipv4.tcp_no_metrics_save=1
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_tw_recycle=0
net.ipv4.tcp_max_tw_buckets=1400000
net.core.somaxconn=250000
net.ipv4.tcp_keepalive_time=900
net.ipv4.tcp_keepalive_intvl=15
net.ipv4.tcp_keepalive_probes=5
net.ipv4.tcp_fin_timeout=10

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Alex, 2016-06-30
@nesterwsx

dmesg+error.log?
+ useful to add tcp charts to zabbix from netstat -s

T
TaN, 2016-06-30
@TaN

There is exactly the same problem. Also, nginx stops responding when the logs are rotated.
The reason could not be found.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question