D
D
Dmitry2017-01-26 01:53:27
PHP
Dmitry, 2017-01-26 01:53:27

Why can phpb processes crash in fast_cgi + apache-worker mode?

There is a VPS server based on Debian 8, it runs apache-2.4.10 in mpm-worker mode,
for php I use mod_fcgi, without suexec.
PHP 5.6.29-0+deb8u1, without caching modules.
settings in general, nothing special:

<IfModule mod_fcgid.c>
FcgidIPCDir /var/run/mod_fcgid
FcgidProcessTableFile /var/run/mod_fcgid/fcgid_shm
FcgidMaxRequestLen 20000000
FcgidConnectTimeout 10
FcgidIdleTimeout 10
PHP_Fix_Pathinfo_Enable 1
FcgidMaxProcesses 200000
</IfModule>

php wrapper
#!/bin/bash
PHPRC=$PWD/../etc/php5
export PHPRC
umask 022
export PHP_FCGI_CHILDREN
export PHP_FCGI_MAX_REQUESTS=99999990
SCRIPT_FILENAME=$PATH_TRANSLATED
export SCRIPT_FILENAME
exec /usr/bin/php5-cgi

On the server 32GB of RAM + 8 cores, on ssd disks. The main content is a little static + light php scripts, the data is taken and written to redis on the same server. There are about 30 domains on the server, all work on the same principle.
The strange thing is that periodically, for unknown reasons, php starts to fall on the go, first there are several messages like this:
(104)Connection reset by peer: [client 1.1.1.1:29617] mod_fcgid: error reading data from FastCGI server  
[core:error] [pid 8840:tid 47795850016512] [client 1.1.1.1:29617] End of script output before headers: script.php

after which there is a hard fall of one domain (the most loaded one)
mod_fcgid: can't apply process slot for /path/to/wrapper/php

At the same time, the rest of the domains work without problems, each domain has its own wrapper.
It is treated for a while in different ways, but the following combination seems to help
/etc/init.d/apache2 stop
 rm -f /var/run/mod_fcgid/* 
 /etc/init.d/apache2 stop

Added ulimit -s unlimited to the /etc/init.d/apache2 script . The option to increase FcgidMaxProcesses does not help, it still falls from time to time. Of the symptoms - Crashes after restart (added domains to the server - after activation and "reload" for apache2 this situation occurred, also after rotating the logs and just on the go. There is a suspicion that all this is somehow connected with timeouts for FCGI and some still with limits, but I can’t figure out which ones, there is a similar config and with a higher load on another server, but it has 2 to 5 times more traffic.
I suspect something like - when Apache is reset, reloaded or something else, there are hanging fcgi tables somewhere in the system, when Apache starts, it expects from these tables (processes or how to call them correctly I don’t know) a response, data or some other activity, but it does not occur, as a result, some limits are clogged and the wrapper cannot create more processes for processing requests, as a result, Apache throws 503 and everything goes down.
Here on the server, the load is on average 30-50 requests per second.
I don’t consider the option to install nginx + php-fpm because I don’t see the point, php-fpm is not very different from fcgi mode, I think that under heavy load there may be similar problems, but I don’t want to test without serious arguments.
The main recommendation on the Internet is to increase the number of processes for FcgidMaxProcesses, but there is no sense, as well as from PHP_FCGI_MAX_REQUESTS.
Since all other domains work, I think the reason is in some php settings, but I can't figure out which ones. All domains work from www-data.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Anatoly Shipitsyn, 2017-02-01
@norguhtar

Wouldn't it be easier to switch to mod_proxy_fcgi + php-fpm? If you use Proxy via Handler , all rewrites will work. To automatically restart php-fpm when it suddenly crashes, you can use supervisord

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question