V
V
Vitaly R2018-02-01 10:11:50
PHP
Vitaly R, 2018-02-01 10:11:50

Why does apache create many processes, which eventually crashes the system?

Good day to all.
There is a zabbix 3.0 web server on a virtual machine running Debian 3.2.0-4-amd64 (vmware 12 cores 8 GB of RAM). On it:
NGINX - frontend, Apache - backend, php5.
Mysql on another physical machine
The load on the server is quite large. Zabbix has a huge number of cards with thousands of elements (I think the main problem is in them). Users can simultaneously work in the region of 100.
The situation is as follows. Relatively recently, the web server began to slow down.
Processes "/usr/sbin/apache -k start" spawn. Constantly. Regardless of the time of day and, accordingly, the workload of the server.
They multiply until the memory runs out and the server stops responding.
We reboot Apache - everything works fine and smartly. Services begin to multiply, but still everything works quickly and smartly.
10 minutes pass, services become 25, the percentage is loaded at 100% (all cores are 100 each), but still the web responds quickly enough.
20 minutes pass, there are already 40 services, but we don’t care, everything works cool for us.
30 minutes pass, there are already 60 services, we still endure, the memory eats up 5-6 gigs, but delays are already starting.
50 minutes pass, there are already 80 services, we can’t stand it and, feeling real brakes, we send Apache to reboot.

Then everything is in a circle. It doesn't matter what time it is, during work hours or late at night. The server dies after about an hour.
In apache2, the error.log starts swearing only when the memory is full and the server can no longer spawn processes. In other logs it seems to be clean as well.
I'm tired of fighting with this already. I ask you not to give advice in the style of "reinstall windows". This will be the most extreme stage, so far it is important to find the root of evil, and not cut down the entire forest.
I also want to add that before it was just nginx + php5-fpm and the situation was exactly the same, only php5-fpm processes were crying. (this was worse, because it was impossible to work when loading at 100% of all cores, and in the current situation, the fecundity of processes practically does not affect the speed of work until they gobble up all the memory).
I will also say the main phrase: "before everything worked fine for several months. I did not change any settings and it"
Where to look? What log to look at?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
@
@smple, 2018-02-01
_

1. the number of processes that can be described in the apache config (look for worker) is the same for php-fpm, the output must be opened in the config and specify the maximum number of handlers.
2. Updates come to Zabix from various devices, maybe it’s worth sending them not so often and looking at Zabix configs as well as settings for collecting updates? because it can be tritely sent too much data that it is unable to process, and then the decision is either to reduce the amount of data, or to scale the server (vertically or horizontally)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question