Why does the server completely freeze for a few seconds?

Question

linux

Greetings!
Only a very experienced system administrator can help. )
There is an iron server running Oracle Linux with a production database on it without the ability to reboot. In preparation for standby server.
The server has one HDD with the operating system and the Oracle distribution kit. The database itself lies in SANe.
From time to time the server becomes very bad. More than 1000 concurrent sessions appear in the database, IOWAT jumps to 100 percent for absolutely all processes and disks, and the processor load does not grow. After 5-10 seconds, everything passes without intervention. Competing sessions in the database gradually end and the server works fine for a while.
It is noticed that the longer the server works (starts after 2 weeks), the more often the problem occurs. First, once a day. After a couple of months, every 5-10 minutes and the server has to be stopped.
Normally, IOWAIT is no more than 2%, the processor load is 20-25%.
How can you localize the problem?
Thank you!

Answer 1

It's not clear from your description

More than 1000 concurrent sessions appear in the database

and therefore because of this

the server gets really bad

or vice versa - problems with the server cause an increase in the session queue?
You need to determine the root cause and then look, if these are sessions, what happens in them, why are there so many who generate them. Otherwise, look for a problem in the gland.

Answer 2

What do atop, iotop say? What is the load on the percent, memory, disks, network? Is there and how much swap is used? Is there iowait?