Ubuntu server hacked. What actions should be taken?

Question

ubuntu

Good afternoon!
Recently discovered that our ubuntu server has been hacked.
The server administrator cannot connect to the mysql database using the old account. Another user has appeared in the system with administrator privileges.
There are two questions:
1. In a similar situation, when you find out that your server has been hacked. What else is worth paying attention to?
For example, has the bootloader config been changed or have the rights of some specific files been transferred to another user?
2. What should be done to restrict hacker's access to our server?
I look forward to your comments.

Answer 1

Zero out everything. Put the OS on bare partitions. Restore the code of their repository, and the database from backups.

Answer 2

If you have never done forensics, the best option is to take your data and your code, view them with your eyes, clean them from malicious code and transfer them to a new server

Answer 3

2. What should be done to restrict hacker's access to our server?

Here, in my opinion, everything has already been repeated twenty times - administrative access is only through ssh and only by keys. Gui is evil.

Answer 4

In addition to ssh, there may be other vulnerable services - ftp, mysql, munin.
Also, vulnerabilities can be in the CMS or web application code.
If you deploy a vulnerable application from scratch, then the holes from it will not disappear and after a certain time the site will be hacked again. In order to find the original problem, you can use vulnerability scanners.
Metascan , Acunetix , Qualys .