What to do if root access is compromised on the WEB server?

Question

linux

So by chance it turned out that the authorities gave the root password to an outsider.
He has no legal obligations with the company.
The server has private info of third parties.
There are no hints of dishonesty of this person, but if we start from the importance of maintaining the confidentiality of data - what actions can be taken to protect ourselves from the consequences of this visit to the server ???
There are virtual machines on the server, there are many web services that contain tens of thousands of files in different programming languages, so the advice to simply reinstall the system will not solve anything. In addition, virtual machines have their own direct access from the outside.

Answer 1

It all depends on what the rights were issued for, for how long and what work was carried out.
Change your password to get started.
It can be assumed that a third-party person could install something, do something, the logs will tell you what was done in the system, unless, of course, they were lost)))
If the authorities ask, just indicate that: 1. you didn’t pass the password 2. there wasn’t tasks log actions
Well, avoid this in the future.

Answer 2

It is unlikely that some kind of rootkit was inserted there, for you a reason to think about extended logging of actions in the cli, as well as monitoring all login attempts.

Answer 3

If you do not care about security, then you must reinstall the OS: the system is compromised.
An alternative option is to hope that the person is respectable.
Next time, do not assign many roles to one machine, use virtual machines and configuration management tools.

Answer 4

For the future, set up a backup, at least system files, settings, etc. to have something to compare with. what has changed, what is new.
backup **pu covers in many cases.
in your case, try to see what files have changed in the entire system over the last ** days that have passed since the root was compromised. the solution is not complete (as I understand it, it is unrealistic to manually validate all the files of the system), the work will be nowhere more debility, but at least an attempt.
But now there is a universal excuse.

Answer 5

The first step is to change your password.
Then study the logs - if there are any :)
Then evaluate the changes in the files for the time interval between the password leak and its change.
If virtual machines have their own roots, then they are most likely not compromised - you have to dodge very cool to change something in the disk image of the machine from the host machine.
But the host machine will still have to be rearranged. Although the leak, if it was - it has already happened and then the human factor - what kind of person it was, what kind of data it was, how interesting it was for him to steal it and what he could steal in the first place.