Extfs crashed after free space ran out?

Question

Extended file system

This has happened several times already. if the space on the screw runs out, then simply deleting large files (usually logs) does not help - the file system still answers that zero is free. on reboot it runs fsck and finds a bunch of inode errors. at the end of the process, the system turns into a cocktail of indexes that is reset to “lost + found”
happened under debian 6 ext4 on a virtual machine, ubuntu ext4 on a local machine, centos 5 ext3 on a dedicated computer
who can advise how to avoid such horrors in the future?

Answer 1

Complete loss of data, it seems to never end.
To free up space - kill (preferably 9th) the process whose logs took up a lot of space (don't forget about logrotate and syslog-ng/klogd and others - they keep some logs).
Well, you shouldn't include all sorts of barrier=0, noatime and other tunings for the sake of the integrity of the file system, if this thing can be extinguished by power (that is, it's not so scary on a laptop).

Answer 2

Well, if you just have remote logs that do not make room, then the stump is clear that instead rm huge_fileyou need to delete it like this :>huge_file. In this case, the empty space will be freed immediately, because. the unfortunate file will not hang in the proc'e waiting until all the processes that exploit it mercilessly finally finish their work. And you don't have to kill anything.
And so that the fs flew due to overflow - I have never seen it. Although the very fact of overflow was observed more than a dozen times.

Answer 3

prevent overflow? set quotas?
As far as I know, all negative FS refer to overflow.