D
D
DmitryKoterov2013-01-11 21:49:34
linux
DmitryKoterov, 2013-01-11 21:49:34

How to make Linux stop flushing the filesystem metadata cache from memory?

I noticed that if you do rsync on a very large number of files (for example, on the whole machine - i.e. on "/" - when you run rdiff-backup: millions of files), then it takes a very long time and greatly increases LA, even if in reality rsync does not detect any changes. If, however, immediately after the process is completed, you start it again, then everything happens HUNDREDS of times faster (the first rdiff-backup can run for 5 hours, and the second immediately after it can run for 2 minutes). The same applies to any other operation - for example, "find /".
I know that the OS caches information about the structure of the file system (where what is, how much it takes and when it changed), but over time this cache seems to be forced out of memory by other data (because on a DBMS machine with a large database). So the daily rdiff-backup forces the whole structure to be brought back into memory again - and this causes a huge number of seeks and hangs up the machine.
Is there some way to tell Linux to always keep this metadata cache in memory and never clear it, even if it's not used for a long time? (I want to emphasize that this is metadata - the cache of the CONTENT of the files, let it be as it was, and everything is fine with it.)

Answer the question

In order to leave comments, you need to log in

2 answer(s)
M
motl, 2013-01-18
@motl

echo 0 > /proc/sys/vm/vfs_cache_pressure
www.kernel.org/doc/Documentation/sysctl/vm.txt

V
vsespb, 2013-01-11
@vsespb

1. you can do more vm.swappiness, maybe some not very necessary program or not very necessary memory (leaked in the program) will go into the swap, then it will remain there. there will be more space for the cache
2. in general, the same problem was with rsnapshot + ext4. generally slows down terribly. Moreover, this type of backup takes up a lot of space in the form of inodes. so I'm trying to switch to duplicity (where the backup files are not millions)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question