write-through RAM disk, or massive caching of file system?

write-through RAM disk, or massive caching of file system? I have a program that is very heavily hitting the file system, reading and writing randomly to a set of working files. The files total several gigabytes in size, but I can spare the RAM to keep them all mostly in memory. The machines this program runs on are typically Ubuntu Linux boxes.
Asked by: Guest \| Views: 290

Total answers/comments: 3

Comments display order:

Guest [Entry]

"Linux by default uses any spare RAM as a file cache, so no configuration is necessary for that.

You may want to consider using ext4 as the filesystem. It uses quite a number of techniques to speed up disk access, including delayed allocation which:

This has the effect of batching together allocations into larger runs. Such delayed processing reduces CPU usage, and tends to reduce disk fragmentation, especially for files which grow slowly. It can also help in keeping allocations contiguous when there are several files growing at the same time.

Data loss is pretty rare due to the use of journaling.

Ext4 is now the default filesystem in recent releases of Linux, though you will probably want to make sure the kernel you use is at least 2.6.30"

Guest [Entry]

"Linux uses your RAM very efficiently.
I don't think you really need to work on getting things cached with large files.
To check this out for yourself, try these things.

The Linux disk cache is very unobtrusive. It uses spare memory to greatly increase disk access speeds, and without taking any memory away from applications.

Check how much RAM your system has and if you can scale it up.
There is no need for a RAM-disk for this purpose,
the disk cache will work well with all available RAM space."

Guest [Entry]

RapidCache provides a RAM disk with write-through, but is currently "in beta".