Sparse file

From Wikipedia, the free encyclopedia

A sparse file: The empty bytes doesn't need to be saved, thus they can be spared out – instead there are only a few metadata which contain the information of the sparse bytes
A sparse file: The empty bytes doesn't need to be saved, thus they can be spared out – instead there are only a few metadata which contain the information of the sparse bytes

In computer science, a sparse file is a type of computer file that attempts to use file system space more efficiently when the file itself is mostly empty. This is achieved by not writing data to disk when it has been allocated but not actually filled with data. Instead, brief information about these empty regions is stored, which takes up much less disk space. These regions are only written to disk at their actual size when data is written to them; the file system transparently converts empty sections into blocks filled with zero bytes when read at runtime. Most modern file systems support sparse files, including most Unix variants and NTFS. Sparse files are commonly used for disk images, database snapshots, log files and in scientific applications.

Contents

[edit] Creating sparse files in Unix

If executed manually, the Unix command:

dd if=/dev/zero of=sparse-file bs=1 count=0 seek=1M

Will create a file of one megabyte in size, but with no data stored on disk (only metadata).

[edit] Detecting sparse files in Unix

Sparse files have different apparent and actual file sizes. This can be detected by comparing the output of:

du -s -B1 --apparent-size sparse-file

and:

du -s -B1 sparse-file

[edit] Advantages

The advantage of sparse files is that storage is only allocated when actually needed: disk space is saved, and large files can be created even if there is insufficient free space on the file system.

[edit] Disadvantages

Disadvantages are that sparse files may become fragmented; file system free space reports may be misleading; filling up file systems containing sparse files can have unexpected effects; and copying a sparse file with a program that does not explicitly support them may copy the entire, uncompressed size of the file, including the sparse, mostly zero sections which are not on disk—losing the benefits of the sparse property in the file.

[edit] See also

[edit] References

[edit] External links