fdupes

From Wikipedia, the free encyclopedia

fdupes is a program written by Adrian Lopez to scan directories for duplicate files, with options to list and delete them. It first compares file sizes and MD5 signatures, and then performs a byte-by-byte check for verification.

fdupes is written in C and is released under the MIT License.

[edit] Similar programs

Other programs that can find duplicates and run under *nix:

duff
dupmerge - runs on various platforms (Win32/64 with Cygwin, *nix, Linux etc.)
fdf - Perl/c based and runs across most platforms (Win32, *nix and probably others). Uses MD5, SHA1 and other checksum algorithms
freedup *mirror - POSIX C compliant and runs across platforms (Windows with Cygwin, Linux, AIX, etc)
fslint
liten - Pure Python deduplication command line tool, and library, using md5 checksums and a novel btye comparison algorithm. (Linux, Mac OS X, *nix, Windows)
rdfind
ua - Unix/Linux command line tool, designed to work with find (and the like).