freedup is a program to scan directories or file lists for duplicate files. The file lists may be provided to an input pipe or internally generated using find with provided options. There are more options to specify the search conditions more detailed. Other options influence the performed actions, i.e. whether to display only or to specify what kind of link under which circumstances. freedup first compares file sizes, then the MD5 signatures, and before taking actions a byte-by-byte check for verification is performed. An interactive mode allows to decide individually which files to link soft or hard or to delete.
fdupes is written in POSIX compliant C and is released under the GNU General Public License.
Other programs that can find duplicates and run under *nix: