Md5deep
Encyclopedia
md5deep is a software package used in the computer security
, system administration
and computer forensics
communities for purposes of running large numbers of files
through any of several different cryptographic
digests. It was originally authored by then Special Agent of the Air Force Office of Special Investigations
, Jesse Kornblum, and is still maintained by him.
The name md5deep is misleading. As of version 2.0, the md5deep package contains several different programs
capable of performing MD5
, SHA-1, SHA-256, Tiger192 and Whirlpool digests, each one of them named by the digest type followed by the word deep—thus, the name may confuse some people into thinking it only provides the MD5 algorithm when the package supports many more.
md5deep can be invoked in several different ways. Typically a user will operate it recursively
, where md5deep walks through one directory
at a time giving digests of each file found, and recursing into any subdirectories within. Its recursive behavior is approximately depth-first
, which has the benefit of presenting files in lexicographical order
. On Unix-style systems, similar functionality can be often obtained by combining find
with hashing utilities such as md5sum
, sha256sum, or tthsum.
md5deep exists for Windows
and most UNIX
-based systems, including Mac OS X
. It is present in Mac OS X's Fink
and MacPorts
projects. Binary packages exist for most free
UNIX systems. Many vendors initially resist including md5deep as they mistakenly believe its functionality can be reproduced with a single line of shell scripting. The matching functionality of the program, however, cannot be done easily in shell.
Because md5deep was written by an employee of the U.S. government, it is in the public domain
. Other software surrounding it, such as graphical front-ends, may be copyrighted.
Computer security
Computer security is a branch of computer technology known as information security as applied to computers and networks. The objective of computer security includes protection of information and property from theft, corruption, or natural disaster, while allowing the information and property to...
, system administration
System administrator
A system administrator, IT systems administrator, systems administrator, or sysadmin is a person employed to maintain and operate a computer system and/or network...
and computer forensics
Computer forensics
Computer forensics is a branch of digital forensic science pertaining to legal evidence found in computers and digital storage media...
communities for purposes of running large numbers of files
Computer file
A computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...
through any of several different cryptographic
Cryptography
Cryptography is the practice and study of techniques for secure communication in the presence of third parties...
digests. It was originally authored by then Special Agent of the Air Force Office of Special Investigations
Air Force Office of Special Investigations
The Air Force Office of Special Investigations , is a Field Operating Agency of the United States Air Force that provides professional investigative services to commanders throughout the Air Force...
, Jesse Kornblum, and is still maintained by him.
The name md5deep is misleading. As of version 2.0, the md5deep package contains several different programs
Computer program
A computer program is a sequence of instructions written to perform a specified task with a computer. A computer requires programs to function, typically executing the program's instructions in a central processor. The program has an executable form that the computer can use directly to execute...
capable of performing MD5
MD5
The MD5 Message-Digest Algorithm is a widely used cryptographic hash function that produces a 128-bit hash value. Specified in RFC 1321, MD5 has been employed in a wide variety of security applications, and is also commonly used to check data integrity...
, SHA-1, SHA-256, Tiger192 and Whirlpool digests, each one of them named by the digest type followed by the word deep—thus, the name may confuse some people into thinking it only provides the MD5 algorithm when the package supports many more.
md5deep can be invoked in several different ways. Typically a user will operate it recursively
Recursion
Recursion is the process of repeating items in a self-similar way. For instance, when the surfaces of two mirrors are exactly parallel with each other the nested images that occur are a form of infinite recursion. The term has a variety of meanings specific to a variety of disciplines ranging from...
, where md5deep walks through one directory
Directory (file systems)
In computing, a folder, directory, catalog, or drawer, is a virtual container originally derived from an earlier Object-oriented programming concept by the same name within a digital file system, in which groups of computer files and other folders can be kept and organized.A typical file system may...
at a time giving digests of each file found, and recursing into any subdirectories within. Its recursive behavior is approximately depth-first
Depth-first search
Depth-first search is an algorithm for traversing or searching a tree, tree structure, or graph. One starts at the root and explores as far as possible along each branch before backtracking....
, which has the benefit of presenting files in lexicographical order
Lexicographical order
In mathematics, the lexicographic or lexicographical order, , is a generalization of the way the alphabetical order of words is based on the alphabetical order of letters.-Definition:Given two partially ordered sets A and B, the lexicographical order on...
. On Unix-style systems, similar functionality can be often obtained by combining find
Find
In Unix-like and some other operating systems, find is a command-line utility that searches through one or more directory trees of a file system, locates files based on some user-specified criteria and applies a user-specified action on each matched file...
with hashing utilities such as md5sum
Md5sum
md5sum is a computer program that calculates and verifies 128-bit MD5 hashes, as described in RFC 1321. The MD5 hash functions as a compact digital fingerprint of a file. As with all such hashing algorithms, there is theoretically an unlimited number of files that will have any given MD5 hash...
, sha256sum, or tthsum.
md5deep exists for Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
and most UNIX
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
-based systems, including Mac OS X
Mac OS X
Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
. It is present in Mac OS X's Fink
Fink
The Fink project is an effort to port and package open-source Unix programs to Mac OS X. Fink uses dpkg and APT , as well as its own frontend program, fink ....
and MacPorts
MacPorts
MacPorts, formerly called DarwinPorts, is a package management system that simplifies the installation of software on the Mac OS X and Darwin operating systems. It is a free/open source software project to simplify installation of other free/open source software...
projects. Binary packages exist for most free
Free software
Free software, software libre or libre software is software that can be used, studied, and modified without restriction, and which can be copied and redistributed in modified or unmodified form either without restriction, or with restrictions that only ensure that further recipients can also do...
UNIX systems. Many vendors initially resist including md5deep as they mistakenly believe its functionality can be reproduced with a single line of shell scripting. The matching functionality of the program, however, cannot be done easily in shell.
Because md5deep was written by an employee of the U.S. government, it is in the public domain
Public domain
Works are in the public domain if the intellectual property rights have expired, if the intellectual property rights are forfeited, or if they are not covered by intellectual property rights at all...
. Other software surrounding it, such as graphical front-ends, may be copyrighted.
See also
- md5sumMd5summd5sum is a computer program that calculates and verifies 128-bit MD5 hashes, as described in RFC 1321. The MD5 hash functions as a compact digital fingerprint of a file. As with all such hashing algorithms, there is theoretically an unlimited number of files that will have any given MD5 hash...
- sha1sumSha1sumsha1sum is a computer program that calculates and verifies SHA-1 hashes. It is commonly used to verify the integrity of files. It is installed by default in most Unix-like operating systems...
from GNU Core UtilitiesGNU Core UtilitiesThe GNU Core Utilities or coreutils is a package of GNU software containing many of the basic tools, such as cat, ls, and rm, needed for Unix-like operating systems... - MD5MD5The MD5 Message-Digest Algorithm is a widely used cryptographic hash function that produces a 128-bit hash value. Specified in RFC 1321, MD5 has been employed in a wide variety of security applications, and is also commonly used to check data integrity...
- SHA-1
- SHA-2SHA-2In cryptography, SHA-2 is a set of cryptographic hash functions designed by the National Security Agency and published in 2001 by the NIST as a U.S. Federal Information Processing Standard. SHA stands for Secure Hash Algorithm. SHA-2 includes a significant number of changes from its predecessor,...
(SHA-224, SHA-256, SHA-384, SHA-512) - mtree similar utility from FreeBSD.