SHA hash functions
From Wikipedia, the free encyclopedia
-
"SHA" redirects here. For the IATA airport code, see Shanghai Hongqiao Airport.
The SHA (Secure Hash Algorithm) hash functions, namely SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512, are five related cryptographic hash functions designed by the National Security Agency (NSA) and collectively published as a US government standard. SHA-1 is employed in a large variety of popular security applications and protocols, including TLS, SSL, PGP, SSH, S/MIME, and IPSec. It was considered to be the successor to MD5, an earlier, widely-used hash function. Both are reportedly compromised.
The other four variants (SHA-224, SHA-256, SHA-384, and SHA-512) are sometimes collectively referred to as SHA-2 functions or simply SHA-2. No attacks have yet been reported on the SHA-2 variants, but since they are similar to SHA-1, researchers are worried, and are developing candidates for a new, better hashing standard [1][2].
Contents |
[edit] SHA-0 and SHA-1
The original specification of the algorithm was published in 1993 as the Secure Hash Standard, FIPS PUB 180, by US government standards agency NIST (National Institute of Standards and Technology). This version is now often referred to as "SHA-0". It was withdrawn by the NSA shortly after publication and was superseded by the revised version, published in 1995 in FIPS PUB 180-1 and commonly referred to as "SHA-1". SHA-1 differs from SHA-0 only by a single bitwise rotation in the message schedule of its compression function; this was done, according to the NSA, to correct a flaw in the original algorithm which reduced its cryptographic security. However, the NSA did not provide any further explanation or identify what flaw was corrected. Weaknesses have subsequently been reported in both SHA-0 and SHA-1. SHA-1 appears to provide greater resistance to attacks, supporting the NSA's assertion that the change increased the security.
SHA-1 (as well as SHA-0) produces a 160-bit digest from a message with a maximum length of 264 - 1 bits, and is based on principles similar to those used by Professor Ronald L. Rivest of MIT in the design of the MD4 and MD5 message digest algorithms.
[edit] Cryptanalysis of SHA-0
At CRYPTO 98, two French researchers presented an attack on SHA-0 (Chabaud and Joux, 1998): collisions can be found with complexity 261, fewer than the 280 for an ideal hash function of the same size.
In 2004, Biham and Chen found near-collisions for SHA-0 — two messages that hash to nearly the same value; in this case, 142 out of the 160 bits are equal. They also found full collisions of SHA-0 reduced to 62 out of its 80 rounds.
Subsequently, on 12 August 2004, a collision for the full SHA-0 algorithm was announced by Joux, Carribault, Lemuet, and Jalby. This was done by using a generalization of the Chabaud and Joux attack. Finding the collision had complexity 251 and took about 80,000 CPU hours on a supercomputer with 256 Itanium 2 processors.
On 17 August 2004, at the Rump Session of CRYPTO 2004, preliminary results were announced by Wang, Feng, Lai, and Yu, about an attack on MD5, SHA-0 and other hash functions. The complexity of their attack on SHA-0 is 240, significantly better than the attack by Joux et al [3], [4].
In February 2005, an attack by Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu was announced which could find collisions in SHA-0 in 239 operations [5], [6].
[edit] Cryptanalysis of SHA-1
In the light of the results on SHA-0, some experts suggested that plans for the use of SHA-1 in new cryptosystems should be reconsidered. After the CRYPTO 2004 results were published, NIST announced that they planned to phase out the use of SHA-1 by 2010 in favour of the SHA-2 variants [7].
In early 2005, Rijmen and Oswald published an attack on a reduced version of SHA-1 — 53 out of 80 rounds — which finds collisions with a complexity of fewer than 280 operations [8].
In February 2005, an attack by Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu was announced [9]. The attacks can find collisions in the full version of SHA-1, requiring fewer than 269 operations (a brute-force search would require 280).
The authors write: "In particular, our analysis is built upon the original differential attack on SHA0 [sic], the near collision attack on SHA0, the multiblock collision techniques, as well as the message modification techniques used in the collision search attack on MD5. Breaking SHA1 would not be possible without these powerful analytical techniques" [10]. The authors have presented a collision for 58-round SHA-1, found with 233 hash operations. The paper with the full attack description was published in August 2005 at the CRYPTO conference.
In an interview, Yin states that, "Roughly, we exploit the following two weaknesses: One is that the file preprocessing step is not complicated enough; another is that certain math operations in the first 20 rounds have unexpected security problems." [11].
On 17 August 2005, an improvement on the SHA-1 attack was announced on behalf of Xiaoyun Wang, Andrew Yao and Frances Yao at the CRYPTO 2005 rump session, lowering the complexity required for finding a collision in SHA-1 to 263. [12]
In academic cryptography, any attack that has less computational complexity than a brute force search is considered a break [13]. This does not, however, necessarily mean that the attack can be practically exploited. It has been speculated that finding a collision for SHA-1 is within reach of massive distributed Internet search.
In terms of practical security, the major concern about this new attack is that it might pave the way to more efficient ones. Whether this is the case has yet to be seen, but a migration to stronger hashes is believed to be prudent. A collision attack does not present the same kinds of risks that a preimage attack would. Many of the applications that use cryptographic hashes, such as password storage or document signing, are only minimally affected by a collision attack. In the case of document signing, for example, an attacker could not simply fake a signature from an existing document—the attacker would have to fool the private key holder into signing a preselected document. Reversing password "encryption" (e.g. to obtain a password to try against a user's account elsewhere) is not made possible by the attacks. Constructing a password that works for a given account requires a preimage attack, and access to the hash of the original password (typically in the shadow file) which may or may not be trivial.
At the Rump Session of CRYPTO 2006, Christian Rechberger and Christophe De Cannière claimed to have discovered a collision attack on SHA-1 that would allow an attacker to select at least parts of the message [14][15].
[edit] Longer variants
NIST has published four additional hash functions in the SHA family, each with longer digests, collectively known as SHA-2. The individual variants are named after their digest lengths (in bits): "SHA-256", "SHA-384", and "SHA-512". They were first published in 2001 in the draft FIPS PUB 180-2, at which time review and comment were accepted. FIPS PUB 180-2, which also includes SHA-1, was released as an official standard in 2002. In February 2004, a change notice was published for FIPS PUB 180-2, specifying an additional variant, "SHA-224", defined to match the key length of two-key Triple DES. These variants are patented in U.S. Patent 6,829,355 .
SHA-256 and SHA-512 are novel hash functions computed with 32- and 64-bit words, respectively. They use different shift amounts and additive constants, but their structures are otherwise virtually identical, differing only in the number of rounds. SHA-224 and SHA-384 are simply truncated versions of the first two, computed with different initial values.
These new hash functions have not received as much scrutiny by the public cryptographic community as SHA-1 has, and so their cryptographic security is not yet as well-established. Gilbert and Handschuh (2003) have studied the newer variants and found no weaknesses.
[edit] SHA sizes
Algorithm | Output size (bits) | Internal state size (bits) | Block size (bits) | Max message size (bits) | Word size (bits) | Passes | Operations | Collision |
---|---|---|---|---|---|---|---|---|
SHA-0 | 160 | 160 | 512 | 264 - 1 | 32 | 80 | +,and,or,xor,rotl | Yes |
SHA-1 | 160 | 160 | 512 | 264 - 1 | 32 | 80 | +,and,or,xor,rotl | With flaws |
SHA-256/224 | 256/224 | 256 | 512 | 264 - 1 | 32 | 64 | +,and,or,xor,shr,rotr | No |
SHA-512/384 | 512/384 | 512 | 1024 | 2128 - 1 | 64 | 80 | +,and,or,xor,shr,rotr | No |
Note: internal state here means the "internal hash sum" after each compression of a data block; see Merkle-Damgård construction for more details.
[edit] Applications
SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512 are the required secure hash algorithms for use in U.S. Federal applications, including use by other cryptographic algorithms and protocols, for the protection of sensitive unclassified information. FIPS PUB 180-1 also encouraged adoption and use of SHA-1 by private and commercial organizations.
A prime motivation for the publication of the Secure Hash Algorithm was the Digital Signature Standard, in which it is incorporated.
The SHA hash functions have been used as the basis for the SHACAL block ciphers.
The copy prevention system of Microsoft's Xbox game console relies on the security of SHA-1.
SHA-1 hashing has also been employed by many file sharing applications to link multiple sources for the same file, that may not have the same name, as well as to avoid matching nonidentical sources that may have the same name.
[edit] Example hashes
The following are some examples of SHA digests. ASCII encoding is assumed for all messages.
[edit] SHA1 hashes
SHA1("The quick brown fox jumps over the lazy dog") = 2fd4e1c6 7a2d28fc ed849ee1 bb76e739 1b93eb12
Even a small change in the message will, with overwhelming probability, result in a completely different hash due to the avalanche effect. For example, changing d to c:
SHA1("The quick brown fox jumps over the lazy cog") = de9f2c7f d25e1b3a fad3e85a 0bd17d9b 100db4b3
The hash of the zero-length message is:
SHA1("") = da39a3ee 5e6b4b0d 3255bfef 95601890 afd80709
[edit] SHA-256 hashes
SHA256("abc") = ba7816bf 8f01cfea 414140de 5dae2223 b00361a3 96177a9c b410ff61 f20015ad SHA256("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq") = 248d6a61 d20638b8 e5c02693 0c3e6039 a33ce459 64ff2167 f6ecedd4 19db06c1
SHA256("The quick brown fox jumps over the lazy dog") = d7a8fbb3 07d78094 69ca9abc b0082e4f 8d5651e4 6d3cdb76 2d02d0bf 37c9e592
//avalanche effect when changing the last word to "cog" SHA256("The quick brown fox jumps over the lazy cog") = e4c4d8f3 bf76b692 de791a17 3e053211 50f7a345 b46484fe 427f6acc 7ecc81be
The hash of the zero-length message is:
SHA256("") = e3b0c442 98fc1c14 9afbf4c8 996fb924 27ae41e4 649b934c a495991b 7852b855
[edit] Official validation
Implementations of all FIPS-approved security functions can be officially validated through the CMVP program, jointly run by the National Institute of Standards and Technology (NIST) and the Communications Security Establishment (CSE). For informal verification, a package to generate a high number of test vectors is made available for download on the NIST site; the resulting verification however does not replace in any way the formal CMVP validation, which is required by law for some usages. As of October 2006, there are more than 500 validated implementations of SHA-1, with less than ten of them capable of handling messages with a length in bits non-multiple of eight (see SHS Validation List). Note that some implementations available on the net do not digest the NIST validation vectors correctly, although they do pass the examples in the SHA-1 standard.
[edit] A description of SHA-1
Pseudocode for the SHA-1 algorithm follows:
Note: All variables are unsigned 32 bits and wrap modulo 232 when calculating Initialize variables: h0 := 0x67452301 h1 := 0xEFCDAB89 h2 := 0x98BADCFE h3 := 0x10325476 h4 := 0xC3D2E1F0 Pre-processing: append the bit '1' to the message append k bits '0', where k is the minimum number >= 0 such that the resulting message length (in bits) is congruent to 448 (mod 512) append length of message (before pre-processing), in bits, as 64-bit big-endian integer Process the message in successive 512-bit chunks: break message into 512-bit chunks for each chunk break chunk into sixteen 32-bit big-endian words w(i), 0 ≤ i ≤ 15 Extend the sixteen 32-bit words into eighty 32-bit words: for i from 16 to 79 w(i) := (w(i-3) xor w(i-8) xor w(i-14) xor w(i-16)) leftrotate 1 Initialize hash value for this chunk: a := h0 b := h1 c := h2 d := h3 e := h4 Main loop: for i from 0 to 79 if 0 ≤ i ≤ 19 then f := (b and c) or ((not b) and d) k := 0x5A827999 else if 20 ≤ i ≤ 39 f := b xor c xor d k := 0x6ED9EBA1 else if 40 ≤ i ≤ 59 f := (b and c) or (b and d) or (c and d) k := 0x8F1BBCDC else if 60 ≤ i ≤ 79 f := b xor c xor d k := 0xCA62C1D6 temp := (a leftrotate 5) + f + e + k + w(i) e := d d := c c := b leftrotate 30 b := a a := temp Add this chunk's hash to result so far: h0 := h0 + a h1 := h1 + b h2 := h2 + c h3 := h3 + d h4 := h4 + e Produce the final hash value (big-endian): digest = hash = h0 append h1 append h2 append h3 append h4
Note: Instead of the formulation from the original FIPS PUB 180-1 shown, the following equivalent expressions may be substituted into the appropriate ranges of the above pseudocode for improved efficiency:
(0 ≤ i ≤ 19): f := d xor (b and (c xor d)) (alternative) (20 ≤ i ≤ 39): f := b xor c xor d (unchanged) (40 ≤ i ≤ 59): f := (b and c) or (d and (b or c)) (alternative 1) (40 ≤ i ≤ 59): f := (b and c) + (d and (b xor c)) (alternative 2, due to Colin Plumb) (40 ≤ i ≤ 59): f := (b and c) or (d and (b xor c)) (alternative 3, due to Jeffrey Riaboy) (60 ≤ i ≤ 79): f := b xor c xor d (unchanged)
[edit] A description of SHA-2
Pseudocode for the SHA-256 algorithm follows. Note the great increase in mixing between bits of the words compared to SHA-1.
Note: All variables are unsigned 32 bits and wrap modulo 232 when calculating Initialize variables (first 32 bits of the fractional parts of the square roots of the first 8 primes 2..19): h0 := 0x6a09e667 h1 := 0xbb67ae85 h2 := 0x3c6ef372 h3 := 0xa54ff53a h4 := 0x510e527f h5 := 0x9b05688c h6 := 0x1f83d9ab h7 := 0x5be0cd19 Initialize table of round constants (first 32 bits of the fractional parts of the cube roots of the first 64 primes 2..311): k(0..63) := 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2 Pre-processing: append the bit '1' to the message append k bits '0', where k is the minimum number >= 0 such that the resulting message length (in bits) is congruent to 448 (mod 512) append length of message (before pre-processing), in bits, as 64-bit big-endian integer Process the message in successive 512-bit chunks: break message into 512-bit chunks for each chunk break chunk into sixteen 32-bit big-endian words w(i), 0 ≤ i ≤ 15 Extend the sixteen 32-bit words into sixty-four 32-bit words: for i from 16 to 63 s0 := (w(i-15) rightrotate 7) xor (w(i-15) rightrotate 18) xor (w(i-15) rightshift 3) s1 := (w(i-2) rightrotate 17) xor (w(i-2) rightrotate 19) xor (w(i-2) rightshift 10) w(i) := w(i-16) + s0 + w(i-7) + s1 Initialize hash value for this chunk: a := h0 b := h1 c := h2 d := h3 e := h4 f := h5 g := h6 h := h7 Main loop: for i from 0 to 63 s0 := (a rightrotate 2) xor (a rightrotate 13) xor (a rightrotate 22) maj := (a and b) xor (a and c) xor (b and c) t2 := s0 + maj s1 := (e rightrotate 6) xor (e rightrotate 11) xor (e rightrotate 25) ch := (e and f) xor ((not e) and g) t1 := h + s1 + ch + k(i) + w(i) h := g g := f f := e e := d + t1 d := c c := b b := a a := t1 + t2 Add this chunk's hash to result so far: h0 := h0 + a h1 := h1 + b h2 := h2 + c h3 := h3 + d h4 := h4 + e h5 := h5 + f h6 := h6 + g h7 := h7 + h Produce the final hash value (big-endian): digest = hash = h0 append h1 append h2 append h3 append h4 append h5 append h6 append h7
Note: The ch
and maj
functions can be optimized the same way as described for SHA-1.
SHA-224 is identical to SHA-256, except that:
- the initial variable values
h0
throughh7
are different, and - the output is constructed by omitting
h7
.
SHA-512 is identical in structure, but:
- all numbers are 64 bits long,
- there are 80 rounds instead of 64,
- the initial values and additive constants are extended to 64 bits, and
- the shift and rotate amounts used are different.
SHA-384 is identical to SHA-512, except that:
- the initial values
h0
through h7 are different, and - the output is constructed by omitting
h6
andh7
.
[edit] See also
[edit] References
- Eli Biham, Rafi Chen, Near-Collisions of SHA-0, Cryptology ePrint Archive, Report 2004/146, 2004 (to appear CRYPTO 2004) [16]
- Florent Chabaud, Antoine Joux: Differential Collisions in SHA-0. CRYPTO 1998. pp56–71
- Henri Gilbert, Helena Handschuh: Security Analysis of SHA-256 and Sisters. Selected Areas in Cryptography 2003: pp175–193
- Xiaoyun Wang, Hongbo Yu and Yiqun Lisa Yin, Efficient Collision Search Attacks on SHA-0, CRYPTO 2005 [17].
- Xiaoyun Wang, Yiqun Lisa Yin and Hongbo Yu, Finding Collisions in the Full SHA-1, CRYPTO 2005 [18].
[edit] External links
[edit] Description of the algorithms
- Proposed NIST SHS (SHA0)
- Specifications for a SECURE HASH STANDARD (SHS) (SHA0)
- RFC 3174, US Secure Hash Algorithm 1 (SHA1)
- RFC 4634, US Secure Hash Algorithms (SHA and HMAC-SHA)
- FIPS PUB 180-2, Secure Hash Standard (including SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512)
- The NIST Secure Hash Standard (including SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512)
[edit] Cryptanalysis
- Interview with Yiqun Lisa Yin concerning the attack on SHA-1
- Lenstra's Summary of impact of the February 2005 cryptanalytic results
- Explanation of the successful attacks on SHA-1 (3 pages, 2006)
[edit] Implementations
- Brian Gladman's implementation of SHA functions
- Fast software implementation of SHA-2
- LGPL-licensed SHA-1 source code written in C
- A Javascript SHA-1 calculator showing intermediate values
- Another SHA-1 implementation in JavaScript, UTF-8 support
Hash algorithms: Gost-Hash | HAS-160 | HAVAL | MDC-2 | MD2 | MD4 | MD5 | N-Hash | RIPEMD | SHA family | Snefru | Tiger | VEST | WHIRLPOOL | crypt(3) DES |
MAC algorithms: Data Authentication Code | CBC-MAC | HMAC | OMAC/CMAC | PMAC | UMAC | Poly1305-AES | VEST |
Authenticated encryption modes: CCM | EAX | GCM | OCB | VEST Attacks: Birthday attack | Collision attack | Preimage attack | Brute force attack |
Standardization: CRYPTREC | NESSIE Misc: Avalanche effect | Hash collision | Hash functions based on block ciphers |
History of cryptography | Cryptanalysis | Cryptography portal | Topics in cryptography |
Symmetric-key algorithm | Block cipher | Stream cipher | Public-key cryptography | Cryptographic hash function | Message authentication code | Random numbers |