Gutmann method
Encyclopedia
The Gutmann method is an algorithm
for securely erasing
the contents of computer hard drives, such as file
s. Devised by Peter Gutmann
and Colin Plumb, it does so by writing a series of 35 pattern
s over the region to be erased.
The selection of patterns assumes that the user doesn't know the encoding mechanism used by the drive, and so includes patterns designed specifically for three different types of drives. A user who knows which type of encoding the drive uses can choose only those patterns intended for their drive. A drive with a different encoding mechanism would need different patterns.
Most of the patterns in the Gutmann method were designed for older MFM
/RLL
encoded disks. Relatively modern drives no longer use these older encoding techniques, making many of the patterns specified by Gutmann superfluous. Moreover, since about 2001, ATA IDE and SATA
hard drive manufacturer designs include support for the “Secure Erase” standard, obviating the need to apply the Gutmann method when erasing an entire drive.
The method was first presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory, in July 1996.
For example:
Analog signal: +11.1 -8.9 +9.1 -11.1 +10.9 -9.1
Ideal Digital signal: +10.0 -10.0 +10.0 -10.0 +10.0 -10.0
Difference: +1.1 +1.1 -0.9 -1.1 +0.9 +0.9
Previous signal: +11 +11 -9 -11 +9 +9
This can then be done again to see the previous data written:
Recovered signal: +11 +11 -9 -11 +9 +9
Ideal Digital signal: +10.0 +10.0 -10.0 -10.0 +10.0 +10.0
Difference: +1 +1 +1 -1 -1 -1
Previous signal: +10 +10 -10 -10 +10 +10
However, even when overwriting the disk repeatedly with random data it is theoretically possible to recover the previous signal. The permittivity
of a medium changes with the frequency of the magnetic field. This means that a lower frequency field will penetrate deeper into the magnetic material on the drive than a high frequency one. So a low frequency signal will, in theory still be detectable even after it has been overwritten hundreds of times by a high frequency signal.
The patterns used are designed to apply alternating magnetic fields of various frequencies and various phases to the drive surface and thereby approximate degaussing
the material below the surface of the drive.
Each of patterns 5 to 31 was designed with a specific magnetic media encoding
scheme in mind, which each pattern targets. The drive is written to for all the passes even though the table below only shows the bit patterns for the passes that are specifically targeted at each encoding scheme. The end result should obscure any data on the drive so that only the most advanced physical scanning (e.g. using a magnetic force microscope
) of the drive is likely to be able to recover any data.
The series of patterns is as follows:
Encoded bits shown in bold are what should be present in the ideal pattern, although due to the encoding the complementary bit is actually present at the start of the track.
to the file) without immediately removing any of its contents. At this point the file can be fairly easily recovered by numerous recovery applications. However, once the space is overwritten with other data, there is no known way to use software to recover it. It cannot be done with software alone since the storage device only returns its current contents via its normal interface. Gutmann claims that intelligence agencies
have sophisticated tools, including magnetic force microscope
s, which together with image analysis
, can detect the previous values of bit
s on the affected area of the media (for example hard disk
).
The National Bureau of Economic Research
criticized Gutmann's claim that intelligence agencies are likely to be able to read overwritten data. Published Government security procedures clearly consider an overwritten disk to still be sensitive.
Companies specializing in recovery
of damaged media (e.g., media damaged by fire, water or otherwise) cannot recover completely overwritten files. No private data recovery company currently claims that it can reconstruct completely overwritten data.
Gutmann himself has responded to some of these criticisms and also criticized how his algorithm has been abused in an epilogue to his original paper, in which he states :
Algorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
for securely erasing
Data remanence
Data remanence is the residual representation of data that remains even after attempts have been made to remove or erase the data. This residue may result from data being left intact by a nominal file deletion operation, by reformatting of storage media that does not remove data previously written...
the contents of computer hard drives, such as file
Computer file
A computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...
s. Devised by Peter Gutmann
Peter Gutmann (computer scientist)
Peter Gutmann is a computer scientist in the Department of Computer Science at the University of Auckland, Auckland, New Zealand. He has a Ph.D. in computer science from the University of Auckland. His Ph.D. thesis and a book based on the thesis were about a cryptographic security architecture...
and Colin Plumb, it does so by writing a series of 35 pattern
Pattern
A pattern, from the French patron, is a type of theme of recurring events or objects, sometimes referred to as elements of a set of objects.These elements repeat in a predictable manner...
s over the region to be erased.
The selection of patterns assumes that the user doesn't know the encoding mechanism used by the drive, and so includes patterns designed specifically for three different types of drives. A user who knows which type of encoding the drive uses can choose only those patterns intended for their drive. A drive with a different encoding mechanism would need different patterns.
Most of the patterns in the Gutmann method were designed for older MFM
Modified Frequency Modulation
Modified Frequency Modulation, commonly MFM, is a line coding scheme used to encode the actual data-bits on most floppy disk formats, hardware examples include Amiga, most CP/M machines as well as IBM PC compatibles. Early hard disk drives also used this coding.MFM is a modification to the original...
/RLL
Run Length Limited
Run length limited or RLL coding is a line coding technique that is used to send arbitrary data over a communications channel with bandwidth limits. This is used in both telecommunication and storage systems which move a medium past a fixed head. Specifically, RLL bounds the length of stretches ...
encoded disks. Relatively modern drives no longer use these older encoding techniques, making many of the patterns specified by Gutmann superfluous. Moreover, since about 2001, ATA IDE and SATA
Sata
Sata is a traditional dish from the Malaysian state of Terengganu, consisting of spiced fish meat wrapped in banana leaves and cooked on a grill.It is a type of Malaysian fish cake, or otak-otak...
hard drive manufacturer designs include support for the “Secure Erase” standard, obviating the need to apply the Gutmann method when erasing an entire drive.
The method was first presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory, in July 1996.
Technical overview
One standard way to recover data that has been overwritten on a hard drive is to capture and process the analog signal obtained from the drive's read/write head prior to this analog signal being digitized. This analog signal will be close to an ideal digital signal, but the differences will reveal important information. By calculating the ideal digital signal and then subtracting it from the actual analog signal, it is possible to amplify the signal remaining after subtraction and use it to determine what had previously been written on the disk.For example:
Analog signal: +11.1 -8.9 +9.1 -11.1 +10.9 -9.1
Ideal Digital signal: +10.0 -10.0 +10.0 -10.0 +10.0 -10.0
Difference: +1.1 +1.1 -0.9 -1.1 +0.9 +0.9
Previous signal: +11 +11 -9 -11 +9 +9
This can then be done again to see the previous data written:
Recovered signal: +11 +11 -9 -11 +9 +9
Ideal Digital signal: +10.0 +10.0 -10.0 -10.0 +10.0 +10.0
Difference: +1 +1 +1 -1 -1 -1
Previous signal: +10 +10 -10 -10 +10 +10
However, even when overwriting the disk repeatedly with random data it is theoretically possible to recover the previous signal. The permittivity
Permittivity
In electromagnetism, absolute permittivity is the measure of the resistance that is encountered when forming an electric field in a medium. In other words, permittivity is a measure of how an electric field affects, and is affected by, a dielectric medium. The permittivity of a medium describes how...
of a medium changes with the frequency of the magnetic field. This means that a lower frequency field will penetrate deeper into the magnetic material on the drive than a high frequency one. So a low frequency signal will, in theory still be detectable even after it has been overwritten hundreds of times by a high frequency signal.
The patterns used are designed to apply alternating magnetic fields of various frequencies and various phases to the drive surface and thereby approximate degaussing
Degaussing
Degaussing is the process of decreasing or eliminating an unwanted magnetic field. It is named after Carl Friedrich Gauss, an early researcher in the field of magnetism...
the material below the surface of the drive.
Method
An overwrite session consists of a lead-in of four random write patterns, followed by patterns 5 to 31 (see rows of table below), executed in a random order, and a lead-out of four more random patterns.Each of patterns 5 to 31 was designed with a specific magnetic media encoding
Code
A code is a rule for converting a piece of information into another form or representation , not necessarily of the same type....
scheme in mind, which each pattern targets. The drive is written to for all the passes even though the table below only shows the bit patterns for the passes that are specifically targeted at each encoding scheme. The end result should obscure any data on the drive so that only the most advanced physical scanning (e.g. using a magnetic force microscope
Magnetic force microscope
Magnetic force microscope is a variety of atomic force microscope, where a sharp magnetized tip scans a magnetic sample; the tip-sample magnetic interactions are detected and used to reconstruct the magnetic structure of the sample surface. Many kinds of magnetic interactions are measured by MFM,...
) of the drive is likely to be able to recover any data.
The series of patterns is as follows:
| Pass | Data Written | Pattern written to disk for targeted encoding scheme | |||
|---|---|---|---|---|---|
| In Binary Binary numeral system The binary numeral system, or base-2 number system, represents numeric values using two symbols, 0 and 1. More specifically, the usual base-2 system is a positional notation with a radix of 2... notation |
In Hex Hexadecimal In mathematics and computer science, hexadecimal is a positional numeral system with a radix, or base, of 16. It uses sixteen distinct symbols, most often the symbols 0–9 to represent values zero to nine, and A, B, C, D, E, F to represent values ten to fifteen... notation |
(1,7) RLL Run Length Limited Run length limited or RLL coding is a line coding technique that is used to send arbitrary data over a communications channel with bandwidth limits. This is used in both telecommunication and storage systems which move a medium past a fixed head. Specifically, RLL bounds the length of stretches ... |
(2,7) RLL Run Length Limited Run length limited or RLL coding is a line coding technique that is used to send arbitrary data over a communications channel with bandwidth limits. This is used in both telecommunication and storage systems which move a medium past a fixed head. Specifically, RLL bounds the length of stretches ... |
MFM Modified Frequency Modulation Modified Frequency Modulation, commonly MFM, is a line coding scheme used to encode the actual data-bits on most floppy disk formats, hardware examples include Amiga, most CP/M machines as well as IBM PC compatibles. Early hard disk drives also used this coding.MFM is a modification to the original... |
|
| 1 | (Random) | (Random) | |||
| 2 | (Random) | (Random) | |||
| 3 | (Random) | (Random) | |||
| 4 | (Random) | (Random) | |||
| 5 | 01010101 01010101 01010101 | 55 55 55 | 100... | 000 1000... | |
| 6 | 10101010 10101010 10101010 | AA AA AA | 00 100... | 0 1000... | |
| 7 | 10010010 01001001 00100100 | 92 49 24 | 00 100000... | 0 100... | |
| 8 | 01001001 00100100 10010010 | 49 24 92 | 0000 100000... | 100 100... | |
| 9 | 00100100 10010010 01001001 | 24 92 49 | 100000... | 00 100... | |
| 10 | 00000000 00000000 00000000 | 00 00 00 | 101000... | 1000... | |
| 11 | 00010001 00010001 00010001 | 11 11 11 | 0 100000... | ||
| 12 | 00100010 00100010 00100010 | 22 22 22 | 00000 100000... | ||
| 13 | 00110011 00110011 00110011 | 33 33 33 | 10... | 1000000... | |
| 14 | 01000100 01000100 01000100 | 44 44 44 | 000 100000... | ||
| 15 | 01010101 01010101 01010101 | 55 55 55 | 100... | 000 1000... | |
| 16 | 01100110 01100110 01100110 | 66 66 66 | 0000 100000... | 000000 10000000... | |
| 17 | 01110111 01110111 01110111 | 77 77 77 | 100010... | ||
| 18 | 10001000 10001000 10001000 | 88 88 88 | 00 100000... | ||
| 19 | 10011001 10011001 10011001 | 99 99 99 | 0 100000... | 00 10000000... | |
| 20 | 10101010 10101010 10101010 | AA AA AA | 00 100... | 0 1000... | |
| 21 | 10111011 10111011 10111011 | BB BB BB | 00 101000... | ||
| 22 | 11001100 11001100 11001100 | CC CC CC | 0 10... | 0000 10000000... | |
| 23 | 11011101 11011101 11011101 | DD DD DD | 0 101000... | ||
| 24 | 11101110 11101110 11101110 | EE EE EE | 0 100010... | ||
| 25 | 11111111 11111111 11111111 | FF FF FF | 0 100... | 000 100000... | |
| 26 | 10010010 01001001 00100100 | 92 49 24 | 00 100000... | 0 100... | |
| 27 | 01001001 00100100 10010010 | 49 24 92 | 0000 100000... | 100 100... | |
| 28 | 00100100 10010010 01001001 | 24 92 49 | 100000... | 00 100... | |
| 29 | 01101101 10110110 11011011 | 6D B6 DB | 0 100... | ||
| 30 | 10110110 11011011 01101101 | B6 DB 6D | 100... | ||
| 31 | 11011011 01101101 10110110 | DB 6D B6 | 00 100... | ||
| 32 | (Random) | (Random) | |||
| 33 | (Random) | (Random) | |||
| 34 | (Random) | (Random) | |||
| 35 | (Random) | (Random) |
Encoded bits shown in bold are what should be present in the ideal pattern, although due to the encoding the complementary bit is actually present at the start of the track.
Criticism
The delete function in most operating systems simply marks the space occupied by the file as reusable (removes the pointerData pointer
In computer science, a pointer is a programming language data type whose value refers directly to another value stored elsewhere in the computer memory using its address...
to the file) without immediately removing any of its contents. At this point the file can be fairly easily recovered by numerous recovery applications. However, once the space is overwritten with other data, there is no known way to use software to recover it. It cannot be done with software alone since the storage device only returns its current contents via its normal interface. Gutmann claims that intelligence agencies
Intelligence agency
An intelligence agency is a governmental agency that is devoted to information gathering for purposes of national security and defence. Means of information gathering may include espionage, communication interception, cryptanalysis, cooperation with other institutions, and evaluation of public...
have sophisticated tools, including magnetic force microscope
Magnetic force microscope
Magnetic force microscope is a variety of atomic force microscope, where a sharp magnetized tip scans a magnetic sample; the tip-sample magnetic interactions are detected and used to reconstruct the magnetic structure of the sample surface. Many kinds of magnetic interactions are measured by MFM,...
s, which together with image analysis
Image analysis
Image analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques...
, can detect the previous values of bit
Bit
A bit is the basic unit of information in computing and telecommunications; it is the amount of information stored by a digital device or other physical system that exists in one of two possible distinct states...
s on the affected area of the media (for example hard disk
Hard disk
A hard disk drive is a non-volatile, random access digital magnetic data storage device. It features rotating rigid platters on a motor-driven spindle within a protective enclosure. Data is magnetically read from and written to the platter by read/write heads that float on a film of air above the...
).
The National Bureau of Economic Research
National Bureau of Economic Research
The National Bureau of Economic Research is an American private nonprofit research organization "committed to undertaking and disseminating unbiased economic research among public policymakers, business professionals, and the academic community." The NBER is well known for providing start and end...
criticized Gutmann's claim that intelligence agencies are likely to be able to read overwritten data. Published Government security procedures clearly consider an overwritten disk to still be sensitive.
Companies specializing in recovery
Data recovery
Data recovery is the process of salvaging data from damaged, failed, corrupted, or inaccessible secondary storage media when it cannot be accessed normally. Often the data are being salvaged from storage media such as internal or external hard disk drives, solid-state drives , USB flash drive,...
of damaged media (e.g., media damaged by fire, water or otherwise) cannot recover completely overwritten files. No private data recovery company currently claims that it can reconstruct completely overwritten data.
Gutmann himself has responded to some of these criticisms and also criticized how his algorithm has been abused in an epilogue to his original paper, in which he states :
Software implementations
- CCleanerCCleanerCCleaner , developed by Piriform is a Utility program used to clean potentially unwanted files and invalid Windows Registry entries from a computer...
and RecuvaRecuvaRecuva is a freeware data recovery program, developed by Piriform, and runs under Microsoft Windows 7, Vista, XP, 2003, and 2000. It is able to recover files that have been "permanently" deleted and marked by the operating system as free space...
, utilities developed by Piriform - Darik's Boot and NukeDarik's Boot and NukeDarik's Boot and Nuke is an open source project hosted on SourceForge. The program is designed to securely erase a hard disk until data is permanently removed and no longer recoverable, which is achieved by overwriting the data with random numbers generated by Mersenne twister or ISAAC...
(DBAN) (whole disk only) - Disk UtilityDisk UtilityDisk Utility is the name of a utility created by Apple for performing disk-related tasks in Mac OS X. These tasks include:*the creation, conversion, compression and encryption of disk images from a wide range of formats read by Disk Utility to .dmg or—for CD/DVD images—.cdr, which is identical to...
a program provided with Mac OS XMac OS XMac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
(whole disk or free space only) - FreeOTFEFreeOTFEFreeOTFE is an open source on-the-fly disk encryption computer program for PCs running Microsoft Windows, and personal digital assistants running Windows Mobile . It creates virtual drives, or disks, to which anything written is automatically encrypted before being stored on a computer's hard or...
and FreeOTFE Explorer (disk encryption system) - LavasoftLavasoftLavasoft is a software development company that produces anti‐spyware software, including Ad‐Aware.The company offers a free, downloadable version of Ad‐Aware titled Ad‐Aware Free Internet Security and three commercial versions called Ad-Aware Total Security, Ad‐Aware Pro Internet Security, and...
Privacy Toolbox - PeaZipPeaZipPeaZip is a file manager and file archiver for Microsoft Windows and GNU/Linux. It supports its native PEA archive format and other mainstream formats, with special focus on handling open formats...
Secure delete function (files/directories only) - shredShred (Unix)shred is a Unix command that can be used to securely delete files and devices so that they can be recovered only with great difficulty with specialised hardware, if at all. It is a part of GNU Core Utilities.-Background:...
program of the GNU Core UtilitiesGNU Core UtilitiesThe GNU Core Utilities or coreutils is a package of GNU software containing many of the basic tools, such as cat, ls, and rm, needed for Unix-like operating systems...
- SlimCleaner developed by SlimWare UtilitiesSlimWare UtilitiesSlimWare Utilities is an American information technologies company that produces cleaning and optimization programs for an international market. The products they produce center on using crowdsourced feedback to generate real time reviews and quality evaluations of other programs.-History:SlimWare...
- srm, also used by Mac OS XMac OS XMac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
- TrueCryptTrueCryptTrueCrypt is a software application used for on-the-fly encryption . It is free and open source. It can create a virtual encrypted disk within a file or encrypt a partition or the entire storage device .- Operating systems :TrueCrypt supports Microsoft Windows, Mac OS X, and...
(disk encryption system) (free space only) - AshampooAshampooAshampoo GmbH & Co. KG is an international computer software company based in Oldenburg, Germany. They are best known for Ashampoo Burning Studio and Ashampoo WinOptimizer. They are general members of the Blu-ray Disc Association and a member of the Intel Software Partner...
HDD Control 2 - Heidi Eraser Windows explorer integration for files and free space (Open source)
- TuneUp UtilitiesTuneUp UtilitiesTuneUp Utilities is a utility software suite for Microsoft Windows designed to help manage, maintain, optimize, configure and troubleshoot a computer system. It is produced and developed by TuneUp Software GmbH, headquartered in Darmstadt, Germany and co-founded by Tibor Schiemann and Christoph...
External links
- Secure Deletion of Data from Magnetic and Solid-State Memory, Gutmann's original paper
- Recovering Unrecoverable Data, the need for drive-independent data recovery.
- A Guide to Understanding Data Remanence in Automated Information Systems
- Clearing and Declassifying Electronic Data Storage Devices
- Secure Erase utility from the Center for Magnetic Recording Research at the University of California, San Diego
- Reliably Erasing Data From Flash-Based Solid State Drives