Data remanence

From Wikipedia, the free encyclopedia

You have new messages (last change).
It has been suggested that Shredding be merged into this article or section. (Discuss)

Data remanence is the residual physical representation of data that has been in some way erased. After storage media is erased there may be some physical characteristics that allow data to be reconstructed. As early as 1960 the problem caused by the retentive properties of computer storage media was recognized. It was known that without the application of data removal procedures, inadvertent disclosure of sensitive information was possible should the storage media be released into an uncontrolled environment. Degaussing, overwriting, data encryption, and media destruction are some of the methods that have been employed to safeguard against disclosure of sensitive information. Over a period of time, certain practices have been accepted for the clearing and purging of storage media.

Contents

[edit] The problem

When most computers delete a file, they do not actually remove the contents of the file. Instead, they simply unlink the file from the file directory system, leaving the contents of the file in the disk sectors. This data will remain there until the operating system reuses those sectors to write new data. Until the old data is overwritten (and this may take months or longer), it can be recovered by programs that read disk sectors directly, such as forensic software (so called because it is used to obtain evidence in criminal investigations and also in legal discovery).

In addition even if a sector is overwritten the phenomenon of data remanence can make deleted data forensically recoverable.

In order to be sure that a deleted file is really deleted, it is necessary to overwrite the data sectors of that file. This process is not simply “erasing” or “formatting” the drives; this is not sufficient, as there are numerous tools available to recover “lost” data on disk drives.

[edit] Clearing

Clearing is the removal of sensitive data from storage devices in such a way that there is assurance, proportional to the sensitivity of the data, that the data may not be reconstructed using normal system capabilities, i.e., through the keyboard. (This may include use of data recovery utilities and advanced diagnostic routines.)

Clearing can be used when the secured physical environment (where the media was used) is maintained. In other words, the media is reused within the same computer and environment previously used.

In an operational computer, clearing can usually be accomplished by an overwrite of unassigned system storage space, provided the system can be trusted to provide separation of the storage space and unauthorized users. For example, a single overwrite of a file or all system storage, if the circumstance warrants such an action, is adequate to ensure that previous information cannot be reconstructed through a keyboard attack, provided the system can be trusted to provide separation of system resources and unauthorized users. Software used for clearing should be under strict configuration controls.

Simply removing pointers to a file, which is all that occurs when a file is deleted in most operating systems, will not generally render the previous information unrecoverable through normal system capabilities. Likewise, reformatting, repartitioning, reghosting or reimagining a system is not guaranteed to write to every area of the disc, even though it will cause the disc to appear as empty to most programs.

[edit] Purging

Purging is the removal of sensitive data from a system or storage device in such a way that there is assurance, proportional to the sensitivity of the data, that the data may not be reconstructed through open-ended laboratory techniques. A computer must be disconnected from any external network before a purge. Purging must be used when the secured physical environment (where the media was used) will not be maintained. In other words, media scheduled to be released from a secure facility to a non-secure facility or environment should be purged.

The United States Department of Defense (DoD) has approved both overwriting and degaussing for purging data, although the effectiveness of overwriting cannot be guaranteed without examining each specific situation. DoD documents like NISPOM often refer to purging as "sanitization".[1]

[edit] Software for purging

To purge the AIS storage media, the DoD requires overwriting multiple times in a prescribed pattern. The number of times an overwrite must be accomplished depends on the storage media, sometimes on its sensitivity, and sometimes on differing DoD component requirements. Software developers must design the software such that the software continues to write to all addressable locations on the media, in spite of intermediate errors. All such errors in usable sectors should be reported with a listing of current content. Unusable sectors must be completely overwritten, because the unusable sector list will not show whether the sector ever contained any sensitive data. If any errors occur while overwriting or if any unusable sector could not be overwritten, then degaussing is required.

Notice that NISPOM requires that Top Secret information be purged by physical destruction or degaussing and not by software rewrites. [1]

There are additional risks to trusting overwrite software to purge disks. The environment in which the software must operate is difficult to constrain. For this reason, care must be exercised during software development to ensure the software cannot be subverted. The overwrite software should be protected at the level of the media it purges, and strict configuration controls should be in place on both the operating system the software must run under and the software itself. The overwrite software must be protected from unauthorized modification.

Despite its issues, software-based data destruction methods are inexpensive and easy to use for the average computer user. Darik's Boot and Nuke is an open source, GPL-licensed data destruction program that fits on a single 3.5 inch floppy disk. Its programmers claim that, when used on boot media, the program can reliably destroy all data on IDE and SCSI hard drives. This is intended to defeat most forensic data recovery methods and bring the program into compliance with United States Department of Defense and Royal Canadian Mounted Police standards, among others.

[edit] Standard patterns for purging

  • For non Top Secret information, NISPOM prescribes overwriting data in 3 passes: with a character, then its complement, and finally with a random character; e.g., overwrite first with 0000 0000, followed by 1111 1111, then 1001 0111. [1]
  • NAVSO P5239-26 standards are more rigorous versions of the same procedure.
  • DoD 5220.22-M prescribes overwriting data in 7 passes.
  • The Royal Canadian Mounted Police(RCMP)[2] recommends a triple overwrite for confidential data, and that drives containing Top Secret data be passed through a commercial disintegrator having a ¼ inch residue screen.
  • Gutmann [3] suggested overwriting data would not necessarily have made it harder for laboratories, using a type of scanning electron microscopes say, to recover data from magnetic storage media. Gutmann has since said that people have misunderstood the Gutmann_method, and that 35 overwrites is pointless.

[edit] The bad track problem

A compromise of sensitive data may occur if media is released when an addressable segment of a storage device (such as unusable or "bad" tracks in a disk drive or inter-record gaps in tapes) is not receptive to an overwrite. As an example, a disk platter may develop unusable tracks or sectors; however, sensitive data may have been previously recorded in these areas. It may be difficult to overwrite these unusable tracks. Before sensitive information is written to a disk, all unusable tracks, sectors, or blocks should be identified (mapped). During the life cycle of a disk, additional unusable areas may be identified. If this occurs and these tracks cannot be overwritten, then sensitive information may remain on these tracks. In this case, overwriting is not an acceptable purging method and the media should be degaussed or destroyed.

[edit] Degaussing

Degaussing is a process whereby the magnetic media is erased. Degaussing requires a degausser device that is designed and approved for the type of media being purged. The U.S. General Services Administration maintains a list of approved degaussers.

Degaussing often renders hard drives inoperable. This can prevent computers from being recycled, say for educational use. The sensitivity of the data stored on the computer and the feasibility of software purging should be weighed before degaussing hard drives.

The DoD has approved overwriting for clearing, but not purging, magnetic floppy disks. Degaussing is the preferred method. Degaussed floppy disks can generally be reformatted and reused.

[edit] Disk encryption

Several software products, including Apple's Mac OS X and PGP, can encrypt all data before it is stored on a hard disk or other storage medium. If enabled from the time when the computer is first purchased or first used for sensitive information, disk encryption using a sufficiently secure method can alleviate the need for degaussing and destruction.

[edit] Systemic problems with clearing and purging in practical systems

Disk storage systems in modern computers systems are both buffered at the disk and within typical operating systems. It is not generally possible to have any assurance that a particular sector on disk has actually been written, as the data may have been buffered at the disk (or within the operating system) and not written prior to a power down or reset. In addition, some disk management software is intelligent and may notice that the same sector is being re-written several times, and simply cancel all but the last.

Despite overwrite of a disk sector, it has been shown that, with sufficient care and resources, the chance of recovering some or all of the supposedly erased and overwritten data may be possible. Peter Gutmann[3] of the University of Auckland investigated this possibility in the mid-1990s and his paper recounting his results contains surprising results. However, there have been several rebuttals to Gutmann's paper. Daniel Feenberg, National Bureau of Economic Research writes that the chances of overwritten data being recovered from a modern hard drive are quite remote[4]. Furthermore, no private data recovery service is currently capable of reconstructing overwritten data. Electron microscopy makes recovery possible, but the process would likely be both prohibitively expensive and far from a sure thing.

In addition, a similar specter of recoverability has been observed in RAM, and it is therefore, generally, not possible to assume that removing the power from volatile RAM will always render the data stored in it unrecoverable. There are slow memory biasing mechanisms in some RAM circuits, usually connected to charge migration in semiconductor structures, which can retain data across such power cycling.

When the data being temporarily stored is of an extremely sensitive nature, such as cryptographic keys, considerable care is required, and must be based on the particular characteristics of the operating system, RAM, and long term storage in use. There is no universal solution, and for a particular system, there may be no solution at all which ensures data is unrecoverable.

A practical solution would require, at minimum: operating systems which guarantee that a particular system call will securely erase and overwrite a specific block of memory; storage devices which will guarantee that specific data will be committed to storage without optimization; and an interface which controls that mechanism. There are no readily available operating systems nor storage devices which provide such facilities.

[edit] CDs, DVDs, etc.

Optical media are not magnetic and cannot be erased by degaussing. Write-once media, such as CD-ROM, CD-R, DVD-R, etc., cannot be purged by software or a degausser. They must be destroyed. Read/write optical media, such as CD-RW and DVD-RW can be cleared by overwriting under software control. It is not known if such software purging is effective, and in any case, it would be a lengthy process. Destruction is usually the best approach. Some shredding machinery, even inexpensive commercial ones, can do so.

[edit] Flash memory devices

Data stored on devices that use flash memory, such as USB flash drives and memory cards, can often be recovered even after it has been erased. No generally accepted method for disposing of these devices seems to be available. Data tends to "burn in" the longer it is stored. Overwriting with random data may be superior to erasing, especially if it can be done several times, each a week or more apart. However, this will reduce the device's lifetime. Since these units are so small, secure storage until the original data is no longer sensitive may be the simplest approach.

[edit] Destruction

Data destruction can be contracted out
Data destruction can be contracted out

It is good practice to purge media before submitting it for destruction. Media may generally be destroyed by one of the following methods:

  • Destruction at an approved metal destruction facility, i.e., smelting, disintegration, or pulverization.
  • Incineration.
  • Application of corrosive chemicals, such as acids, to recording surfaces.
  • Application of an abrasive substance (emery wheel or disk sander) to a magnetic disk or drum recording surface. Make certain that the entire recording surface is completely removed before disposal. Also, ensure proper protection from inhaling the abraded dust.

[edit] See also

[edit] Standards

[edit] Software

[edit] References

  1. ^ a b c DoD 5220.22-M NISP Operating Manual (NISPOM), unofficial PDF, Section 8-306, Clearing and Sanitization Matrix
  2. ^ Royal Canadian Mounted Police, "Hard Drive Secure Information Removal and Destruction Guidelines", October 2003(PDF)
  3. ^ a b Peter Gutmann, Secure Deletion of Data from Magnetic and Solid-State
  4. ^ Daniel Feenberg, Can Intelligence Agencies Recover Overwritten Data?
Retrieved from "http://en.wikipedia.org../../../d/a/t/Data_remanence.html"