Storage Efficiency
Encyclopedia
Storage efficiency is the ability to store and manage data that consumes the least amount of space with little to no impact on performance; resulting in a lower total operational cost. Efficiency addresses the real-world demands of managing costs, reducing complexity and limiting risk. The Storage Industry Networking Association (SNIA
) defines storage efficiency in the SNIA Dictionary
as follows:
The efficiency of an empty enterprise level system is commonly in the 40%–70% range, depending on what combination of RAID
, mirroring and other data protection technologies are deployed, and may be even lower for highly redundant remotely mirrored systems. As data is stored on the system, technologies such as deduplication and compression may store data at a greater than 1-to-1 data size-to-space consumed ratio, and efficiency rises, often to over 100% for primary data, and thousands of percent for backup data.
Snapshot
technology—known formally as "delta snapshot technology"—gives the ability to use the same dataset multiple times for multiple reasons, while storing only the changes between each dataset. Some storage vendors integrate their snapshot capabilities at the operating system and/or application level, enabling access to the data the snapshot's are holding at the system and/or application management layers. Terminology around snapshots and "clones" is currently confusing, and care must be taken when evaluating vendor claims. In particular, some vendors call full point-in-time copies "snapshots" or "clones", while others use the same terms to refer to shared-block "delta" snapshots or clones. And some implementations can only do read-only snapshots, while others are able to provide writable ones as well.
Data deduplication
technology can be used to very efficiently track and remove duplicate blocks of data inside a storage unit. There are a multitude of implementations, each with their separate advantages and disadvantages. Deduplication is most efficient at the shared storage layer, however, implementations in software and even databases exist. The most suitable candidates for deduplication are backup
and platform virtualization, because both applications typically produce or use a lot of almost identical copies. However, some vendors are now offering in-place deduplication, which deduplicates primary storage.
Thin provisioning
technology. This is a technique to prevent under-utilization by sharing the allocated, but not yet utilized capacity. A good example is Gmail
, where every Gmail account has a large amount of allocated capacity. Because most Gmail users only use a fraction of the allocated capacity, this "free space" is "shared" among all Gmail users.
Backup and restore. Using snapshots, time used for both backup and restore RTO
can be minimized. This can greatly reduce cost, and reduce hours of downtime to seconds of downtime. Snapshots also allow for better RPO
values.
Reducing floorspace. When less storage is required to store a given amount of data, less data center floorspace is required.
Reducing energy use. When fewer spindles are required to store a given amount of data, less power is required.
Provisioning efficiency. Writable delta snapshot technology allows for very fast provisioning of writable data copies. This reduces waiting time in processes that require that data. Examples are data mining
, test data
, etc. Snapshot integration at the OS
and/or application level also leads to faster provisioning, because system and/or application managers are able to manage their own snapshots without having to wait for storage managers and/or provisioning procedures.
, HDS
, IBM
and HP.
Storage Networking Industry Association
An association of producers and consumers of storage networking products, whose goal is to further storage networking technology and applications.The Storage Networking Industry Association, or SNIA, was incorporated in December, 1997, and is a registered 501 non-profit trade association...
) defines storage efficiency in the SNIA Dictionary
SNIA Dictionary
The SNIA Dictionary is a special-purpose dictionary containing terms and definitions applicable to the storage and storage networking industry. Published bi-annually by the Storage Networking Industry Association , the SNIA Dictionary is owned and managed by the SNIA Technical Council, a group of...
as follows:
The efficiency of an empty enterprise level system is commonly in the 40%–70% range, depending on what combination of RAID
RAID
RAID is a storage technology that combines multiple disk drive components into a logical unit...
, mirroring and other data protection technologies are deployed, and may be even lower for highly redundant remotely mirrored systems. As data is stored on the system, technologies such as deduplication and compression may store data at a greater than 1-to-1 data size-to-space consumed ratio, and efficiency rises, often to over 100% for primary data, and thousands of percent for backup data.
Technologies
Different technologies exist at different and sometimes multiple levels:Snapshot
Snapshot (computer storage)
In computer systems, a snapshot is the state of a system at a particular point in time. The term was coined as an analogy to that in photography. It can refer to an actual copy of the state of a system or to a capability provided by certain systems....
technology—known formally as "delta snapshot technology"—gives the ability to use the same dataset multiple times for multiple reasons, while storing only the changes between each dataset. Some storage vendors integrate their snapshot capabilities at the operating system and/or application level, enabling access to the data the snapshot's are holding at the system and/or application management layers. Terminology around snapshots and "clones" is currently confusing, and care must be taken when evaluating vendor claims. In particular, some vendors call full point-in-time copies "snapshots" or "clones", while others use the same terms to refer to shared-block "delta" snapshots or clones. And some implementations can only do read-only snapshots, while others are able to provide writable ones as well.
Data deduplication
Data deduplication
In computing, data deduplication is a specialized data compression technique for eliminating coarse-grained redundant data. The technique is used to improve storage utilization and can also be applied to network data transfers to reduce the number of bytes that must be sent across a link...
technology can be used to very efficiently track and remove duplicate blocks of data inside a storage unit. There are a multitude of implementations, each with their separate advantages and disadvantages. Deduplication is most efficient at the shared storage layer, however, implementations in software and even databases exist. The most suitable candidates for deduplication are backup
Backup
In information technology, a backup or the process of backing up is making copies of data which may be used to restore the original after a data loss event. The verb form is back up in two words, whereas the noun is backup....
and platform virtualization, because both applications typically produce or use a lot of almost identical copies. However, some vendors are now offering in-place deduplication, which deduplicates primary storage.
Thin provisioning
Thin Provisioning
Thin provisioning is the act of using virtualization technology to give the appearance of more physical resource than is actually available. If you always have enough resource to simultaneously support all of the virtualized resources then you are not thin provisioned...
technology. This is a technique to prevent under-utilization by sharing the allocated, but not yet utilized capacity. A good example is Gmail
Gmail
Gmail is a free, advertising-supported email service provided by Google. Users may access Gmail as secure webmail, as well via POP3 or IMAP protocols. Gmail was launched as an invitation-only beta release on April 1, 2004 and it became available to the general public on February 7, 2007, though...
, where every Gmail account has a large amount of allocated capacity. Because most Gmail users only use a fraction of the allocated capacity, this "free space" is "shared" among all Gmail users.
Major advantages
Actively increasing storage efficiency using these techniques has the following advantages:Backup and restore. Using snapshots, time used for both backup and restore RTO
Recovery Time Objective
The recovery time objective is the duration of time and a service level within which a business process must be restored after a disaster in order to avoid unacceptable consequences associated with a break in business continuity....
can be minimized. This can greatly reduce cost, and reduce hours of downtime to seconds of downtime. Snapshots also allow for better RPO
Recovery point objective
-Recovery point objective :When computers used for normal "production" business services are affected by a "Major Incident" that cannot be fixed quickly, then the Information Technology Service Continuity Plan is performed, by the ITSC recovery team...
values.
Reducing floorspace. When less storage is required to store a given amount of data, less data center floorspace is required.
Reducing energy use. When fewer spindles are required to store a given amount of data, less power is required.
Provisioning efficiency. Writable delta snapshot technology allows for very fast provisioning of writable data copies. This reduces waiting time in processes that require that data. Examples are data mining
Data mining
Data mining , a relatively young and interdisciplinary field of computer science is the process of discovering new patterns from large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics and database systems...
, test data
Test data
Test Data are data which have been specifically identified for use in tests, typically of a computer program.Some data may be used in a confirmatory way, typically to verify that a given set of input to a given function produces some expected result...
, etc. Snapshot integration at the OS
OS
OS may refer to:* O.S. Old Stonyhurst, an old boy of the ancient Jesuit public school, Stonyhurst College* O.S. Engines, a Japanese manufacturer of model aircraft engines* Ocean Science, an Oceanographic Journal published by the European Geosciences Union....
and/or application level also leads to faster provisioning, because system and/or application managers are able to manage their own snapshots without having to wait for storage managers and/or provisioning procedures.
Major commercial players
All major vendors are implementing one or more of these technologies, because storage efficiency is becoming more and more popular. Customers are facing exponentially growing storage requirements and a strong demand for cutting cost. The major vendors are NetApp, EMCEMC Corporation
EMC Corporation , a Financial Times Global 500, Fortune 500 and S&P 500 company, develops, delivers and supports information infrastructure and virtual infrastructure hardware, software, and services. EMC is headquartered in Hopkinton, Massachusetts, USA.Former Intel executive Richard Egan and his...
, HDS
Hitachi Data Systems
Hitachi Data Systems is a company providing mid-range and high-end storage systems, software and services. It is a wholly owned subsidiary of Hitachi Ltd. and part of the Hitachi Information Systems & Telecommunications Division....
, IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
and HP.