There’s no question that all-flash storage systems offer substantial performance advantages over hard drive-based systems. But all-flash systems can also cost quite a bit more, especially when considered in terms of raw capacity, or the capacity of the system before enabling deduplication and compression.
However, the lower latency and higher throughput of these systems can easily support compression and in-line deduplication and still outperform hard drive-based systems. As these technologies have improved, this kind of storage has gone from a specialty offered by a few vendors to nearly ubiquitous, with almost every storage vendor now offering all-flash. These systems often use off-the-shelf SSDs such as the Samsung PM863a and SM863a.
Assessing the Need for All-Flash Storage
Prospective customers must consider how much benefit they’re likely to gain from compression and in-line deduplication. These technologies vary in their effects based on the types of data they’re processing. Deduplication looks for multiple instances of the same pattern of data this might be anything from the same files in multiple different directories to repetitions of small amounts of data within multiple files, such as a logo or photo embedded in different documents.
When duplicates are identified, subsequent versions are replaced with a link to the first one. This can be processor-intensive, especially when performed within files rather than just with identical files.
Discover the Cost Advantages of SSDs
White Paper
Read this white paper to learn about the total cost of ownership advantages of using SSDs. Download Now
This process was originally done after data was written, and was called post-processing deduplication. In-line deduplication identifies and removes duplicate data as new files are written, in real time, which requires both intensive processing power and very fast storage. The benefits can vary: If a storage system is being used for virtual desktop infrastructure (VDI) or server virtualization, where there are dozens or hundreds of operating system images, most containing hundreds of megabytes of the same files, deduplication can provide huge savings in the total amount of storage used — one hundred Windows 10 VDI images will use only a little more space than one image.
On the other hand, something like compressed video files, which are already compressed and have little to nothing in common between files, won’t gain much savings from deduplication or compression. This is also true of encrypted files, which can include database files that are used to store data that’s actually duplicated often but can’t be deduplicated due to encryption of the contents.
Factoring in Guarantees
Because many vendors advertise all-flash storage capacities based on 3x-5x reductions from the effects of deduplication and compression, and the actual reductions can vary greatly based on the types of data being stored, vendors selling all-flash systems may offer tools to evaluate data in advance of purchase to determine how much reduction will be achieved based on the actual data in a customer’s data center. Other vendors will offer guarantees of total storage if a customer doesn’t get a usual amount of data reduction, the vendor will add additional capacity to the all-flash system to reach the agreed-upon effective capacity.
Is All-Flash Right for You?
While some research is necessary to get a good feel for how much of an effect compression and in-line deduplication will have on your data, the typical overall rate is 3x-5x, which means that an all-flash system with 10 terabytes of actual capacity can hold 30-50 TB of data after processing.
The cost for 10 TB of all-flash storage may be less than the cost for 50 TB of hard drive-based storage, which means that the extra performance of an all-flash system is essentially free. Most experienced admins are deeply suspicious of claims that seem too good to be true, and there are cases where the typical rates cannot be achieved, but there are also cases where effective capacity may be many times better than even 5x — and there are excellent tools to determine which might be the case for your data.
By effectively analyzing what type of storage system would work best for your enterprise’s data, you’ll be able to find methods that not only process data faster, but help ensure data isn’t taking up unnecessary space.
Check out our full line of SSDs to find the perfect storage solution to address your organization’s needs.
