System scalability Beyond each approach's discovery technology, organizations must also consider the system's ability to scale. Different products support different levels of index scalability. This is important not only for system robustness (absolutely critical in any system storing one copy of data to serve numerous applications and users), but also because index scalability impacts the deduplication ratio.For a deduplication product to extract duplicate data, the duplicate data must be there to find. Primary application or even archive data rarely has the same level of native data duplication as backup. Hence, one deduplication approach does not fit all. A deduplication approach which is more "weighty" in resource usage may be valuable in backup, but it may make no sense to use it on primary or archive data sets where the duplicate data simply doesn't exist. Rule No. 2: Price performance is important. As with any data management technology, data transfer and compute speed is important. This is particularly true when the deduplication technology is "in-line." In this case, the performance of the deduplication product must be fast enough not to throttle the backup process. Even with deduplication offerings that run "deferred", be sure the system delivers enough performance to assure that yesterday's backup data is stored, replicated off-site (if desired), extracted to tape (if desired) and deleted (by policy) before the next day's backup window. The system should be able to provide sufficient performance without the need for unique, high-cost proprietary hardware.
A system supporting a "block pool" of unique data only to five terabytes will need to store a duplicate string every time it crosses the 5TB boundary, while a system with a 140TB index won't store similar data until it hits 140 terabytes. If these two systems had exactly the same deduplication effectiveness, the more scalable system would still have a deduplication ratio 28 times higher and would store 1/28th the volume of data! This is direct savings to the bottom line.