Future Storage, Flash, and Cloud?

A while ago, I was listening to the GreyBeards on Storage podcast. One idea suggested by a guest is that application storage is going to end up in one of two forms. There will only be flash and cloud: specifically, flash in a mobile device and object storage in the cloud. I don’t buy that as the end state, but I do see flash and cloud as the two growth areas of storage. I am seeing flash as a performance tier close to consumption, and a separate persistent capacity tier. I am also hearing a lot more about the persistent tier being farther away, across town or across the country. At recent Tech Field Day events, I saw a couple of companies that are putting flash storage close to users and persistent storage farther away. ClearSky provides a managed primary storage service, and Avere wants to alleviate your NAS performance problems.

I first saw the idea of separating performance from capacity at an early Tech Field Day event. Since I’m talking about Tech Field Day events, I’d better refer you to my standard TFD disclosure page. Infinio has launched its RAM-based storage caching for vSphere. Infinio uses a VM on each ESXi server to provide this performance tier. You may be more familiar with PernixData. It launched with a flash-based storage cache for vSphere. Both Infinio and Pernix use a cache in the ESXi server to deliver better performance than the shared storage array could deliver. Both of these products are confined to a data center; both the capacity and performance tier are in the same location. Once you separate the performance from the capacity, the next question is how far can you physically separate them? For highly transactional workloads, you probably can’t get too far before latency of metadata access becomes an issue. With more conventional file system access, you can probably stretch a lot farther.

The ClearSky solution uses an appliance in your data center as the flash-based performance tier. The capacity tier lives in ClearSky’s data center in the same city. The data center is connected to the performance tier over a dedicated data circuit. Both tiers use flash for fast access; the capacity tier is simply farther away. Metro area network latency is typically lower than the seek latency for a hard disk. Keeping the capacity tier in the same city means that the worst-case latency is comparable to a hybrid array on-premises. The service is fully managed primary storage as a service. ClearSky also has some smart elements that use cloud-based object storage, including as a backup repository.

Avere is focused on file shares, providing a cache for better performance from your file server. By placing the cache close to the users, the file server can be in another location without too much performance impact. You could deploy Avere into each branch office and leave your file servers in HQ. Avere can also be an NAS front end to an object store, which is closer to the flash plus object model. Avere recently launched its own object store, based on SwiftStack, to use as the persistent tier.

Another product that does local cache and object storage is Dropbox. My files are available to me instantly from the SSD in my laptop. The local files are just a cache; if my laptop doesn’t survive its next five-foot drop onto concrete, then my files are still safe on the Dropbox cloud. Apple’s iCloud, Microsoft OneDrive, and Google Drive all work this way, with a local cache and a remote cloud store.

The issue I have with the mobile-flash-plus-object-in-the-cloud view of the future is that it ignores the processing that happens in data centers. Whether the data center is on-premises or cloud-based, there is still processing. Image thumbnails and video at different resolutions need to be created, not to mention the analytics and intelligence that are part of any business process. There will always be a need for applications in a data center, and those applications will need high-performance storage.

There is one element of the flash-plus-cloud prediction I agree with: there will soon be no place for a hard drive outside of data centers. The last hard drives I own are in my media server or USB backup devices. Both these uses are being superseded by cloud services. New Zealand finally has Netflix, so streaming video means my media is in the cloud. We also have a national fiber-to-the-home network, so I can use a cloud-based backup. The other killer is that SSD capacity is now sufficiently cheap to have all my photos and other documents on SSD, not to mention that my primary home lab is all SSD.