Virtualized Replication: vSphere APIs Expand

As a delegate for Tech Field Day 6 in Boston, I was introduced to SRM Replication as well as ZeRTO a third party replication tool. They seem to be as different as night and day but are they? Both work within the vSphere environment to replicate virtual disks regardless of storage type, and apparently hook into the same location within VMware’s API stack. This shows a maturity of VMware’s API stack that until now has been unknown and secret. In this one area, Microsoft Hyper-V is beating VMware vSphere: The availability of well known APIs that are easy for Third Parties to use. I now see a change in VMware’s behavior, can they continue this growth?The hook into the API stack for both is apparently at the same level as where vShield Endpoint Security hooks into the stack, using a virtual SCSI filter. Each component has their own filter. This shows a level of API maturity that was not there for quite some time.   These egress filters are employed as data leaves the VM over the virtual SCSI bus. Apparently these filters cannot only be unidirectional (as is the case for Endpoint security), but also multi-directional, to in essence re-read an entire disk array without the VM knowing such a re-read is happening.

This is a powerful set of tools that allows for live work loads to be replicated without relying on snapshots. In Figure 1, we see that the SCSI Filter is handled by a transport layer that talks to a SCSI Filter Driver which then communicates with the replication VM, in this case the ZeRTO product, which then uses standard network mechanisms to transport the data to another ZeRTO appliance.
Such a secondary appliance can live within the same cluster, datacenter, hybrid or public cloud. ZeRTO is the start of a very interesting STorage as a Service cloud environment. The question would be how the data is stored (encrypted) and transferred (encrypted) and who owns the keys to the data (the tenant).
This API has been around since vSphere 4 was announced and has matured to the point that it is now usable by non-antivirus vendors. While who can use this API is still quite limited it shows a growth in functionality of VMware’s vSphere APIs so that third parties can add their own functionality. It would be very interesting if such an API moved the way the vShield APIs have moved, away from requiring a third party driver, to requiring just a virtual appliance.
If the low-level vSphere APIs grew to be driver agnostic and were extremely well documented, then third parties could add in their own tools much easier for vSphere. In this one area Microsoft is far ahead of VMware as their APIs are very well documented and anyone can use them to hook their products and tools into Hyper-V and this is one reason the Hyper-V ecosystem is growing so fast. If VMware was to do the same thing, then more and more third parties could develop code and tools which would also increase the size of the ecosystem surrounding vSphere.