Federated Clouds? Possible?

Mike DiPetrillo’s post entitled VMware is Building Clouds sparked some interesting thoughts and discussion about what it means to have federated clouds and how do you define such federation? Is federated required to make ‘cloud’ ubiquitous or are we already there? But is the discussion really about federated clouds or simplistic data object movement between the VMs or about cloud management?
When we talk about cloud data we often speak about moving virtual machines around, but is that the proper way to do this? Given that a VM could be 100s of gigabytes, could the movement between clouds happen fast? What about the runtime and security contexts that surround the data to be moved (whether a VM or not). Does this move with the data object or does it stay within the source environment. I have this question regardless of cloud type.First the comment that started the entire conversation from Simon Crosby:

I hear from almost every Enterprise and service provider that vCloud is not what they want. Why? A significant concern is that it locks you into a single vendor cloud model that is utterly undifferentiated yet appallingly expensive, nickel and dime-ing you on a per-VM basis for every possible feature. Project Redwood – a massive undertaking from my friend the Sheriff of Redwood Valley to build the perfect, yet perfectly mismatched cloud platform – really is deadwood.

And Mike DiPretrillo’s response:

There’s even companies like CloudSwitch that will help you move stuff around. Go ahead and try and use them to yank stuff back out of the cloud to your own datacenter. Don’t mind the fact that now you have yet another tool you have to go use to find your VMs and manage them and that none of your other management tools talk through that interface. You’re starting to get my point. VMware is still an open system. The vCloud API is in place and allows anyone to write tools to talk to these vClouds running around the world and many people have those tools written today. Go use the tool that makes sense for you. You also get the choice between a lot of different cloud providers and can move freely between them. Is one charging you too much or doesn’t offer services that you want or in the location you want? Go use another one. Just drag and drop in vCenter Cloud Connector and you’re over to the new cloud in a flash. That’s federation. That’s open. That’s choice.

Okay I do agree that the vCloud APIs are a fairly open set of APIs and promotes movement of data objects (namely VMs) between VMware vSphere-based clouds that actually use the vCloud Director. But is this really open in the sense that it is an open standard supported by all vendors. This is not the case. A short excerpt of the Twitter conversation is to the right.
So as I read this, I have to ask is the Cloud ubiquitous? and at the moment I have to answer that it is not. Yes we have some mighty large clouds out there: Amazon, Verizon, AT&T, Azure, and Savvis. But can these all inter-operate with each other?
The answer to that is No. And that is why I do not think we are even close to having a ubiquitous cloud. Nor do I think we are even close to having an Open cloud infrastructure within the world. Mike points out that if all your ‘clouds’ were to run vCloud Director and therefore VMware vSphere, then we could approach a ubiquitous cloud. But that is what Simon was alluding to: vendor lock-in.
But Mike further points out that “Openness doesn’t mean you have to manage others.” I whole-heartedly agree with that statement, but I was not really asking about management but about data movement. Currently the smallest well-defined data object to move around the cloud is a VM. A VM is a very large conglomerate of operating system, application, application and customer data. At the moment, there is no accepted definition of what constitutes a VM. There is the DMTF OVF format but this is only accepted as a input element not as a runtime element. OVF does not also include application runtime, network, and security states as well as other much needed information to seemlessly move a VM from one cloud to another.
Did I just slip gears, not really. The real test of openness and  the ubiquitous nature of the cloud will not be how the data is managed nor how the data initially enters the cloud, but by how fast and effortlessly I can move data between disparate clouds. The problem now is that even with vCloud Director it is not easy to move data between clouds.  To make this work, we need to work towards the following:

• a common light-weight data container that also includes runtime, application, network, and security state
• a movement by all cloud providers to support the data container
• active work towards real-time movement of this light-weight data container between cloud providers and technologies

In essence, I would like to aim towards a cloudMotion (Just made up this word) standard that allows me to seemlessly move my data from Amazon to Terremark to Rackspace and then to my own data center without the loss of transactions, data, and time. One use case for cloudMotion would be that I know the hurricane is coming and I must move all my data quickly from the coast, 100s of miles inland. So speed of motion will be a very important component of such an endeavor.
cloudMotion will require hypervisor and cloud vendors to start working together instead of trying to out do each other. I belief something like cloudMotion is possible but will require some serious work between VMware and the other vendors. Perhaps this can be achieved via DMTF, but given its track record with OVF acceptance, I do not have high hopes. Perhaps another standards body is required. But for cloud to be ubiquitous we must concentrate on the data and how to move it quickly between vendors.
Perhaps this is where vendors like RightScale and CloudSwitch as well as the RedHat DeltaCloud  and Openstack initiatives can assist in this are. However, without buy-in to the idea and active involvement from VMware, Amazon, Microsoft, Citrix, and RedHat the concept of cloudMotion is just not possible and therefore a ubiquitous cloud is just not possible. The real choice is about letting me decide where to place my data not limiting me to a select few vendors that run a given cloud environment and by allowing me to transfer my data objects with security and runtime context at any time even as they process data.