OpenStack – Is it now Safe to Get Involved?

The OpenStack Governance Model is changing to remove the dominance of Rackspace by establishing an independent foundation. We were very critical of the old model, so we spoke to Rackspace to understand what was happening.

The first point is that things have got very heated in the war of words between Citrix and Rackspace regarding Citrix’s disengagement from OpenStack and creation of the the CloudStack project at Apache. It’s a real shame that there isn’t a single Open Source IaaS infrastructure.  In fact there are three fairly credible options – Eucalyptus, CloudStack and OpenStack, with (on the face of it) the first two supporting Amazon APIs and the third not supporting Amazon APIs.  So, the question is how this might have come about. Eucalyptus can’t really come together with either OpenStack or CloudStack without re-licening under Apache, with would break the Eucalyptus business model. The Amazon API question is more subtle.  There is nothing to stop OpenStack implementing an API which is Amazon-like, and indeed it has done so.  The question is whether an API can be said to be supported without a licence being in place with the originator of the API. Citrix can negotiate such a licence on its own behalf and so can Eucalyptus for its enterprise customers, but a foundation like OpenStack can’t easily negotiate such a licence on behalf of all of its consumers, nor would Rackspace be motivated to enter into such a discussion.

Second, the clash between OpenStack and CloudStack is currently only in the area of the Compute subproject, it is in principle possible to mix other components. How practical this is is unclear, and in due course the two projects may end up feature-matching subproject by subproject.  However, as we discuss later in this post, there are advantages to the way OpenStack is set up in handling multiple subprojects.

Rackspace were keen to point out that the day-to-day interactions within the project were already operating under established open source rules of engagement, and that in the creation of a new Governance Model the high-level structures were being put into place simply to safeguard these low-level processes.  Essentially to give consumers of the infrastructure confidence in the long-term viability of the contribution and management processes that were already in place, and to avoid problem if Rackspace were to be acquired by an organization which didn’t share its current objectives.

Now, of course, this stinks of “Marketing”, so we decided to drill in more deeply into the reorganization and see whether this made sense.

The Governance change involves the creation of a top-level Board of Directors appointed by major contributors and elected from more minor contributors – both smaller companies and individuals. The role of the Board of Directors is to make strategic direction decisions and to ensure the rules of engagement are kept fair – in any case a lot of this is governed by Anti-Trust legislation in various jurisdictions, and the people at Board level will be of sufficient stature to deal with these issues. The appointment of senior people to the board from major contributors also reflects the fact that many of the individuals working on the project will be employees of one or other of those companies (developers don’t live on air) and there needs to be authority within the contributing company to commit those individuals to work on the project.

The next level down is the Technical Committee which is elected – rules slightly unclear, but one would assume it is an election amongst the technical contributors.  The link between the Technical Committee and the Board is very light – you don’t want the senior people at the board level to be telling the technical people what to do, just ensuring the processes that operate at the lower level are appropriate, and that the activity is within the scope the organization wishes to address.  In practice, however, the Technical Committee can end up as a “Talking Shop”, with technical people competing to be elected to improve their Resumes and then scoring points off each other instead of agreeing anything concrete.  The thing that makes it all work is the project management process whereby the components of the system are forced to interoperate by being tested with each other and released as a single functioning entity.  It is worth pointing out that OpenStack’s special-purpose foundation model offers some advantages over the more general approach of Apache where cross-subproject planning and interoperability testing doesn’t happen so easily  – the projects are too disparate.

To drive the project at this level, however, you need dedicated project management resource to ensure that  project plans are created, prioritized and scoped, to manage dependencies and to track activity  against the plan, ensuring additional resource is applied to the plan if a particular high-priority element is  falling behind schedule. The interesting question is “Whose resource?”, because the project manager may need to ask any of the contributing organizations to step up to the plate, not just the one that happens to employ him.   Rackspace has supplied one of its own employees as a project manager who has an overview of the entire project, however we would venture to suggest this doesn’t scale, perhaps it should be distributed, or perhaps it should be actual Foundation Employees (rather than employees of the contributing companies) who manages the process – to avoid bias.

In the meanwhile, there are elements of the OpenStack approach that are designed to shame contributing companies into providing the resources to deliver enhancements. There is a very public process of creating technical blueprints for subsequent implementation, at this point commitments are made to provide the resources to actually deliver the enhancements, and if a contributor’s blueprints are regularly late in delivery or miss the release then this information is visible to the rest of the community. Then there is also a design summit – a huge meeting of technical people (mainly North-America-based ) who get together to do detailed design every six months and plan the following release.

The design summit is part of the process of integrating new developers into the project, along with public “Merge Reviews” to which anyone can contribute, however all of these projects do suffer from the fact that the initial group of contributors naturally remain in place, and end up as a voting clique in the meritocracy.  Clearly without Rackspace nothing would have happened, and you can’t move instantly to a diverse group of committers. It will be interesting to run the stats over time about the number of individual lines of code contributed by non-Rackspace employees over time.

One area where the project can more easily create diversity is in the creation of new projects. As thing stand the project is very focused on a core set of features: Compute, Object Storage, Image Delivery, Identity, Dashboard and Network, of which Object Storage (Swift) seems to stand out as slightly less core than the others.  Around it there are a number of open source projects not under the governance model, and not integrated with the release cycle, and we would suggest that OpenStack is likely missing an opportunity here to strengthen its market presence by folding these in. There is an opportunity for OpenStack to out-innovate Amazon within these projects, and thereby start to erode dominant market position, and maybe this could be an early initiative for the Foundation’s Board.