There are a number of companies that are in a race to own the enterprise landscape when it comes to infrastructure automation and development pipelines (aka continuous integration and continuous deployment). What is unfolding here is very similar to what we have witnessed in the cloud market.
We have already seen Amazon, Google, and Microsoft take hold of the public cloud market, forcing legacy hardware vendors to change or die. We are now seeing a similar scenario unfolding in the automation space. The difference is that many of the automation companies saw this coming and have strategies in place, whereas in the legacy hardware industry, many companies have been playing catch-up, and it’s too late.
Orchestration Is Hot
Why is orchestration so hot these days? Merriam-Webster’s Collegiate Dictionary defines “orchestrate” as “to arrange or combine so as to achieve a desired maximum effect.” In the world of IT, that means efficient automation. We have been automating things forever. What is different today is that we have transitioned to a world of cloud computing, containers, microservices, and horizontally scaling applications. Not only has the underlying tooling changed, but our architectures are made up of many more moving parts than ever before.
We are moving more workloads to virtual infrastructure as opposed to physical infrastructure, and in many cases, we expect and design the virtual infrastructure to fail. We can no longer go look at and touch physical servers that don’t change often. Instead, we need to dynamically build and tear down infrastructure on demand.
All of this leads us to a world where our infrastructure and applications need to be fully automated and self healing, and to behave more like a small cell within a large, complex living organism. The challenge the industry has is that it takes so many different tools to orchestrate the complete build pipeline that companies are challenged with integration and complexity issues.
This is why orchestration is so hot and so many companies are in the race to become the de facto standard that enterprises rely on. Who is in the race? Maybe a better question is “Who is not in the race?” I’ll break it down into a handful of categories.
Legacy Hardware/Virtualization Vendors
Companies like VMware and Citrix that have had a hold on enterprise operations for a long time are offering orchestration tools for their loyal customers. These vendors focus on managing hybrid clouds by offering a common platform for orchestration, management, and visualization of cloud assets across any cloud. VMware offers vCloud and vRealize products to help companies manage, visualize, and orchestrate their applications across a hybrid cloud. Citrix has its solution, CloudPlatform, which provides similar services. There are many companies, like Dell, HP, EMC, and other hardware vendors, that offer similar capabilities.
The challenge with some of these products is they tend to be very proprietary and were designed after they realized the game was changing. In contrast, many competing solutions were built from the ground up to tackle orchestration and hybrid cloud management. Another challenge is that these solutions are only attractive to system admins and operations folks, whereas a lot of modern companies also include developers in the evaluation process. With the maturity of DevOps in many large organizations, modern tools need to satisfy both developer and operation needs.
Cloud Management Platforms Pure Plays
These solutions are very similar to the legacy vendor orchestration tools I mentioned, but they were built specifically for cloud management and are independent from and agnostic to any cloud endpoint. Companies like RightScale, Scalr, ServiceMesh (bought by CSC), and Enstratius (bought by Dell) were built a few years back by visionary startups who saw early where the cloud computing market was heading. They all have built easy-to-use platforms for managing multiple cloud endpoints and integration with your favorite tooling (Chef, Puppet, etc.)
One of the challenges with these platforms is that many enterprises are not mature enough yet to use a tool as sophisticated as these. These tools come with a steep price tag, and if the company is not competent enough with running applications in a hybrid cloud, it doesn’t get the full value out of these tools. These tools are operations focused and will integrate with whatever solutions the development teams have constructed for managing their pipeline.
CI/CD Pipelines Solutions
This is where the real war is for market share. Configuration management software like Chef, Puppet, Ansible, and Salt all have been fighting over developers’ and operators’ mindshare. In fact, most enterprises I go into have more than one of these solutions in place.
The challenge here is that many enterprises have gone off and written thousands and thousands of scripts and have coded themselves into a complex corner. In addition, there are so many permutations of operating systems, networks, disk storage devices, cloud endpoints, and now container technologies that there seems to be no end in sight to the amount of scripting it takes to stand up heterogenous environments.
Companies like Chef and Puppet have beefed up their offerings over the last few years in an attempt to own the orchestration space. Chef offers Chef Server and Puppet offers Puppet Enterprise as their one-stop shops for managing and orchestrating applications across any cloud endpoint.
The good news is that these solutions target both developers and operators and were built from the ground up to tackle automation and orchestration. The bad news is that much of the work that developers and operators have been doing for years is being abstracted away by the next generation of tools.
Enter containers. If you haven’t played with Docker yet, I strongly advise that you do. What used to take a ton of code in a CM tool like Chef or Puppet can now be done with a simple configuration using Docker. Docker hides a lot of the complexity of multicloud configurations so that developers and operators don’t spend so much time and effort trying to stand up infrastructure and deploy applications.
Early versions of Docker did a great job of porting applications within containers across clouds’ endpoints, but it still required a lot of hands-on configuration to deal with the permutations of network, storage, and database configurations. All of those issues, as well as security issues, are being addressed within the Docker ecosystem by various companies, some of which Docker has acquired.
Docker Hub is in a great position to win or lead the market in the orchestration war. The war is at the intersection of infrastructure automation and build automation. Docker Hub fits nicely in this area. Look at the large ecosystem that has grown in the container space. Docker has become much more than a container company. It is trying to be the one-stop shop for building and deploying applications on any infrastructure configuration, with the goal of making it simple and streamlined.
Another key player is Jenkins. Jenkins has the lion’s share of the market when it comes to build automation. In fact, any tool that plays in this space gains little traction if it is not able to plug into Jenkins. CloudBees offers managed Jenkins as a Service so your engineers can focus on automation rather than installing, patching, scaling, and configuring your Jenkins cluster. Here is the CloudBees platform approach:
Speaking of platforms, there are a ton of startups looking to be the de facto “DevOps” platform. Companies like Codeship.io, StackStorm, CloudMunch, and many others each have their own twist on orchestration but aim to be the control center for automating your pipelines.
One company I recently talked to is taking a different approach to automation, and that is RackN. RackN is an early stage startup with roots in OpenStack and data center implementations. CEO and Founder Rob Hirschfeld understands that most enterprises will live in a hybrid cloud world having to support many cloud endpoints. RackN strives to abstract away all the complexities of containers and infrastructure configuration across any endpoint in support of what Rob calls the “hybrid DevOps.” Rob says that it is rare that a company will go “mono cloud,” and the trick to remaining agile is to be able to abstract away the differences between the cloud endpoints so that developers are not forced to use a particular cloud that might not meet their needs.
Then there are PaaS vendors. Let’s not forget that the entire premise of PaaS is to let the PaaS vendor manage all of the infrastructure, clouds, and middleware so the developers can just focus on business requirements. All of the leading PaaS players are leveraging and/or integrating with almost all of the technologies mentioned in this article.
Summary
There are far too many companies to mention that are competing to be the orchestrator of choice. There are also many different layers in which these companies are competing. There is the cloud management platform layer, where the RightScales and VMwares of the world are competing. There is the infrastructure automation layer, where Chef, Puppet, and Docker are staking claims. And finally, you have DevOps platforms, where companies like CloudBees, CloudMunch, and a ton of others are playing.
This is an interesting space to watch. I think the winners will eventually be gobbled up by large organizations that innovate through acquisition. Docker has a track record for buying startups that fit a gap in its ecosystem. I wonder how many offers Docker has turned down already?
The number-one benefit of cloud computing is agility. Agility can only be achieved if we can make infrastructure, code builds, and code deployments simple, automated, and repeatable. Orchestration plays a key role in this. No matter how amazing your cloud vendor of choice is, if you don’t have flawless orchestration, you won’t like your results.