In Part I of this series on Do Containers Change Enterprise IT, we discussed the impact of containers on security. This time, we will discuss the impact on data protection, which encompasses backup, disaster recovery, and business continuity. Since the applications are changing with the use of containers to be highly distributed and deployed through infrastructure as code, what to protect now becomes a major question. How we protect is well known, but what changes once containers are in use.
So how does what we protect change? We still need to protect our data and deployments, just like we did previously. However, containers offer a scale and ease of deployment never seen before. Containers are invoked by automation engines, with the infrastructure being deployed as code as well as upon well-known, easy-to-deploy physical or virtual environments. Containers package code and infrastructure in such a way that what they run upon is no longer a major issue. So do we need to protect those environments as well?
A highly distributed cloud-native application has built into it business continuity capability, as such applications are designed to be resilient within themselves. If one part dies, another piece takes over automatically. Where these components live is not much of an issue. However, the data created and used by the containers needs to be stored not in each container, but within some centralized and distributed database to ensure the data is available at all times to all components. The containers themselves become stateless, yet the application may be stateful within various layers of the application. For that, we need to store data somewhere outside the container.
Yet, containers and the applications within them may need to be configured differently for the environments they run upon. Of course, we always have to fine-tune applications, but this should be handled by automation engines used for deployment. Ultimately, it looks like data protection resolves to just the data and these all-important automation engines.
Unfortunately, this is not always the case today, but tomorrow perhaps. Yet even so, we are missing several other key components. Here are some questions to ask around data protection of containers:
- How do we define the application so that we can restore it automatically? In many cases doesn’t the automation engine contain all we need? Perhaps; however, there are application dependencies we may not know about. The dependencies can be between multiple containers, multiple applications, a legacy system (such as a database), or other components we have yet to understand.
- How can we automatically adjust and adapt data protection policies throughout the lifecycle of the application, as applications are updated and deployed? We need to use analytics to determine how often to back up the dependencies of each containerized application based not only on utilization, but on amount of change to an environment.
- How can we automatically test our recovery to determine if we can safely deploy these highly distributed applications? Is this just restoring all containers? Or do we restore the automation engine and the data sources it uses, then use it to spin out the application? If we do not know the ultimate dependencies, how can we fully deploy the application?
- How can the results of our application testing and adaptive policies feed back to the definition of the application so that over a short period of time, the full application is available for restoration?
- How can we determine what part of our containerized application needs to be restored? Shouldn’t our restoration be adaptive and automatic as well? If only one component of our application needs to be restored, can we automate that deployment and ignore the rest?
These questions apply to all systems, not just containers. As we move to more and more automation, the ultimate dependency for an application moves from the deployed application to the tools used to deploy the application. Those all-important automation engines. For these questions, we could easily substitute in “virtual machine” for “container” and still have the same set of questions. While the questions do not change, what we back up does change.
We need to discover the application and its ultimate dependencies in an automated fashion, either as or after the application is deployed, so that restoration can be handled intelligently and quickly. To that end, data protection should not just be a set and forget, but should be part of deployment tools and should use more traditional application detection mechanisms to automatically define the application and its dependencies. Data protection needs to understand more than the container, more than the virtual machine—it needs to include the network, utilization, amount of change, and ultimate dependencies. The ultimate dependencies could be within any layer of the Substrates of IT (pictured below):
- back up the deployment service configuration
- back up the deployment service repositories
- back up the deployment of any IT management agents required underneath the containers
- be part of the deployment service to capture how the application is deployed, such as the Infrastructure as Code used to deploy this specific application
- back up the configuration of storage, network, etc. required by the application
- determine other dependencies, such as databases, network configuration, security, and authentication mechanisms in use
- use performance management to adjust policy based on utilization and amount of change
In essence, data protection needs to change to be able to automatically restore an entire environment for an application without human intervention. Humans should only review the results and set priorities for restoration based on user impact.
We are far from this today.