In the last three virtualization and cloud security podcasts, Mike Foley, Sr Technical Marketing Architect for vSphere Security, mentioned security disaster recovery plans. There is a growing need for such plans. The 174th podcast covered this need, as well as the why and the how of putting such plans together. Unlike traditional disaster recovery, security disaster recovery is just what it sounds like, recovering from a disastrous security event. How would your organization respond to such an event? Is it about incident response? It is more than this. While you listen to the podcast, consider these thoughts.

There is disaster recovery from an unknown event, as well as disaster avoidance and business continuity. We have plans to recover, and we test those plans as required (hopefully yearly). Do you? Disaster recovery is about recovering the business, not about recovering individual systems, although individual systems are certainly involved. What does disaster recovery mean from a security perspective or in response to a security issue? If our systems were hit by ransomware (or any attack, really) and the attack was successful, how do we respond?
Many may say this is the reason we have incident response folks. The problem is that there are few incident response folks. This is the most unfilled job of the over 1M security jobs available. And if you have incident response folks, even they need to have instructions to follow in the ultimate of disasters. This is where the security disaster recovery plan comes into play. It considers all the options.  Such options include but are not limited to:

  • Means to verify and if necessary remediate underlying infrastructure such as firmware, hypervisor, storage controllers, switch controllers, etc.
  • Means to verify and if necessary remediate/restore switch, firewall, software defined storage, software defined network, and other security network element configurations
  • Means to verify and if necessary remediate/restore credential, key, and other security managers such as Active Directory, KMIP servers, and even DNSSEC installations or configurations
  • Means to verify and if necessary remediate/restore hardening of infrastructure, physical or virtual machines, containers, and other contructs within your secure hybrid cloud
  • Means to verify and if necessary remediate/restore security settings of applications and services

In essence, the security disaster recovery plan works hand in hand with the normal disaster recovery plan but covers all the security aspects of the recovery and goes further to apply security and compliance verification at every step. This is a big deal. This will take time to create. It is not enough to have a backup of your firewall; what if the underlying infrastructure was hacked (not impossible with admin escapes)? This is also the opportunity to create some brand new and very interesting scripts. The verification alone is an example of one form of scripting required.
The creation of a security disaster recovery plan is a chance for security to become a major part of the business. Such a plan needs to work with the business, as well as with the rest of IT, to restore a secured environment so that the hack just does not happen again. The plan is a blueprint for protecting the business assets from a security perspective, not a general data protection perspective, yet the two go hand in hand.

Closing Thoughts

Implementing a security disaster recovery plan is the responsibility of the CISO. During a security disaster recovery, the CISO’s job is similar to the space program’s capcom (“capsule communicator”) job. That job is to be the communications channel between the folks doing the recovery and the business. They need to keep recovery on track, understand any issues that may arise, and translate them to the business and back again. To go one step further, there also needs to be the equivalent of the flight director, whose job it is to go to everyone asking for a go/no-go decision when proceeding to the next step. Some steps are important, some are trivial, but we need to follow the book; ensuring that we do is the job of the flight director. Capcom is one of those the flight director asks for a decision: is the business ready?
These roles could filled by anyone. What is important is that the business be considered; after all it is the business we are recovering. The plan should also be vetted and tested, just like any other disaster recovery plan. Do you know how to recover when a security disaster hits? Do you know how to recovery security when a traditional disaster hits?