Analytics within the Secure Hybrid Cloud

A big part of the secure hybrid cloud is the need for multi-tenant analytics to determine when security events and compliance issues happen. However, analytics cover many different aspects of security within the hybrid cloud, from being a control point for compliance to handling vulnerability scanning. What are the requirements for multi-tenant analytics?
Multi-tenant analytics are one of the more important aspects of the secure hybrid cloud (Figure 1) and have some fairly basic requirements:

  • Ability to gather data (logging, security status, scan results, network data, firewall events, etc.) from within the data center
  • Ability to gather data from the transitional areas of the secure hybrid cloud with the ability to use identity, device, location, etc. to map actual users
  • Ability to gather data from each of the clouds used by the organization(s)
  • Maintain data and access to data in a multi-tenant form
  • Secure data at rest as well as in motion
Secure Hybrid Cloud
Figure 1: Secure Hybrid Cloud (click image to expand)
Big Data Security Tools encompass the ability to analyze log files, network packets, and some aspects of security tools, but not all out of the box. You can always write your own rules and alerts, but the failure of current big data tools to already have alerts for hybrid cloud specific events is a definite area for growth. While the tools can handle firewall, switch, and many other well-understood traditional security  tools, they should also possess the ability to integrate into modern hybrid cloud systems in a multi-tenant manner. This implies something fairly specific.
Data placed within the analytics solution should be tied to one of the many tenants within a secure hybrid cloud, perhaps even split by data classification. Analytics need to preserve access based on classification levels as well as on tenant by datum or data stream and not necessarily as access to the entire pool. We also have to worry about the privacy of the data within a security analytics data store due to the value of this data, which can describe a secure hybrid cloud, the security layout, and what is part of an application. This data should be encrypted at rest and in motion throughout the analytics environment.
Multi-tenant analytics is the control point for many aspects of secure hybrid cloud, not only as a repository for all the hybrid cloud data but also as a control point for compliance, vulnerability scanning, and risk analysis. In a large hybrid cloud, you may need to handle terabytes of data per hour and report on compliance, vulnerability, and risk violations in a timely fashion across the entire hybrid cloud per tenant, without the need to have a separate analytics environment per tenant.
Today, the best approach is for each tenant to create an analytics environment per tenant and for the cloud service providers to use one within the hybrid cloud. However, that is quite a lot of extra processing and requires more capacity–analytics environments tend to be rather large and use a lot of storage and compute power. Instead, the cloud providers should provide shared multi-tenant analytics they can use themselves but also provide to each tenant. This, however, requires all logs to represent a tenant down through the hardware, and that is not currently a possibility.
The future will be interesting as the landscape of big data security analytics changes to include privacy, data classification, and risk controls. As security analytics grow, the need to include the basics of security into Hadoop and other tools will also grow.
We have discussed tools such as HP HAVEn, RSA Silvertail, Splunk App for Enterprise Security, but we should also add into this mix tools such as RSA Security Analytics. These tools need to ingest data streams from vulnerability scanners such as Qualys (a Security as a Service tool), BeyondTrust’s Retina, Anti-virus and Anti-malware from Symantec, Trend Micro, and others. Logs and alerts from Symantec Critical System Protection and other agent based security tools, logs from identity tools, and logs from application monitoring tools.
All the big data for security tools need to take streams of data from the other tools and come up with breach and compliance violation events as quickly as possible with the ability to drill down to the exact event by the exact user.
How do you secure your analytics within the hybrid cloud? What streams of data can they ingest?