One of the great advantages of the public cloud is its elasticity, the ability it gives systems to provision and deprovision resources as workloads increase and decrease. Much has been written about how building RESTful services is crucial to deploying elastic services in the cloud. I concur that writing code loosely coupled with the underlying infrastructure and abstracting things like business rules, business processes, and systems configurations into independent modules is a key to elasticity. What I have not seen discussed enough is how we should be abstracting the different types of server farms away from each other to eliminate tightly coupled dependencies between compute resources.
Dependencies Kill Elasticity
Case in point: the following image shows a common anti-pattern that prevents systems from being truly elastic.
You can see in the image that the web server farm will need to be configured to know exactly what all of the servers in the application server farm are. Likewise, the application server farm needs to be configured to know what all of the servers in the worker node server farm are. When servers come and go in each of these distinct server farms, somehow all servers that are dependent on that server farm need to know that the servers exist and also need to be configured to connect to them with the proper permissions.
In most cases, a design like this introduces risks when servers are added to or removed from any server farm, because of the tight coupling (strong dependencies) between server farms. A better solution would be to treat each server farm as a “black box,” so that a server in one farm does not care what servers exist in the other farm. Instead, a server just makes a call to a black box and receives back the appropriate response. The next section shows how we can accomplish that.
Achieving Elasticity through Abstraction
In the following image, you can see four different places where we changed the architecture from being tightly coupled and elasticity-challenged to being loosely coupled and highly elastic.
The first change is that we added an external facing elastic load balancer in front of the web tier. Any incoming traffic from the web accesses the system through a static IP associated with the load balancer. This allows us to add and remove servers (and IPs) at will behind the load balancer without impacting any systems or services that create the incoming traffic.
Next, we added an internal load balancer. This load balancer is the “black box” that the web tier talks to so that we can add and delete servers (and IPs) at will in the application tier. It also allows us to apply access and tighter security policies in one place, where they can be consistently enforced across the entire application tier.
The third change we made was adding a queue in front of the worker node server farm. With this design, the application tier performs its required work and then hands the workload off to a queue, which is responsible for the delivery of the workload to the work server farm. Now the application tier does not care what servers or how many exist in the worker server farm. It is only concerned with communicating with the queue.
The fourth change is that instead of depending on local storage of servers that come and go, we use a storage service, in this case AWS’s S3 (Simple Storage Service), and access it via API. Now none of the servers in the web and application tiers are depending on any local disk from any servers in either farm. Instead, they use S3 as their black box for storing files and other data.
Summary
Load balancers, queues, and storage APIs are just a few of many methods of decoupling server farms from each other in modern cloud architectures. The message here is that if architects want to build highly elastic systems in the cloud, both the code and the infrastructure must be designed to be elastic.