Application Awareness in the Cloud? Is it possible?

As we move closer to the cloud (private or public) the vendors are starting to circle the wagons around capacity planning, charge back and availability. In order to do this, the vendors (hardware and software) need to enable their components to be application aware. What exactly does this mean and what have we learned from our past?

A bit of history. Back in the main frame days, there was one computer being shared with batch jobs, online applications (think web today) and databases. There were more but lets start with this list. With only one computer to run all these applications on, there was a need to prioritize the different applications based on available resources and the priorities of the application (i.e. business unit). IBM and others created some thing called Workload Manager. The goal was to provide metrics at the resource (CPU, memory, I/O and network) and application (which application was using the resources and which one had priorities over other applications) level.

Moving back into today's world, in the physical (not virtual world), this is easy. Each application was installed onto one server. The application could use all the resources it wanted as it was the only application on that particular server. With the average CPU utilization rates of around 10%, this model is a huge waste of resources and power. Virtualization has in many ways, taken us back to the mainframe days (without the single computer).

So how do we enable the cloud to be application aware? A good start is to look to our past and what Workload manager did for us in the mainframe days. VMware has done a good job of prioritizing VM's with their resource management functions. This helps with ensuring resources are available on a given ESX host at start time and also for memory and CPU resources during run time. This assumes the VM is a single application, which in most cases it is. With the cloud, many applications from potentially many different customers will be running in the same environment. How do we prioritize these workloads based on service level agreements, price points (think phone companies charging more for day time calls, vice night time calls) and resource availability.

I propose the need for deeper awareness of the applications and resources by utilizing another piece of mainframe history. That being the concept of Systems Management Facility (SMF) and Resource Management Facility (RMF) type records. These records provide the ability to collect resource usage for performance and tuning, application usage for charge back and workload management for prioritization of applications across the cloud.

VMware has started the charge by opening up their API and some third party vendors have started bringing products to market for reporting and alerting on storage, CPU/Memory and network usage. Next steps are to tag the resources allowing application aware reporting and prioritization in the cloud.