Optimizing Virtualized Workloads

The pros and cons of virtualization are well documented, as are the advantages and disadvantages of cloud computing architectures. So, if you are managing applications and IT services in such environments, the choice has been made. Accordingly, you are now more likely focused on how to get the very best, cost-effective performance out of these resources and its infrastructure. This is especially true for those of you who are running mission critical workloads in a public cloud, like Azure or Amazon Web Services (AWS).

Although the following is applicable to private clouds, this discussion concentrates on third party applications and services that can be utilized to optimize workloads in a public cloud, specifically Amazon Web Services. AWS was selected because it holds the lion’s share of companies running business applications and revenue generating services in a public cloud. As for the meaning of “optimize” in this discussion, it’s all about increasing compute performance and concurrency loads, decreasing workload latency, and enabling greater consolidation of virtualized systems using less expensive, cloud resources and infrastructure.

As you are aware, virtual machines normally share physical processors, DRAM, I/O and network interfaces, and persistent storage. Consequently, to increase the productivity of these shared resources and, in turn, optimize the applications and services running within them, it’s necessary to minimize the impact of resource sharing contentions between VM instances. The techniques that are especially important in this regard, and that deliver high workload performance, low latency, and higher workload densities fall into three broad categories; in-memory computing, compression, and pipelining.

In-memory computing is a hot-topic these days. And as Intel’s 3D xPoint technology and Zaius P9 Servers from Google/Rackspace become widely used, the topic will take on even more importance. But don’t forget about the valuable benefits of compression and pipelining in workload productivity and performance. Let’s take a very quick look at these two techniques before diving into in-memory computing alternatives.

There is an impression that compression and decompression burn too many CPU cycles to be cost and resource effective. This just isn’t the case. Compression and deduplication algorithms have improved greatly, and combined with far more powerful CPU’s, there is barely any cost to compression at all. In fact, it can be faster to compress data before sending it onto the network because the network stack uses more cycles per byte than many compression engines. Accordingly, compression can improve not just compute performance on lower cost instances, but it can lower the cost of network usage in the cloud as well.

If your workloads involve vast disk scans like analytics and/or streaming data, then utilizing data pipelining is valuable. The usual purpose of a data pipeline is to collect, aggregate, process and move data at cloud scale. It’s about getting data that is accessed together, stored together. The key is not just the method of laying down the data in an organized way, but improving the methods of scanning for and retrieving data, all of which can increase the productivity of cloud infrastructure and improve user experience.

Implementing compression and pipelining can be tricky. Fortunately, there are several 3rd party tools and utilities in the AWS-Marketplace that can be provisioned as needed. Search for “compression” or “pipelining” in the AWS-Marketplace to find these Amazon Machine Images (AMI’s).

As for in-memory computing, AWS users often first think of Amazon’s own ElastiCache service before considering other in-memory computing alternatives. ElastiCache is a viable in-memory computing option, and you can get an overview of its capabilities here. But as with many options, one size never fits all. This is especially true when it comes to addressing the challenges of optimizing virtualized workloads using less expensive cloud infrastructure. It’s good to have more than one option.

In-memory computing means using a type of middleware that leverages DRAM to store active data, often across clusters, processing its operations in parallel. Searching the AWS-Marketplace using the “in-memory” search term, will bring back many potentially good 3rd party options. The ones that best address workload performance generally fall within the following categories:

  • Caching utilities & tools
  • In-memory databases
  • In-memory data-engines

This last category, in-memory data-engines, seems to have a lot of permutations. AWS-Marketplace sellers of such systems are all trying to differentiate their offerings. Accordingly, some use the term; “data grid” and “data fabric” or “software defined memory engine” to uniquely identify their products. Basically, they are all improving on basic caching utilities and tools using middleware. They enable data to be stored in DRAM across a cluster of servers and processed in parallel. For larger, more expensive AWS-EC2 instances, these 3rd party options work well in delivering better workload performance. They can be worth the effort to deploy if you are already using, or your budget permits provisioning, larger EC2 instances.

Simpler, and less costly approaches to increasing compute performance and concurrency loads while decreasing latency, are available in the first category above. They can be found by searching the AWS-Marketplace for “workload optimizers.” Some directly compete with the capabilities of Amazon’s ElastiCache by providing Redis, Memcached, and other caching engines that users can configure specifically for their workloads.

The other options in this group are often delivered as pre-configured EC2-AMI instances. These pre-configured EC2-AMI’s usually come with all the value of caching utilities and also include the added benefits of data persistence already built in. They are similar to other data persistence caching utilities, but don’t require adding application code to use them. Just deploy applications and services on these pre-optimized EC2 instances to get the performance of larger, more expensive instances from smaller, less costly instances. SysAdmins and DevOps teams usually provision these instances or develop their own. You can learn about this topic here.

Be aware, some of these pre-configured EC2-AMI’s distribute in-memory capabilities across clusters, referred to as “distributed caching,” and others do not. But they all optimize workloads, usually by orders of magnitude. So, select the option that best meets your budgets and performance goals.

In-memory databases are most often used for big-data and real-time analytics. But developers are finding new use-cases for these platforms as they take advantage of marrying the performance of DRAM with the capabilities of flash, SSD, and HDD storage tiers. Consequently, in-memory databases are becoming a more viable alternative to optimizing virtualized workloads.

Basically, in-memory databases seek to mitigate, if not completely eliminate, the need for data-marts and warehouses, data aggregation, and traditional indexing methods; and the need for separate transaction and analytic databases. These databases can shrink the data footprint as well, speeding up both analytical and transactional performance. This can reduce the cost of cloud storage and maximize the productivity of larger, memory-enhanced, AWS-EC2 instances.

Although in-memory databases avoid most of the traditional architectural tenets of relational database systems that are optimized for disk-resident data by focusing instead on DRAM, flash and SSD storage tiers; many in-memory databases do support traditional ACID compliance. Look for those if your applications have such requirements. Search for “in-memory databases” in the AWS-Marketplace to find them and the other in-memory database options.

All the techniques of in-memory computing, compression, and pipelining outlined above can be used to increase compute performance and concurrency loads, decrease latency, and empower consolidation of virtualized workloads. The challenge is selecting the ones that best address your performance goals and budgets. So, it is probably best to start with the less expensive, easier options as referenced above first, and work up.

Think about it. What will the difference mean to your business if you can deliver workload performance that is 10 times, 40 times, or even 100 times faster than before using less expensive AWS-EC2 instances.