How can companies deal with “dynamic unpredictability?”

In many corporate IT situations, this is the million-dollar question – how to handle the dynamic unpredictability that comes from putting significant amounts of digital enterprise operations into cloud or virtualization systems.

IT professionals skilled at assessing and managing cloud and virtualization systems will be familiar with a wide range of issues that lead to dynamic resource use. First, there is the relationship between virtual machines and hosts, and the setup of servers and other components of the system. There is the nature of peak time demand on systems, as well as downtime. Then there is scalability – as systems scale, they can experience something called virtual machine sprawl or project bloat where more instances than necessary are created, which leads to confusion across the entire system. In general, the dynamic handling of workloads causes its own chaos, a chaos that companies have to proactively handle in order to make efficient use of resources. In addition, the changing use of various applications may require a company to have an application decommissioning strategy, or suffer the demands of obsolete applications on a system.

On the storage side of the equation, there’s also a lot of dynamic demand. Companies may need to deal with storage tiering, where hot or more frequently used data needs to be moved to a particular area of storage, or other types of data sets require specific handling. Certain data may have to be placed on a separate tier. All of this can require significant amounts of real-time management. Memory constraints can cause problems, and the improper assignment of virtual machines can create bottlenecks that may need to be manually resolved. In this sense, system administrators often play the role of a busy “traffic cop,” trying to direct workloads and data-handling tasks to and away from given VMs and hosts in a system.

Companies have to do all this while managing a myriad of services from popular providers like Amazon Web Services or Microsoft Azure.

One of the most basic ways to handle dynamic unpredictability is to manually adjust these systems over time. Many companies have gotten proactive about brainstorming and creatively fine-tuning systems by getting a visual look at how virtual machines and other components work in real time. This can help companies start to handle and peak time demand and other issues.

However, some of the companies getting the most out of cloud or virtualization systems have started to use automation platforms that will intelligently make changes in VM assignments or resource allocations without the constant input from a human decision-maker. These autonomic systems often include lots of data visualization, with dashboards and reporting elements that show how the dynamic unpredictability of digital systems is being managed through the principle of machine learning.