Critical Testing Criteria for VM Management

Any more than a handful of virtual machines are a handful to manage. Leaving aside the fairly well-established concern of VM (virtual-machine) sprawl- orphaned VMs that are consuming resources with no reason or purpose- real issues today include how to charge for virtual machine services, keeping VMs up-to-date and ensuring that advanced capabilities including fail over and capacity planning are used to their full advantage. Here are 10 criteria that can be used to focus your testing of management tools in the present era of machine virtualization.

• Charge back- Probably the hottest topic to get serious attention today. As hypervisor platform vendors make resource counters available in order to track VM compute, storage and network usage, IT managers will have the information they need to bill business units for services rendered. Charge-back systems should be evaluated for flexibility (currency units for global use), show back/charge back, bill presentment and integration with bill processing. It's a bonus when your customers can get up-to-date reports on charges on a self-service basis.
• Lifecycle- This is a basic feature of virtual management systems and is a "check box" feature. A good VM management tool will be able to show running and dormant systems including basic information about the owner/creator. Spend some time in the lifecycle section of any VM management tool to get a feel for how well it handles systems as they spin up and become dormant based on the needs of your organization. Good tools will track when and how long a VM has been up and information about the operating system, applications and patch level.
• Cross platform- It's a lot easier to be an effective IT manager when all systems are visible. Ensure that your management tools can see VMs that are hosted on the hypervisor platforms used in your organization. If acquisitions are in your future, it's a business advantage when the IT department can quickly integrate the VMs in use at the acquired business. Therefore, even if your organization is settled on one platform, it's worthwhile considering a cross-platform management tool over one that is vendor specific.
• Business continuity- Will your VM management system continue along with your business continuity plan? Keeping track of VMs under normal circumstances is tricky enough. When business as usual is interrupted for any reason, you can be sure your management system will be called on to provide up-to-the minute information. Add your VM management system to your business continuity plan and devote a slice of your testing time to gauge how well your management tools recover.
• High availability- This is similar to business continuity but with a twist. Because HA (high availability) systems work in tandem with a primary/secondary relationship that is necessarily disguised so that applications can "take a lickin' and keep on tickin'," it's worth seeing how your VM management system handles such specially configured systems. HA systems are usually monitored through a vendor-specific console. If there are integration points between your management platform and the vendor's console, understanding how HA events are logged and reported will usually reveal how much thought went into the management tool.
• Service catalog- Catching a VM right out of the gate is the very best place to start managing your virtual environment. If you management tool includes a service catalog feature, so much the better. If not, than testing the integration of your management tool with the service catalog is the next best thing. Don't have a service catalog or self-service provisioning system? Put that on your evaluation short list. In the not-too-distant future, providing a VM environment without a self-service provisioning system will seem as antiquated as placing a person-to-person phone call.
• Configuration- Here I'm referring to the VM configuration monitoring. Your management system should be able to report the current patch level of the operating system and applications on your VMs. You'll likely use a separate tool to actively manage patching and strict configuration management. However, knowing a basic history of your VMs in terms of current version is totally within the realm of a good VM management tool.
• Resource use- It should be easy to see the current CPU, memory and disk usage of each VM and aggregate totals based on groups of machines. This doesn't need to be anything more than a dashboard in the management system. The real work of balancing VM and physical host resources- along with decisions about how to move workloads around to balance performance based on policy- should be handled by the hypervisor platform.
• Capacity planning- This is a good feature to check in a VM management tool. Check to see if the advice you are given about the amount of resource needed and the time frame within which it should be added actually make sense. Keep in mind that if you use too little data you'll likely get wildly incorrect recommendations. Capacity planning is a tricky thing by nature. Even so, the capacity predictions you get from the system- when you provide the right data- should be within the realm of reality. Otherwise, keep shopping.
• Reporting- Reports should be easy to generate and distribute. This means there should be plenty of standard reports that come with the product and these reports should provide the basic information needed for each target group. Execs need summaries; IT needs details. Look for reports that restrict scope to only the information needed by the recipient- there's no reason to give the e-mail team reports on finance infrastructure.