Virtual environments are about to get a benchmark to call their own.
VMware is leading the effort to offer a new benchmark that will measure how workloads perform in virtual environments. VMware will announce the new VMmark benchmark on July 23.
It should be no surprise that VMware, which is based in Palo Alto, Calif., and is considered the leading developer and vendor of virtualization technology, took the lead in developing a benchmark to test how application perform in these type of environments.
The development of the VMmark began in October, when VMware began developing a benchmark to test virtual machines, said Andrea Eubanks, a senior director for enterprise and technical marketing at VMware.
While only in limited use, Dell and Sun Microsystems have already published results based on the beta version of the benchmark. On July 23, VMmark will be open for general use, Eubanks said.
“Our customers have been asking us for some guidance on how to deploy enterprise workloads on virtual machines and what sort of mileage they can expect to get out of them and what do they have to do to meet best practices,” said Eubanks “These are the type of questions that go along with virtualization and VMmark is a way for to find the right profile for enterprise workloads within virtual environments.”
Charles King, an analyst with Pund-IT, said while theres always a drawback to having a company so closely associated with a technology developing benchmarks—in this case VMware – many customers need a standard to test how an x86 server handles workloads running in virtual environments.
“On the virtualization side of things, I think its very tough for business customers to have a strong sense of what kind of performance they can get from an application running in virtual partitions in x86 servers,” King said.
“You really want a whole testing processor to test the critical application performance that you want. In this case you, they have created a benchmark methodology and you get a server with either an Intel or AMD processor and this amount of RAM and then test to see what kind of performance you get from this machine.”
The problem with traditional benchmarks, Eubanks said, is that they were designed to test how one application performs on a single server. With virtualization, several applications can run on the same physical piece of hardware and are not typically bound by the usual resources such as CPU or I/O.
What VMware did is use several existing benchmarks that allow for running multiple, heterogeneous workloads in parallel to provide for a score to measure the system performance. This allows users to compare virtual environments.
The VMmark benchmark scales by running multiple, so-called “tiles” until the host machine reaches saturation. Each of these tiles is a set of six workloads, including a data base server, mail server, Web server, Java transaction server, and a standby server, for failover or quick deployments.
The final score, according to VMware, is based on the performance of the workloads at a given number of tiles. The two main benefits to customers, Eubanks said, is that the VMmark benchmark can show the right number of VMs a system can support, and it can give a proper measurement of the VMs density of a physical server.
Editors Note: The Standard Performance Evaluation Corporation was not involved in VMwares development of the VMmark benchmark program. This story, published on July 23 on eWEEK.com, incorrectly reported that it had been involved.