SPEC Launches SOA Benchmark Effort

The Standard Performance Evaluation Corp. has formed a new working group to develop standard methods of measuring performance for typical middleware, database and hardware deployments of applications based on the service-oriented architecture.

SPEC member companies that have so far committed to developing a new SOA application measurement standard include IBM, Oracle and VMware, SPEC officials said. SPEC announced the formation of the group on Sept. 9.

“Service-oriented architectures are being deployed within enterprises of many sectors, including government, banking, retail and manufacturing,” said Andrew Spyker, an SOA run-time architect and chair of the new group, in a statement. “The benefits for companies deploying SOA include business flexibility and cost optimization. An industry-standard benchmark will help SOA users understand best practices for improving performance and help vendors deliver performance optimizations based on typical customer scenarios.”

Meanwhile, SPEC is interested in hearing from enterprise architects, IT managers and other potential users of SOA techniques to ensure that the working group understands customer needs and can develop the best possible benchmarking solutions, SPEC officials said. Also, organizations that are not currently SPEC members are invited to join the new working group. Membership is open to vendors, universities, research and development organizations, and users of SOA technologies, SPEC said.

Moreover, in developing the new SOA benchmark, SPEC will draw on its expertise in creating widely used system-level benchmark suites. SPEC is known for its benchmarks in areas such as power, CPUs, graphics, Web servers, mail servers and much more.

The new SPEC SOA working group plans an initial benchmark designed around three parts of a typical SOA deployment infrastructure:

• Services on top of application servers using Web services;

• Enterprise Service Bus (ESB) technologies that connect and mediate the services; and

• Choreographing services into larger composite applications through BPEL (Business Process Execution Language) technologies.

In addition, a key aspect of the benchmark will be measuring SOA technology use in ways that are typical in customer deployments while being flexible enough to cover current vendor implementations. Thus, SPEC expects that a wide range of computer server manufacturers, systems integrators and SOA technology software vendors will run the benchmark and report results, SPEC officials said.