Google's Octane JavaScript Benchmark Targets the Modern Web

Google announced the release of a new JavaScript benchmark suite, known as Octane, which breaks tradition with existing benchmarks and adds five new tests to Google's V8 benchmark.

Google has released a new JavaScript benchmark suite known as Octane.

In a blog post about the new benchmark, Stefano Cazzulani, a Google product manager, said the Web is evolving and so should the JavaScript benchmarks that measure its performance. Octane, he said, is Google's way of measuring a browser's performance when running the complex and demanding Web applications that users interact with daily.

"Most of the existing JavaScript benchmarks run artificial tests that were created on an ad hoc basis to stress a specific JavaScript feature," Cazzulani said. The WebKit SunSpider JavaScript benchmark tests the core JavaScript language only, not the Document Object Model (DOM) or other browser APIs. It is designed to compare different versions of the same browser, and different browsers to each other.

Then there is Google's V8 JavaScript benchmark, which only tests JavaScript performance-with a heavy emphasis on testing the performance of recursion, said John Resig, creator of the jQuery JavaScript library in an early post. Mozilla released the Dromaeo JavaScript benchmark, which tests JavaScript performance as well as DOM and library functions. It provides a much stronger look at how browsers might perform under a normal Web-browsing situation, Resig said. Mozilla also released another JavaScript engine known as Kraken, about which former Mozilla software engineer Rob Sayre–co-creator of the Atom specification, said:

More than SunSpider, V8 and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open Web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.

Meanwhile, Octane breaks with the JavaScript benchmark tradition and extends the former V8 Benchmark Suite with five new benchmarks created from full, unaltered, well-known Web applications and libraries, Cazzulani said. A high score in the new benchmarks directly translates to better and smoother performance in similar Web applications.

The first new test in Octane is Box2DWeb, which runs a JavaScript port of a popular 2D physics engine that is behind many well-known simulations and Web games. The second test, Mandreel, puts a JavaScript port of the 3D Bullet Engine to the test with a twist: The original C++ source code for the engine is translated to JavaScript by Onan Games' Mandreel compiler, which is also used in countless Web-based games.

The third test, Pdf.js, is based on Mozilla's PDF reader and shows how JavaScript applications can replace complex native browser plug-ins. It measures how fast the browser decodes a sample PDF document. Fourth is the GB Emulator, which is derived from an open-source emulator of a famous game console running a 3D demo. And finally, CodeLoad measures how quickly a JavaScript engine can bootstrap commonly used JavaScript libraries and start executing code in them. The source for this test is derived from open-source libraries such as Closure and jQuery, Cazzulani said.

Besides an expanded set of benchmarks, Octane also has an interface that makes it easier to read and that adapts automatically to tablet and mobile screens, he added.