The Open Source Development Lab will on Monday announce that it is providing software developers access to a code benchmarking tool known as the Scalable Test Platform.
As first reported by eWEEK , the tool will provide open source developers with a central resource for the stable and scalable testing of enhancements to the Linux kernel across a variety of tests and server configurations, from one-way systems to 16-ways.
Tim Witham, the director of the Lab, an independent, non-profit entity designed for developers adding enterprise capabilities to the Linux operating system, told eWEEK that the STP fulfills a critical need in the Open Source Developer Community – one that has been missing until now.
"Developers do not tend to run performance tests," he said. "Rather, they identify errors and design patches to fix them. We identified the need for a set of machines that remain stable over time and in configuration so that developers can run repeated tests on updated code using the same machine configuration."
Further reading
Developers would be able to perform a range of tests, from performance and functionality tests to regression and stress tests. The service will be free and developers will simply sign-up as lab associates to use the tool. They would then be able to upload kernel patches through its repository and then use this system to run their choice from a set of performance and scalability tests.
As some tests required a significant time to run, they would be entered into a queue and executed as machines became available. Results and/or errors would then be e-mailed to them. Those results could also be listed on the OSDL Web site.
"Not all of our machines will be configured for all tests," Witham said. "If its a single-stream test you probably want to run it on a one-way and not on a 16-way. So the developers can figure out what machines are available and pick the ones they want. They then type in a tag and submit it, at which point it builds the kernel, installs the operating system with the patched kernel, installs and runs the test, logs the results and reports back."
This is significant given the high-level of automation and the fact that results are logged – which allows developers to make comparisons over time based on archived data. The lab also intended to offer a broader range of tests going forward, that would enable "us to grab the kernels as they come out and run the whole suite on it and publish them. So well have the whole baseline as we move through there," he said.
Linus Torvalds, the founder of the Linux operating system, was aware of the new tool, Witham said. "But [he] traditionally doesnt throw his weight behind new initiatives," he said.
While some of the elements of the STP had been developed in-house at the lab, it was essentially a SourceForge project. SourceForge hosts hundreds of open-source projects.
Initial response from kernel and other developers had been positive so far, as they would be able to continue with their development work while using the system to test their patches, he said.
Moving to address some concerns that the tool and results would be used as fuel for marketing wars as they are with other operating systems, Witham said one of the rules of use was that the information would not be used for marketing of any kind.
Witham said its second facility outside Tokyo would hopefully be up and running by the year-end. It would supplement the U.S. facility near Portland, Oregon, and increase its ability to enable projects that expanded the enterprise and carrier-class functionality of Linux and Linux-based software, he said.
