Different Companies, Same Problem

By Usman Muzaffar  |  Posted 2010-07-06 Print this article Print

Different companies, same problem

Again, there is a striking similarity in the problem evolution across companies. Early in the life cycle, developers write an application-specific test harness to validate the nascent product. Indeed, modern best practices-especially the test-driven development (TDD) practice in Agile software development-place such a premium on being able to continually and automatically exercise components as they are written that, in some cases, the test harness matures faster than the product.

The situation is complicated when release engineering promotes the harness (originally created only to assist the developer's edit/compile/debug cycle) to a key piece of infrastructure in the production integration and release process.

Now, suddenly, small details that could be taken for granted in the developer's workflow ("of course your database is at localhost:3306" and "of course gcc 3.3.1 is installed at /usr/tools/bin") require constant attention. There is no incentive for the developers to address the issues that are only faced in production. Conversely, there is no facility for the release engineering team to expose their process to developers.

Three challenges crystallize as the answers to the questions above. First, it's hard to get the environment, including both harness and product, set up on an appropriate box. Second, it's hard to invoke the test. Third, it's hard to mine the results for concrete next actions.

This triumvirate-resource configuration, test execution and results monitoring-is the "last mile" of automation. When ignored, the entire value of automated testing pays a steep penalty. Fortunately, for each of these systemic problems, there are simple (though not necessarily easy) solutions we can apply to great effect.

The resource selection problem is simply this: "Where do I run the test?" At most shops, the answer itself quickly leads to a sysadmin puzzle: "Where or how can I easily log into 20 systems in one click? Where do I have SSH (Secure Shell) keys configured? How will I keep my tests from stomping on my neighbor's? And how do I fire up virtual machines from an ESX server for my config?" and so on.

The correct answer: Stop asking these questions! Get your test harness to do it for you. There are three critical pieces you need to make that a reality-and one big trap to avoid.

Usman Muzaffar is Vice President of Product Management at Electric Cloud. He was part of the team that founded the company. Prior to Electric Cloud, Usman worked as a software engineer at Scriptics and Interwoven (acquired by Autonomy), designing and developing content management, syndication, and distribution systems. He holds a Bachelor's degree in Molecular Biology from Northwestern University. He can be reached at usman@electric-cloud.com.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel