Information in Formation

eWEEK Technology Editor Peter Coffee examines why there's less division than there once was between high-performance and data center computing.

When you hear the phrase "flying in formation," you probably think of the dazzling maneuvers of an air-show team such as the Thunderbirds or Blue Angels.

You see another example, though, whenever you drive on a freeway: vehicles moving at high speed, in close proximity, pursuing similar but independent goals with only each vehicles movements as signals to the drivers who coordinate their actions.

A one-piece design rather than a coordinated "formation flight" of components may seem a simpler approach to a solution.

As a problem grows to larger scale, however, a massive monolith can turn out to be more difficult to design, deploy, debug, maintain and improve. Thats especially true if the problem, or the state of the art in the technologies used to attack it, is still evolving—and even more so if those evolutions proceed at different rates.

People learn to maneuver cars, most of the time successfully, with little formal training. Observing this, MIT Associate Professor David Miller has explored other ways of using cooperating units, coordinated by adaptive algorithms, rather than monolithic systems to perform complex tasks.

Millers remarks, made at an MIT alumni event this month in Santa Monica, Calif., were in the context of student research projects aimed at improving the cost-effectiveness of satellite systems. One group is using magnetic fields to maneuver satellites relative to one another; another group is building small satellites for indoor operation on the International Space Station to serve as low-cost (and low-risk) formation-flight testbeds.

My aerospace-industry readers will be interested in both projects—the first with its potential for letting satellites work near one another without thruster-caused damage or surface contamination, and the second with its promise of more rapid evolution of flight software.

But while Miller and his students do what they do, Id like to consider "formation flight" in the IT milieu. The concept, though not the label, was apparent in the offerings at last weeks SC2004, the conference on high-performance computing, networking and storage, held in Pittsburgh.

The idea of IT formation flight, using such technologies as massive parallelism or cluster computing, has so far been mostly limited to research and specialized applications.

Thats changing.

In a preconference conversation, Dell Engineering Director Reza Rooholamini told eWEEK Senior Editor Jeffrey Burt, "The line between the two worlds has blurred."

In other words, theres less division than there once was between high-performance and data center computing.

/zimages/5/28571.gifClick here to read about IBMs Blue Gene supercomputer that was recently-released to the market.

Conference debuts such as Dells SC1425 cluster node and Hewlett-Packards Unified Cluster Portfolio of hardware, software and services give mainstream IT buyers the kind of manageability and support they expect—even while putting enterprise data centers in the same end of the technology pool as the worlds fastest systems.

The Dell offering includes technology from Toronto-based Platform Computing, whose Platform Rocks cluster management tool kit is based on the San Diego Supercomputer Centers Rocks stack. Thats the same Rocks foundation that enabled a team at last years SC conference in Phoenix to assemble a 128-node Sun Fire V60x system that immediately qualified as a top-500 supercomputer. Dubbed RockStar, the 2-hour construction project debuted at No. 201 on the worldwide list of supercomputing installations at

When I see this kind of cluster technology coming to enterprise markets, I think of a convoy of 18-wheelers. Theyre carrying different loads and working for different owners, but they have a common infrastructure and theyre able to cooperate toward mutual goals—as made famous by the 1975 C.W. McCall song, "Convoy," a CB-jargon saga of massively concurrent speed-trap evasion. Simple communications are all it takes.

And like those highway convoys, but unlike the flashy flying in circles of the Thunderbirds or Blue Angels, an enterprise formation flight needs to know what its delivering and to where.

Technology Editor Peter Coffee can be reached at