Keeping the Orchestra Together: Dynamic Software for Distributed Systems

Peter Coffee: As processing is parceled out to dispersed multiprocessor or network nodes, software tools must adapt to changing tempos.

Theres poetry in the process of turning our ideas into machine instructions. Of all the types of product that I get to review at eWEEK Labs, high-level language compilers are therefore my favorites--especially those whose output options include machine-code listings, with the high-level language statements interspersed as comments. I wish that more of our technologies offered more such disclosure of what theyre doing, and how (and even why) theyre doing it.

What brings this to mind was a presentation at last weeks Microprocessor Forum from Masa Motomura, a project manager at NEC Corp., describing the companys Dynamically Reconfigurable Processor design. The device augments an array of processors with supporting logic that can reconfigure them into a task-specific configuration, switching from one to another in a single clock cycle.

If you think its a challenge to build a C compiler for a single processor, deciding which variables will live in scarce on-chip registers and which will live in memory, imagine the task of "compiling" an entire array of processors into a set of data paths and finite state machines. Thats what the compilation tools for the DRP must do; the high-level prototype tools already developed by NEC provide useful visualizations that expose the structure of their proposed solution to a problem.

In introductory remarks before the NEC presentation, In-Stat/MDR Principal Analyst Max Baron (a senior editor of the revered newsletter Microprocessor Report and one of the Forum hosts), observed that the task of compiling software is vastly complicated as processing becomes distributed over distances that are non-negligible compared with speed-of-light traversals during a single clock cycle. "With an array of 512 processors [as provided by NECs DRP-1 prototype], software needs to be aware of when its asking for something at the opposite corner of the array; how many cycles it needs to allow," Baron said. Hes right, but not only on the scale of microchips.

On the much larger scale of distributed systems within an enterprise, let alone the global scale contemplated by proponents of Web services, the choice between what we can get right away and what we can have if were willing to wait must soon become part of our algorithms. More than one path through a problem, depending on the available resources, may need to be simultaneously pursued, so that when the requesting process says, "I cant wait any longer, give me your best shot right now," theres a complete if perhaps not perfect answer on hand.

"Orchestration" is the verb that Im starting to hear, increasingly often, to describe this difficult mission: Its a good word, including overtones of the way that an orchestral director conducts a piece one way in one kind of setting and another way in a different hall with different acoustics. The audience for many kinds of distributed solution is tired of watching an endless open rehearsal: Lets hope that the downbeat comes soon.

Tell me about the time it takes to do the right thing, and do it right.

Reader Comments: Keeping the Orchestra Together: Dynamic Software for Distributed Systems >>> Be the FIRST to comment on this article!

>>> More Application Development Articles          >>> More By Peter Coffee