eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
I suspect that the human brain is wired, at its lowest level, to take more notice of speed than of size. This makes sense as a matter of survival: A thrown rock is more dangerous than a mountain. That may be an unfortunate legacy, though, if it means that were too easily fascinated by supercomputers with their increasingly affordable speed, rather than taking an equally vigorous interest in organizing and using their mountains of output.
Supercomputers are no longer just for bomb builders and crypto crackers. Of the top 500 supercomputing sites, tabulated twice a year at www.top500.org, more than 200 are currently classified as industrial. Even the thin-air top tenth of the list, dominated by major research installations, includes at least one obviously commercial site: a 1,081-gigaflops IBM machine—at Nestlé, of all places. Now thats what I call hot chocolate.
Nearing the bottom of last weeks 21st Edition of the Top 500 list, where the crucial three-letter term is ROI rather than Ph.D., I found systems being used in the real world by financial analysts, engineers and network operators. Many of them have told me theyre able to work at a genuinely higher level, pursuing new possibilities rather than merely refining established practices. Its the difference, explained one portfolio analyst, that makes it possible to answer a clients questions during a single conversation instead of the following day.
Well and good, but Id rather pull an answer off the shelf than figure it out all over again—especially in any business, such as pharmaceuticals, where “just as good but different” is a costly approach involving additional testing and documentation. Its nice to be able to answer a question on demand, but even more useful to know that its already been answered.
Vendors, regrettably, have every interest in making it easier to do things over than to find out whats already been done. Supercomputer builders make money by enabling ad hoc exploration rather than offering better tools for mapping where weve already looked. Stephen Wolframs controversial book, “A New Kind of Science,” is most notable—but least recognized—for its contribution to sorting out entire families of problems, using tools like Wolfram Researchs Mathematica, rather than merely solving problems one at a time.
In the worlds semiannual festival of processing performance, system builders compared their rankings on the 21st Top 500 list at the International Supercomputer Conference in Heidelberg, Germany (www.isc2003.org). You, too, can be a Top 500 site: At this months CeBIT America show in New York, NEC displayed a multi-Itanium unit about the size of a stereo receiver. You might squeeze onto the 22nd Edition of the list, in November, with a quartet of these units costing under half a megabuck.
Take note, however, of cautionary comments at the Heidelberg conference by Michael Resch, director of the High Performance Computing Center, in Stuttgart, who spoke on “The Death of SuperComputing Centers and Birth of SuperData Centers?” The abstract says it all: “The key pressing problem has turned out to be to manage and understand the output of high speed systems and instruments. Data have moved to the center of the discussion.”
Resch spotlighted a problem that arises today at every scale, from desktops to massive grids. During the decades since the debut of the IBM PC, everyones extolled the exponential growth of PC clock rates: about 45 percent per year. This has not been enough, though, to keep up with the 80 percent annual growth rate of personal storage. We pay the price of this mismatch when we cant find something that were sure weve already done and wind up doing it again.
Visualization tools such as Inxight Softwares Hyperbolic Tree dynamic browser and data clustering algorithms like those in SAS Institutes JMP statistics suite are the kind of aids that application developers should explore and apply to this task.
In the meantime, dont focus too tightly on the greater bang for the buck that supercomputer salesmen proudly claim. Ask them to tell you the rest of the story. Once their systems show you the way to treasure, will they leave you a map that can help you find it again?
Technology Editor Peter Coffee can be reached at [email protected].