Keeping database performance tuned when workloads are relatively predictable is hard enough; pity the poor database administrator who must react to the chaos of infinitely variable queries flowing in over the Internet at unpredictable rates.
Researchers at the University of California at Berkeleys Computer Science division are attacking that problem by working on a new approach to database design that could one day find its way into commercial products. Called the Telegraph Project (after the tie-dyed avenue in Berkeley), the effort is intended to produce a database design that responds to widely varying workloads through what researchers call adaptive query processing optimization.
Basically, the idea is to build a series of distributed data flow engines that deal with queries in different ways. Some—say, heavily pipelined operators—would work better with certain types of queries than others.
Another part of the Telegraph design called Eddies adaptively routes queries or parts of queries to specific data flow engines based on factors such as queue lengths. This process allows the database to adapt to each type of query.
The research isnt close to commercialization yet; the Berkeley team has produced only a prototype and is refining it. However, important vendors are paying attention—for example, researchers at IBMs Almaden Research Center are participating in the Telegraph Project.