eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Pardon my dinosaur scales, but I distinctly remember the advent of the relational database as the universal way to store enterprise data with rigor and reusability. That aging conventional wisdom is challenged, to put it politely, by this months release of a database benchmarking research paper from a team that spans MIT, the universities of Brown and Brandeis, and enterprise data-handling innovators Vertica Systems and Streambase Systems.
If your morning coffee break isnt long enough to get you through this 12-page analysis, Ill share what I found to be an utterly compelling anecdote on Page 2. The authors of the paper report a communication from Eric Brewer, founder of Inktomi, in the 1990s that noted Brewers decision not to use a relational DBMS “when he realized that Inktomi ran exactly one query, a three-way join with constants for the search terms in the user query.” Hard-coding this query, Brewer reportedly found, yielded about 100 times the performance of an RDBMS.
If that doesnt make a few light bulbs suddenly glow above your head, I can think of only two reasons why. Perhaps you got there a long time ago—in which case, congratulations. Or perhaps your reaction to the term “hard-coding” is a panic-stricken look at your IT departments skill set. You may be wondering if the people youve been hiring for their ability to administer Microsofts Exchange Server are up to the task of creating a competitively compelling application—one that makes key business functions run 100 times more quickly than they could with general-purpose packaged software.
As I put the period on that last sentence, I realized that it echoes Ed Posts canonical 1983 essay, “Real Programmers Dont Use Pascal.”
“My first task in the Real World was to read and understand a 200,000 line FORTRAN program,” Post wrote, “then speed it up by a factor of two. Any Real Programmer will tell you that all the Structured Coding in the world wont help you solve a problem like that—it takes actual talent.” And “actual talent” like that, it appears, is currently in short supply.
Language-level and platform-level innovations are currently tending to blur the boundaries between what gets done in the world of the database and what gets done in the world of the application. Hardware innovations, alternatively, with exotic solid-state memory accelerators (or massive quantities, once considered impractical, of general-purpose server RAM) are allowing many applications to run much more quickly without application-specific code.
Frankly, though, it really shouldnt matter whether task-centered optimization takes place at the level of the application, the middleware, the operating system or the hardware. The point is that people are becoming much more free to build a network node that does one thing supremely well, performing that tailoring at any one or more of several levels of the nodes local IT stack—since they can then rely on networks to form confederacies with other specialized nodes.
Coincidentally, I had to interrupt this column at just about this point to take part in a special eWeek Labs TestRun podcast on the subject of our Jan. 8 report, “Security: Next Steps.” Upon returning to the keyboard to finish this, I realized that I had made a point in the context of security that also applies here: Security isnt about the misbehaviors that you prevent, Id asserted in the podcast and in the report itself, but about the capabilities that you ensure.
The same is true for the data-
handling capabilities that your applications need to provide. Relational databases often are extolled for the bad things that they prevent. Well and good, but some of those theoretical benefits may simply not be relevant to what youre trying to do—and the cost of making all bad things impossible may be grossly unreasonable.
When I used the phrase “to finish this” a few lines ago, it had a particular resonance for me because this is my next-to-last eWeek column. As of today, Im doing something else, but I wouldnt want to leave without saying goodbye. In next weeks issue, I hope youll indulge my doing that.