Database Legend: How Real-Time Data Analysis Will Transform Society

By Lisa Vaas  |  Posted 2005-02-15 Print this article Print

Database pioneer and StreamBase founder Mike Stonebraker foresees vast societal shifts as real-time data analysis changes how we treat the infirm, how we keep our soldiers from harm and even how we locate our wandering bambini at Disneyland.

Mike Stonebraker is a database superstar. Not only is the former UC/Berkeley computer science professor the father of the popular relational databases Ingres and Postgres, he was also the founder of Illustra Information Technologies Inc., acquired by Informix, which in turn was acquired by IBM. The next project for this database pioneer takes shape in the form of StreamBase Systems Inc., a company thats churning out software designed to process, analyze and act on real-time data "within milliseconds of its arrival." Stonebraker is StreamBases founder and chief technology officer. StreamBase announced its Stream Processing Engine at the DEMOConference on Monday in Scottsdale, Ariz. Database Editor Lisa Vaas recently got a chance to talk with Stonebraker about the issue of real-time data analysis, about how it leaves relational databases in its dust and, most importantly, how this cutting-edge technology is poised to transform our society. Financial services comes to mind, of course, but what really fires up Stonebraker are prospects like revolutionizing the care of emergency-room patients, the care of soldiers on the front lines or simply the ability to find your child when shes lost at Disney World.
Youve said that streaming data on the fly is something that ordinary relational databases cant handle. Why?
Heres a quick, simple little problem. This was a pilot we were asked to do early on. [It was] a large, mutual funds company. They subscribe to every feed on the planet, [including feeds such as Reuters]. They have a current application that watches each feed to determine if the data is late, so they can say, "Dont trust Reuters now, the feed is screwed up." They defined "late" as [when the] inter-arrival time of ticks between the same stocks is greater than a certain number. You see an IBM tick, and if you dont see another IBM tick in x seconds, its an indication of late data. They wanted to issue an alarm if you saw a late tick. Then they wanted to say, "If you see 100 late ticks that are coming from the feed vendor, then ring the red telephone." The current application is written on top of bare metal in C++. They were unhappy with the performance of the current application, and it was hard to maintain. And expensive. In addition to StreamBases real-time data analysis technology, DEMO showgoers were treated to peeks at more support for mobility from emerging technology companies. Read more here. On this application, they said, "How fast can you go?" We processed about 150,000 messages per second on this, on a $1,500 PC, a commodity piece of hardware. Their current production application does about 3,000 messages per second. The best we could get out of one of the very popular relational databases was 900 messages per second. Next Page: Elephants store data.

Lisa Vaas is News Editor/Operations for and also serves as editor of the Database topic center. Since 1995, she has also been a Webcast news show anchorperson and a reporter covering the IT industry. She has focused on customer relationship management technology, IT salaries and careers, effects of the H1-B visa on the technology workforce, wireless technology, security, and, most recently, databases and the technologies that touch upon them. Her articles have appeared in eWEEK's print edition, on, and in the startup IT magazine PC Connection. Prior to becoming a journalist, Vaas experienced an array of eye-opening careers, including driving a cab in Boston, photographing cranky babies in shopping malls, selling cameras, typography and computer training. She stopped a hair short of finishing an M.A. in English at the University of Massachusetts in Boston. She earned a B.S. in Communications from Emerson College. She runs two open-mic reading series in Boston and currently keeps bees in her home in Mashpee, Mass.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel