Limitations of Row-and-Column Database Systems

 
 
By Charles H. Silver  |  Posted 2010-04-20 Email Print this article Print
 
 
 
 
 
 
 


Limitations of row-and-column database systems

Allow me to review four reasons why rows and columns are as inappropriate in a modern IT environment as bringing an Apple II computer to work.

Reason No. 1: Rows and columns are not scalable for modern data volumes

As the amount of data in the enterprise continues to double each year, so do the sizes and numbers of tables. As the tables get larger, queries from analytics applications must scan through an increasing number of rows and columns to find the selected data. If current trends continue, this will become utterly crippling for IT in this decade because tables will be too large to search despite advances in hardware performance. These same trends will also make tables incredibly burdensome to manage.

Reason No. 2: Tables are a full-time job

Today we've come to accept that large enterprises need a team of IT staff managing tables-creating them, loading them, joining them, reading them into memory, scanning them, sorting them, storing them and reorganizing them. All this table housekeeping is becoming increasingly burdensome for three reasons.

First, large volumes of unstructured data must be loaded into tables and indexed. Second, as data volumes grow, so do the number of tables that must be maintained. And third, performance requirements push IT to constantly, manually structure and restructure tables to achieve acceptable performance.

Reason No. 3: Rows and columns were never designed for analytics

The row-and-column format was created before computerized business analytics existed and worked well for transaction processing. Information stored in rows and columns is not inherently useful for analytics and must be indexed. This indexing process creates delays between when data is ingested and when it will become available for query.

Reason No. 4: Rows and columns are a rigid static structure

Row-and-column-based databases are built for specific applications, often before IT knows how those applications will be used. Then, as usage patterns change and the business needs to look at its data differently, the entire table structure must be manually optimized to achieve acceptable query performance. The next-generation of advanced analytics will not be possible with rigid, static row-and-column structures because of the manual overhead of incorporating unstructured data or enabling ad hoc analytic queries.




 
 
 
 
Charles H. Silver is CEO at Algebraix Data Corporation. Charles has more than 25 years of experience as a successful entrepreneur. Most recently, he sold new media company RealAge Inc. to Hearst Corporation in 2007. Charles founded RealAge in the late 1990s based on a ground-breaking business plan for building revenue and attracting customers. Prior to his nine years at RealAge, Charles built a series of profitable franchises in the retail and real estate markets. After graduating from the University of Michigan in 1981, Charles spent the first few years of his career as a staffer for the governor of Michigan and for a U.S. Congressman. He can be reached at CSilver@algebraixdata.com.
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...

 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Rocket Fuel