Limitations of Row-and-Column Database Systems
Email
Print
Limitations of row-and-column database systems
Allow me to review four reasons why rows and columns are as inappropriate in a modern IT environment as bringing an Apple II computer to work.
Reason No. 1: Rows and columns are not scalable for modern data volumes
As the amount of data in the enterprise continues to double each year, so do the sizes and numbers of tables. As the tables get larger, queries from analytics applications must scan through an increasing number of rows and columns to find the selected data. If current trends continue, this will become utterly crippling for IT in this decade because tables will be too large to search despite advances in hardware performance. These same trends will also make tables incredibly burdensome to manage.
Reason No. 2: Tables are a full-time job
Today we've come to accept that large enterprises need a team of IT staff managing tables-creating them, loading them, joining them, reading them into memory, scanning them, sorting them, storing them and reorganizing them. All this table housekeeping is becoming increasingly burdensome for three reasons.
First, large volumes of unstructured data must be loaded into tables and indexed. Second, as data volumes grow, so do the number of tables that must be maintained. And third, performance requirements push IT to constantly, manually structure and restructure tables to achieve acceptable performance.
Reason No. 3: Rows and columns were never designed for analytics
The row-and-column format was created before computerized business analytics existed and worked well for transaction processing. Information stored in rows and columns is not inherently useful for analytics and must be indexed. This indexing process creates delays between when data is ingested and when it will become available for query.
Reason No. 4: Rows and columns are a rigid static structure
Row-and-column-based databases are built for specific applications, often before IT knows how those applications will be used. Then, as usage patterns change and the business needs to look at its data differently, the entire table structure must be manually optimized to achieve acceptable query performance. The next-generation of advanced analytics will not be possible with rigid, static row-and-column structures because of the manual overhead of incorporating unstructured data or enabling ad hoc analytic queries.
