eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Its fascinating how true innovation works. Its never right the first time, and it is always built upon earlier concepts and invention. In the hardware space, innovation and improvement have resulted in a steady increase in performance relative to price.
For 40 years this has been so, but what of software price performance?
I believe we are in the early stages of seeing the next giants emerge in the technology space. The catalyst is the increasingly lower costs for storage and server infrastructure and the availability of open-source software.
Companies are now emerging that use these intersecting trends as leverage to create whole new solutions that deliver real tangible value to the market.
Its like watching children playing with Lego blocks; you never know what may be coming our way when really smart people sit down with a basket full of open standards, another of open-source software and another of increasingly cheap commodity hardware.
Mix in some tried and true methods used in the past and some really useful solutions are sure to follow.
The data warehousing space offers some of the best examples of the power and utility that these new innovative companies are delivering.
Just a few years ago, large-volume data marts or data warehouses in the multi-terabyte range were difficult to manage and expensive to acquire. They usually required high-end hardware and software from companies like Teradata or IBM that specialized in these niche solutions, or from a hodgepodge of expensive SMP servers, storage and database software from vendors like EMC, Sun and Oracle.
Needless to say, the cost was prohibitively high, too high for all but the largest of companies.
Innovation has now arrived in the warehouse space —and in a huge way. Netezza and Datallegro are two vendors in particular that have caught the imaginations of a number of companies and are likely to only grow in stature. They are both small emerging companies today, but are examples of what may be the next great lineup of companies in the coming decade.
Both companies are great examples of how smart people have employed open-source software and commodity hardware to deliver potentially game-changing new solutions in the data warehousing space. Both offer specialized warehouse appliances that combine open-source software (operating system and database) with commodity hardware (server and storage) in a single appliance model.
They both seek to solve the problem of how to deliver business analytics against very large data stores (4 to over 20TB) quickly and at a lower price point than solutions that came before them. In some cases, they are able to offer dramatically lower cost and dramatically better performance.
Both use an old technique known as massively parallel processing to take the very large job of reading terabytes of data and convert it into a large series of very small jobs done quickly at the same time. The appliance approach has enabled companies to implement even large proof-of-concepts in a matter of days, where it would have taken weeks with more traditional solutions.
Users have reported improvement in query times that are frankly hard to believe, such as 36 hours reduced to 1.5 hours on the appliances. Not only have these solutions improved the time-to-value ratio and the performance, but they have also managed to significantly reduce the maintenance costs for these environments.
These companies represent what I call the continued democratization of IT infrastructure. Think of the great companies like Wal-Mart and FedEx that have so successfully leveraged business analytics that one can almost make the argument that both companies are now in the technology business, rather than retail or package delivery.
With the data analysis capabilities at Wal-Marts disposal, the company may more accurately be described as being in the real estate business. The real estate happens to be shelf space in an aisle in a store, but the company has the information to determine the relative value of shelf locations, and can presumably drive more favorable pricing from vendor-suppliers that want that prime shelf real estate.
While Wal-Mart has plenty of money to invest in analytic infrastructure, open-source software and commodity hardware solutions from companies such as those I have already mentioned are perhaps putting some of that power within the reach of much smaller companies.
I get excited just considering the impact that solutions like these and others to come from companies yet to be born will have on the world economy and society as a whole. So when people say that open source is not innovation, it is commoditization of what came before it, I believe they are just not seeing the larger picture. Putting blocks together to create compelling new solutions is the best kind of innovation.
Charles Garry is an independent industry analyst based in Simsbury, Conn. He is a former vice president with META Groups Technology Research Services.