Deep Engine Supercharges MySQL for Big Data, High Performance

REVIEW: Deep Information Sciences brings intelligent heuristics and machine learning algorithms to MySQL implementations to improve the database's performance.

Deep Engine Review 2

Deep Information Sciences has introduced the Deep Engine storage engine, which is designed to maximize the performance of MySQL and extend that open source database to large-scale data operations, such as real time analytics and big data management.

While some analysts claim that MySQL is at a crossroads and is becoming long in the tooth, the simple fact of the matter is that an estimated 80.5 percent of enterprises are using the open source database somewhere in their operations, according to a recent 451 Research Group survey.

Nevertheless, many enterprises are looking to replace MySQL with other database technologies that are better attuned to working with massive data sets and time- sensitive processing. This means the future of MySQL in the enterprise is murky at best.

The Deep Engine replaces MySQL’s native storage engines, such as InnoDB or MyISAM, and brings machine-learning metrics to MySQL. This boosts performance and enables enterprises to better leverage investments in existing MySQL implementations, without costly migrations to new hardware or adopting other database technologies, the company said.

The Deep Engine’s self-tuning algorithms can quickly adapt to new hardware deployments and fully utilize improvements in hardware as systems are upgraded or replaced. With that in mind, it becomes evident that there is much more to the Deep Engine than just being an alternative storage engine.

Deep likes to refer to it as an adaptive database kernel and information orchestration system that seamlessly evolves the foundation of the database. Perhaps, that is a more fancy way of saying that the Deep Engine can perform functions above and beyond what a storage engine typically does.

A Closer Look at the Deep Engine

The Deep Engine strives to bring adaptability, acceleration and scalability to MySQL environments, while also streamlining operations. While that may sound like marketing speak, each of those elements brings measurable value to existing MySQL implementations and even allows MySQL to be used for businesses cases where it was once thought to be inappropriate.

The issue of adaptability has far-reaching implications, especially where MySQL is concerned. Current storage engines, such as InnoDB, rely on static algorithms to move data around, create indexes and so forth.

Deep Engine changes the equation by using adaptive heuristics to determine the best path to deal with that data. For example, systems that offer large amounts of RAM and multiple processors may still be constrained by disk I/O.

Deep deals with that disk performance limitation by shifting processing and I/O burdens over to memory and by executing multiple threads, which in effect brings concurrent, parallel processing to a MySQL process.

That adaptability also applies to situations where hardware changes are encountered or workloads are shifted to different environments, where the Deep Engine can quickly assimilate the new processing paradigm and optimize how it moves data across resources.

From the acceleration standpoint, the Deep Engine introduces technologies such as advanced, high-speed compression, parallel processing, self- tuning algorithms and predictive indexing. Simply put, acceleration goes hand in hand with adaptability to offer performance levels that can reach 100 times greater than InnoDB's performance in certain circumstances.

Where Deep differs from InnoDB and other storage engines is how it handles elements such as compression. The Deep Engine implements testing algorithms to measure the benefit offered by compression, which determine if space is actually saved and performance impacted.

The Deep Engine offers a compression improvement of as much as 80 percent. What’s more, the adaptable algorithms use heuristics to determine available processing and I/O bandwidth. Deep Engine can also initiate additional threads or optimize memory utilization to further improve performance.

As far as scale is concerned, the performance and optimization improvements the Deep Engine brings to MySQL allow the database to scale to theoretically, an unlimited number of records, without encountering the performance and usability limitations found when relying on InnoDB. Deep has tested MySQL with databases containing a trillion records.

Ultimately, scale is about a lot more than the number of records available; scale is also about speed, which is directly related to I/O. Here, Deep incorporates concepts such as over-indexing, which can have a dramatic impact on SQL query performance.

However, indexes increase storage requirements and add overhead to MySQL optimization processes.