Deep Engine Supercharges MySQL for Big Data, High Performance

By Frank Ohlhorst  |  Posted 2015-04-07 Print this article Print
Deep Engine Review

Deep deals with those issues by incorporating adaptive compression, as well as using multiple threads to speed queries and further enhance performance, resulting in MySQL’s ability to scale up for use with massive datasets.

Hands- on with the Deep Engine

Working hands- on may very well be a misnomer when it comes to the Deep Engine. Apart from installation there is very little, if any direct interaction that a DBA/Network Administrator has with the Deep Engine. Simply put, the Deep Engine works behind the scenes and outwardly appears as little more than a plugin for MySQL.

However, outward appearances can be deceiving. The Deep Engine churns away behind the scenes, bringing easily measurable performance and scalability improvements to MySQL databases. That means quantifying the value of the product becomes a matter of measurement and comparison.

To that end, I tested the Deep Engine’s capabilities on two different servers that were configured with multi-million row MySQL Databases. For testing I worked with MySQL 5.5.40 with built in InnoDB plus the Deep Engine plugin.

Testing Specifications

Database performance testing was accomplished using the guidelines set forth in two public domain testing documents, which can be found at:

Two systems were used to conduct the testing, a hosted system and a local system, which were configured as follows:

  • A Softlayer hosted bare metal server configured as follows:
    • Ubuntu Sever 12.04
    • 12G RAM
    • 24 CPU (3.06 GHz)
    • 100GB SSD (Enterprise grade - Micron RealSSD P300)
  • A local server
    • CentOS 6.4
    • 256 RAM
    • 48 CPU (3.0 GHz)
    • 6TB HDD Raid0 (Seagate 5900 RPM)
Four distinct types of benchmark testing scenarios were used, as outlined below:

sysbench (OLTP):  a classic initial test all customers run to measure max transactions per second.

dbt2 (OLTP/TPC-C):  a classic transaction test example using an online shopping cart that runs customer/order/items/warehouse schema.

dbt3 (OLAP/TPC-H): a classic analytic test applying customer/order/items/warehouse schema.

Iibench (HTAP “like”): a test that involves heavy ingestion of new data with many indexes and queries.

Testing Results

The first test was designed to measure the ability of the DEEP Engine to adapt to workloads and optimize transaction processing. I ran two test iterations using the Sysbench Benchmark. Both iterations were configured with default settings and consisted of 24 clients accessing 5 million rows from 100,000 requests using a uniform distribution type.

The Deep Engine showed a 23- fold improvement over the InnoDB storage engine with Deep handling 8826.84 transactions per second, while the InnoDB storage engine processed 379.69 transactions per second. The test proved that Deep’s adaptive configuration automatically scaled performance up to maximize transactions per second. While InnoDB would require “tuning” to exceed its out of the box performance capabilities.

The second test focused on creating a best- case hardware and configuration scenario for InnoDB, while leaving the Deep Engine to function using its own defaults and with Deep’s adaptive capabilities suspended. The test was used to demonstrate if the Deep Engine offered any performance advantages when relying on algorithms that leverage cache.

Both iterations were configured with default settings and consisted of 24 clients accessing 5 million rows from 100,000 requests using a uniform distribution type. In that scenario, deep was able to roughly double the transaction throughput when compared to InnoDB. Deep mustered 3123.23 transactions per second, while InnoDB processed 1534.45 transactions per second.

The third test was designed to judge data ingestion, a common procedure used when creating very large databases for analytics purposes, such as big data projects. The test used 25 clients to add 50 million rows of information to a database through typical queries. Deep produced an 18 fold improvement over InnoDB, taking 200 seconds to complete the ingestion process, while InnoDB took 3750 seconds.

The Bottom Line

The Deep Engine offers easily measurable performance improvements for those using MySQL as their primary database. Installing the Deep Engine Plugin is very easy and takes no additional configuration, changes to code or other programing or administrative overhead, which makes it the perfect choice for those looking to scale MySQL solutions to even the largest use cases.  Using Deep Engine can potentially delay or prevent the need to transition to much more expensive database systems to tackle big data or other data intensive projects.



Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel