Dynamite Data

Dynamite Data logo

Improving MySQL Performance and Access to Comparative Online Product Information with OCZ Deneva 2 SATA-based SSDs



The Focus

  • Reduce and eliminate disk I/O bottlenecks
  • Improve server application performance
  • Accelerate access to stored data

The Benefits

  • Improved database application read performance by 430%
  • Improved database application write performance by 400%
  • Improved overall database application performance by 420%/li>
  • Improved the average MySQL queries per second by 430%

"We put the call out to transition our core databases to solid-state, and only OCZ could give us the reliability and performance our applications demanded."

Kristopher Kubicki, Chieft Architect
Dynamite Data

Deployed Solutions: Deneva 2 Solid-State Drives

Deneva 2 SSDs address disk I/O bottlenecks and other infrastructure issues by delivering a 400% improvement in IOPS performance and a 430% improvement in MySQL queries.

Dynamite Data provides unprecedented retail channel intelligence for merchants, manufacturers and consumers, offering access to the most comprehensive, timely and actionable information. Through its proprietary technology and sophisticated big data collection and manipulation techniques, Dynamite Data automatically extracts real-time data, analyzing and distributing over 30 million extractions per day from nearly a half billion buy pages and more than 3,000 online merchants worldwide. It empowers clients by delivering the most accurate, real-time information on channel pricing, product assortment, map violations, ratings and reviews, and stock availability for the ultimate competitive advantage.

Dynamite Data’s comparative shopping service was originally established to compile electronic and high-tech products sold through online merchant buy pages (‘e-tail’), and was expanded to include home appliances, sporting goods and children’s toys. Invariably, any product sold online could be developed as an individual Dynamite Data database, and be available for comparative shopping or information retrieval. With three billion buy pages available online globally, Dynamite Data’s objective is to data warehouse all e-commerce products for its retail channel.

As part of its original system, Dynamite Data developed a robot (aka ‘internet bot’) to automatically read and extract product data from online merchant site maps. The patented ‘internet bot’ was capable of extracting approximately 20,000 buy pages per minute or 30 to 50 million buy pages per day, and as of this writing, had nearly a half billion buy pages stored in its MySQL database. The products in the system centered on IT and networking equipment, consumer electronic devices, and over time, Dynamite Data was storing approximately three to four terabytes of data each week that became very taxing on the initial network and storage infrastructure.

As more products were added to the Dynamite Data retail intelligence data and the web pages were organized into user-preferred formats, data traffic increased significantly causing I/O bandwidth bottlenecks. The amplified downloading of product comparisons and pricing scenarios resulted in considerable strain on the conventional enterprise hard disk drives (HDDs) they deployed, resulting in non-optimal system performance. It was apparent that an infrastructure upgrade was needed.

The excessive downloading of product comparisons and pricing scenarios by subscribers, database loading at 30 to 50 million web pages per day, and network I/O activity, caused considerable disk strain in enterprise storage, and using conventional HDDs, penalized system performance resulting in disk I/O bottlenecks.

Though Dynamite Data deployed additional HDDs to solve disk I/O bottlenecks, the results were still not optimal. The more hard drives that were added, the higher the costs rose for power consumption, cooling, and maintenance, and since HDDs inherently have endurance and reliability issues, required Dynamite Data to implement RAID capabilities so a redundant data path would be in place in the event of hard drive failures. The storage segment of Dynamite Data’s enterprise became expensive and inefficient, and adding HDDs to improve performance was not the solution.

This case study addresses the storage challenges that Dynamite Data faced initially in upgrading its retail channel infrastructure to new and improved solid-state capabilities that deliver flash data caching, enterprise, endurance, reliability, scalability, and real-time access to product data. By adding Deneva 2 solid-state drives (SSDs), developed by storage leader OCZ Storage Solutions, Dynamite Data achieved significant improvements in server application performance and data access while reducing overall system costs – solidifying it as a leader in the industry with a trusted and indispensible tool for high profile merchants and manufacturers.

Table of Contents

  1. Introduction
  2. Dynamite Data's Network Infrastructure
  3. Infrastructure Issues
  4. Solid-State Drives
  5. Testing and Implementation
  6. Conclusion