Why Teradata
Viewpoints
Leave fast in the dust
Solid-state drive technology gives the green light to supercharged corporate decision making.
by Jim Dietz
Flash memory technology is pervasive in the personal electronics industry in the form of the familiar USB“jump drives” and memory cards or sticks. Now the technology that enables these vast amounts of non-volatile data storage for a camera, cell phone or personal computer has been extended and enhanced to work within enterprise computer environments such as data warehousing systems.
This marriage of the robust capabilities for data analytics and storage with the extremely quick read and write speed of flash memory-based solid-state drive (SSD) technology opens new doors for organizations that need to do more in minutes than others do in a full day or in seconds rather than hours.
What is SSD?
Flash memory semiconductor devices use an approach that permanently stores one or more data bits in a single memory cell, with billions of cells contained in a memory device. SSD technology economically leverages these devices along with robust management logic to provide immediate, direct access to large amounts—hundreds of gigabytes—of non-volatile data from a large array of these devices.
Click to enlarge
In comparison, the primary storage device today, the hard disk drive (HDD), stores data on a magnetic disk that rotates beneath a read/write head mounted on a movable arm. To extract data, the drive must first move the mounted arm and the attached head into the correct position. Then it must wait for the desired data block to spin into place before the appropriate data may be extracted.
As a result, the SSD exhibits much faster data input/output (I/O) capability over an HDD—a real benefit for the heavy I/O use seen in a data warehouse environment. Initial use indicates that SSDs offer more than 20 times higher performance for typical data warehouse read and write usage compared with mechanical HDDs. (See table.)
Game-changing technology
When used in a robust enterprise data warehouse platform, an SSD enables multiple benefits, such as:
- Fast and consistent query response times
- Scanning and aggregation of millions of rows in less than a second
- Loading of data at the rates demanded by today’s high-speed business transactions
This game-changing technology can enable data warehouse analytics speeds that are even faster than what is currently called “near real time.” The technology opens new doors for organizations that need to make the best decisions at speeds that were previously deemed impossible.
Additionally, SSD technology is green. An SSD device uses 40% less energy than an HDD. And one SSD can achieve the same total performance output as 20 HDDs, thereby significantly reducing drive counts and saving significant amounts of energy and valuable data center floor space.
Who needs it?
Any business that depends on “100% hot” data applications—ones that require continuous high-speed sense and respond processing capabilities—is a good candidate for the unprecedented analytic performance and truly operational business intelligence (BI) that SSD technology offers.
Web-based retailers
In many Web-based retail organizations, analysts continuously monitor users’ behavior and buying trends across their Web sites. A large retailer might focus enormous resources on optimizing its online advertising to capitalize on quick surges of interest or the latest fleeting trend, based on its analysis of user activity. Using an SSD-based solution, these kinds of companies can now realize real-time analysis of click activity for by-the-second optimization of messages to help enhance customer spending.
Additionally, the ability to access near real-time data when combined with other sources of customer data can vastly improve the marketing across all channels when used in an integrated Web analytics environment. This more complete view of the customer can lead to a significant reduction in the rate of abandonment of online shopping carts—said to range as high as 50% of all Web shopping activity. Just a one-point reduction in this abandonment rate can result in a revenue boost of up to many millions of dollars for the site.
Manufacturing organizations
Similarly, in a world of Web-based commerce, manufacturing and retail companies need to optimize their allocation and pricing on the fly based on instantaneous demand data. Companies in these industries have long been searching for a way to continuously analyze large amounts of current business data and reduce their decision-making turnaround time from hours to minutes or seconds.
SSD technology enables manufacturers and retailers to meet those goals through:
- High-currency dashboards that facilitate real-time decisions on allocation, purchasing and demand chain
- Vastly increased analytics speeds that permit precise product distribution and optimized pricing for thousands of retail sites based on up-to-the-second demand data
Financial services firms
Financial institutions also stand to benefit greatly from the super fast I/O capability of SSDs, which, for instance, enables a data warehouse with load times that approach streaming media processing speeds. Such capabilities provide a path for financial organizations to integrate near real-time financial data with standard market information to impart:
- Algorithm-driven stock trading in which experts can perform deep analytics of transactions in a single trading session, rather than contend with current overnight lag times
- Real-time credit card fraud detection performed as the bogus transaction is taking place and based on broad profiles of the account and user
- Anti-money-laundering detection enhanced to move analysis from the batch-oriented task it is today to a fast response pattern recognition query
Network providers
Finally, achieving network security means having network providers spotting and stopping fraud or other unwanted activities before the perpetrators have had a chance to do any damage. For instance, tracking a 90-minute window of activity on a large network can translate into an enormous 10TB of data that must be stored and analyzed for suspicious behavior. Using only HDD technology, it’s unrealistic today for a data warehouse to accomplish real-time monitoring and analysis of this critical period. But using an SSD-based hyper-analytics solution, network companies can:
- Analyze all Internet traffic on a network during an actionable period to spot unusual patterns
- Identify rogue IP addresses as the suspect behavior occurs
The future of SSD
Solid-state drives are here to stay, though solution offerings will dramatically evolve, taking advantage of an optimal hybrid approach of SSD and HDD technologies within the same data warehouse system. Such a combination will help companies match the access needs of their data while optimizing cost.
For example, using hybrid technology, “hot” data (which carries the fastest access need) would be stored on an SSD drive while “cold” data (which is used less often with less demanding access speed) would be stored on the lower-cost, large-capacity HDD. The ideal system will have mechanisms to automatically migrate data to the appropriate storage medium based on the data’s usage pattern, or “temperature.”
Companies that achieve the greatest market success have historically used new technologies optimally to achieve efficiencies and deliver new and valuable services and products. The introduction of SSD technology into the IT infrastructure of the business world will offer countless new avenues of success to the company leaders who can harness its blinding speed and vast data analysis potential to achieve their own business goals.
Jim Dietz, platform marketing manager, has been with Teradata for more than 15 years.