Tech2Tech
Applied Solutions
Why Multi-Temperature Data Matters
Managing data by frequency of use offers enterprise-wide advantages.
by Russell Amer
With the enterprise data warehouse (EDW) serving as the central repository for information, users are increasingly requiring access. One of the least understood, and certainly the least managed, facets of an EDW is how often users tap into the information it holds. However, the frequency at which data is accessed—often described as its “temperature”—can affect the performance and capability of the warehouse. Consequently, analyzing and managing data by its temperature can open up opportunities to provide value across the enterprise.
Benefiting Both Sides of the House
Multi-temperature data (MTD) refers to different elements of data being subject to different frequencies of access. (See table below.) Most data warehouses have hot and cold data, so uncovering the potential business and technical value requires some assessment.
From a business perspective, ask:
- Which users, business areas or application areas are causing hot data to exist?
- Which business areas are the most affected?
- How is the service the business delivers to customers adversely affected?
- What business improvements could be gained from better managing MTD?
Optimizing hot and cold data allows processing capacity to be reclaimed in the data warehouse and additional analysis to be undertaken in any given time period. This results in more granular and precise insight and the ability to add processing for incremental improvements to the enterprise.
To realize the benefits that MTD can bring to the technical side, find out:
- Which data in the EDW is hot or cold, and why
- Who is accessing this data, and how
- How the temperature of the data varies over time, and if anything needs to be done about it
- What EDW performance improvements could be achieved by understanding and managing MTD
- How applications should be architected differently to optimize the data’s temperature profile
- What infrastructure costs could be reduced or exploited by better MTD analysis and management
- If a correlation exists between workload behavior and data temperature, and what it shows
Click to enlarge
The ultimate technical advantages include better management of workloads and optimizing user activity while ensuring that the full processing capacity of the EDW is exploited. Additional benefits will be realized through more efficiently architected applications, better use of existing storage and making sure the right users have access to the right data when they need it.
Management Tools
MTD management opportunities can be exploited with Teradata Virtual Storage, which provides an industry-unique approach that stores strategic levels of hot data in the active EDW along with economical storage of large volumes of cold data. Factors such as data’s lifetime, access frequency and type are considered to automatically and continuously place data onto the appropriate storage device: solid-state drives (SSDs) for hot data and hard disk drives (HDD) for cooler data.
Teradata Virtual Storage optimizes data placement for the highest efficiency of the EDW. As a result, the warehouse is enhanced, enabling faster and slower drives to be mixed within the system for maximum performance. As data temperature changes, it is automatically moved to the most appropriate storage location. For further benefit, Teradata Virtual Storage:
- Continuously grades and migrates data blocks by frequency of access and use
- Eliminates the need for database administrators (DBAs) to manu-ally intervene
- Enhances leverage of Teradata Database multi-temperature warehouse capabilities
In addition, Teradata Virtual Storage provides the configuration flexibility and mixed storage capabilities to efficiently use the data across an organization in a single EDW. Since there’s no need for a DBA or operator to manually manipulate the data, the total cost of ownership (TCO) remains low.
Ward Analytics has developed a tool called HeatSeeker, a data visualization-based application, to analyze MTD. It gives analysts, DBAs and architects a detailed understanding of the data in their Teradata systems from a multi-temperature perspective. By enabling a logical view of the data that reflects the business use and dynamics of the warehouse, and a user-oriented view that uncovers the daily activities of the data consumers, HeatSeeker’s analytical capabilities coupled with Teradata Virtual Storage offer the unique potential to improve the performance, TCO and return on investment (ROI) of the data warehouse.
Get the Most From Your EDW
Managing data by temperature can reclaim precious and expensive storage capacity and processing capability, earning more from an existing data warehouse investment and achieving incremental business value through more effective use of the daily EDW processing. Managing MTD can also improve the end-user experience, positioning users to take full advantage of new business and technology improvements while optimizing their current operations and activities. In short, it delivers value for the entire organization.
Russell Amer is the managing director and co-founder of Ward Analytics. He has been working with data warehousing performance management and data visualization technologies for more than 18 years.