Managing data by frequency of use offers enterprise-wide advantages.

Tech2Tech

Applied Solutions

Why Multi-Temperature Data Matters

Managing data by frequency of use offers enterprise-wide advantages.

With the enterprise data warehouse (EDW) serving as the central repository for information, users are increasingly requiring access. One of the least under­stood, and certainly the least managed, facets of an EDW is how often users tap into the information it holds. However, the frequency at which data is accessed—often described as its “temperature”—can affect the performance and capability of the warehouse. Consequently, analyzing and managing data by its temperature can open up opportuni­ties to provide value across the enterprise.

Take Advantage of Data Temperature

Having data at different temperatures in the enterprise data warehouse (EDW) can provide value and ultimately improve warehouse performance and the user experience. The benefits realized from analyzing and managing data by tempera­ture fall into two categories: business and technical.

Business benefits:

  • Lower total cost of ownership (TCO)
  • Greater return on investment (ROI)
  • New business prospects through enhanced EDW performance
  • Increased conformity to service level agreements

Technical benefits:

  • Architectural and operational efficiency
  • Better user management
  • Optimized EDW performance at all times
  • Improved query/batch processing and backup, archive and restore functions

Benefiting Both Sides of the House

Multi-temperature data (MTD) refers to different elements of data being subject to different frequencies of access. (See table below.) Most data warehouses have hot and cold data, so uncovering the potential business and technical value requires some assessment.

From a business perspective, ask:

  • Which users, business areas or applica­tion areas are causing hot data to exist?
  • Which business areas are the most affected?
  • How is the service the business delivers to customers adversely affected?
  • What business improvements could be gained from better managing MTD?

Optimizing hot and cold data allows processing capacity to be reclaimed in the data warehouse and additional analy­sis to be undertaken in any given time period. This results in more granular and precise insight and the ability to add pro­cessing for incremental improvements to the enterprise.

To realize the benefits that MTD can bring to the technical side, find out:

  • Which data in the EDW is hot or cold, and why
  • Who is accessing this data, and how
  • How the temperature of the data varies over time, and if anything needs to be done about it
  • What EDW performance improve­ments could be achieved by under­standing and managing MTD
  • How applications should be architected differently to optimize the data’s temperature profile
  • What infrastructure costs could be reduced or exploited by better MTD analysis and management
  • If a correlation exists between workload behavior and data temperature, and what it shows
Table: 5 Temperatures of Data

Click to enlarge

The ultimate technical advantages include better management of workloads and optimizing user activity while ensuring that the full processing capacity of the EDW is exploited. Additional benefits will be real­ized through more efficiently architected applications, better use of existing storage and making sure the right users have access to the right data when they need it.

Management Tools

MTD management opportunities can be exploited with Teradata Virtual Storage, which provides an industry-unique approach that stores strategic levels of hot data in the active EDW along with economical storage of large volumes of cold data. Factors such as data’s lifetime, access frequency and type are considered to automatically and continu­ously place data onto the appropriate storage device: solid-state drives (SSDs) for hot data and hard disk drives (HDD) for cooler data.

Teradata Virtual Storage optimizes data placement for the highest efficiency of the EDW. As a result, the warehouse is enhanced, enabling faster and slower drives to be mixed within the system for maxi­mum performance. As data temperature changes, it is automatically moved to the most appropriate storage location. For further benefit, Teradata Virtual Storage:

  • Continuously grades and migrates data blocks by frequency of access and use
  • Eliminates the need for database administrators (DBAs) to manu-ally intervene
  • Enhances leverage of Teradata Database multi-temperature ware­house capabilities

In addition, Teradata Virtual Storage pro­vides the configuration flexibility and mixed storage capabilities to efficiently use the data across an organization in a single EDW. Since there’s no need for a DBA or operator to manually manipulate the data, the total cost of ownership (TCO) remains low.

Capacity Savings

Identifying cold or dormant data, then managing it, can result in capacity savings. Purging or archiving this data often frees up a surprising amount of space, which can generate significant financial benefits by delaying or defer­ring hardware upgrades.

Upgrades are usually related to expected increases in required data capacity, often through new applications being added or existing ones being expanded. Solutions such as Teradata Virtual Storage automatically store hot data on a solid-state drive (SSD) for fast access, while cold data is stored on a large-capacity hard disk drive (HDD). Moving cold or dormant data to HDDs allows it to stay in the data warehouse to be analyzed over longer periods. Look for infrequently accessed data in a variety of places by asking:

  • Is the data in the main “base” tables being used as much as claimed?
  • Are user “sandbox” areas and development and test databases dormant or partially dormant?
  • Are old applications fully retired and all data purged or archived?

Analyzing the use of data may show an over-provision in capacity, which could result in paying for support and maintenance that is not required. In addition, analyzing data usage patterns will give a clearer perspective of the storage that’s required for growth and allow better, more effective financial planning.

Ward Analytics has developed a tool called HeatSeeker, a data visualization-based application, to analyze MTD. It gives analysts, DBAs and architects a detailed understanding of the data in their Teradata systems from a multi-temperature perspective. By enabling a logical view of the data that reflects the business use and dynamics of the warehouse, and a user-oriented view that uncovers the daily activities of the data consumers, HeatSeeker’s analyti­cal capabilities coupled with Teradata Virtual Storage offer the unique poten­tial to improve the performance, TCO and return on investment (ROI) of the data warehouse.

Get the Most From Your EDW

Managing data by temperature can reclaim precious and expensive storage capacity and processing capability, earning more from an existing data warehouse investment and achieving incremental business value through more effective use of the daily EDW processing. Managing MTD can also improve the end-user experience, positioning users to take full advantage of new business and tech­nology improvements while optimizing their current operations and activities. In short, it delivers value for the entire organization.


Your Comment:
  
Your Rating:

Comments
Fuzzy Logix