Cover Story


Connect the dots

Bring the power of analytics to the front lines for bottom-line results.

Organizations increasingly realize that the information in their analytical data stores can produce new incremental value every time it improves the outcome of a business decision. Many are aggressively scouring their business processes, searching for opportunities to make those events more frequent.

The common thread that connects the following scenarios is the use of information to improve the quality and increase the value of everyday business decisions:

Executive Summary

The trend: Innovative organizations are improving front-line decision support by connecting back-office analytical data repositories directly to their operational business applications.

The results: Benefits include improved decision accuracy, increased efficiency, greater business agility and new opportunities for revenue enhancement and cost reduction.

The keys: Making it happen requires flexible, cost-effective integration architecture; rigorous data-quality control; and a data warehouse optimized for performance and availability.

  • At a semiconductor fabrication plant, a shipment of raw material arrives just in time for the second shift’s first run—and promptly fails a critical quality inspection. A supervisor accesses the plant’s manufacturing execution system, which queries the enterprise data warehouse (EDW) for near real-time inventory, work in progress, demand and delivery commitment data. The supervisor quickly reprioritizes the shift’s production schedule to maximize throughput, value and overall resource utilization, all of which keeps the plant operating at peak efficiency while the supplier addresses the quality issue and delivers replacement materials.
  • At a luxury hotel, an early-arriving visitor discovers her reserved suite isn’t ready for check-in. Glancing at the reservation system, the receptionist realizes that this guest is a frequent and valuable customer at the chain’s other properties, where she often patronizes the spa facilities. The receptionist quickly offers a complimentary treatment while the room is being prepared, turning a potential service pratfall into a customer-bonding opportunity.
  • An eastern U.S. railroad company provides access to a customer portal, allowing logistics managers at thousands of businesses to locate shipments, monitor rail cars and synchronize freight-handling crew schedules with arrival times. Each customer designs, runs and stores a personalized report that queries the railroad’s EDW for operating information that’s updated hourly. Customer satisfaction soars; customer service costs plummet.
  • A leading travel service provider delivers personalized offers and promotions to clients each time they visit the firm’s Web site. Each login generates a query to the company’s customer relationship management (CRM) system and EDW, assembling the very next screen with travel information and service offers pre-scored against the customer’s shopping and purchase history. Not surprisingly, bookings are up, with personalized offers generating four to five times as many sales as generic presentations.

Expert Advice

Improving front-line employee access to decision-support data can benefit the business in many important ways, but you can’t expose an analytical resource to live operational workloads without careful preparation.

Here are four recommendations to consider from Bill Gassman, research director at Gartner, and Dan Vesset, program vice president of business analytics research at IDC:

  • Service-oriented architecture (SOA) smooths the way. Manual integration tools are fine for connecting a few operational applications with relatively static data requirements. But for complex, dynamic integration challenges, an SOA approach can reduce costs and accelerate time-to-value.
  • Prepare to perform. Performance optimization and capacity planning are critical when preparing an analytical data warehouse for operational workloads. Parallel processing, prioritizing workloads and optimizing queries definitely pay dividends.
  • Operational means highly available. Remember, when you connect an analytical data warehouse to an operational business application, the data warehouse becomes an operational application, with an entirely new set of service-level expectations.
  • Data quality is a priority. The more often you expose data to front-line users, the more important it is to apply best-practice data management well upstream. Putting poor quality data in wide circulation ensures the ultimate in unfortunate, unintended consequences.

On the front lines

The key to harvesting these types of incremental earning opportunities is to push intelligence from the back-office analytical environment directly to the eyes and fingertips of front-line users, and to deliver it within the context of their primary operational applications.

“We see a real trend toward providing more operational employees with fact-based decision-support functionality,” says Dan Vesset, program vice president of business analytics research at IDC. “Whether it’s a call center operator, a manufacturing floor supervisor or a retail store employee, we see more organizations providing more decision-support functionality. It might be relevant KPIs [key performance indicators] delivered via e-mail or changing scripts in a call center application driven by an underlying analytic solution, but it’s an important trend because it injects more intelligence into operational processes. We call it ‘intelligent process automation.’

“As business becomes more competitive, organizations need to become more agile, and they do so by using information more intelligently.”

“In marketing applications, that extra intelligence can provide a personalized online experience or pricing that adapts dynamically based on behavioral history and current inputs,” Vesset continues. “In the supply chain, it can improve logistical decisions by pushing routing information to delivery drivers through an in-vehicle notebook or hand-held device. In a call center, it can guide an operator through a customer interaction with purchase history, value scores, prompt information and real-time analytics. These opportunities exist wherever you look for them.”

Turning insight into agility

"As business becomes more competitive, organizations need to become more agile, and they do so by using information more intelligently," asserts Bill Gassman, research director at Gartner. “Agility is the ability to deal with unexpected change—an expensive product recall, for instance. And the question is always how quickly can you respond, because these decisions have to be made in a very compressed time frame or they lose their value. The amount and quality of the information you can pull together to support that decision determines the quality of the decision—and sometimes the fate of the organization.

“It’s the organizations that have command of their data that can adapt to large-scale, unpredictable change,” he continues. “Those are the organizations that are highly competitive today and that will be even more so tomorrow. It definitely shows how back-end systems and front-end systems have to work together.”

“We see a real trend toward providing more operational employees with fact-based decision-support functionality.”

The integration challenge

The challenge, of course, is getting those back-end analytical systems connected to front-end operational systems and preparing those systems to handle high-volume, operational workloads.

“There are lots of traditional interchange mechanisms for getting data into and out of a data warehouse—integration and migration tools, largely manual, that can be used to take data from a data warehouse to populate a table inside an enterprise application for user decision support,” observes Vesset. “But we’re moving toward more service-oriented architectures [SOAs]. Instead of delivering scheduled batch loads of analytic content into the operational application, the application now makes certain on-demand calls to the analytic engine or service. And as they emerge, these service-oriented technologies present new opportunities for automation and for greater returns on investment [ROI].”


SOA is a conceptual approach to software engineering that envisions applications as loosely coupled assemblies of modular service components, referred to as composite applications. Because of their relative simplicity and standardized interface definitions, individual services are easy to build, modify, reuse and recombine. Composite applications can be created and maintained more quickly and at lower cost than traditional monolithic systems, and they adapt more flexibly to new business requirements.

There are many types of SOAs, with different levels of support for application integration. Let’s examine three:

Web services

Perhaps the most familiar SOA is Web services. The standards that define Web services were originally developed to enable interoperation between software components regardless of their location on connected network segments (i.e., the Internet). These standards define uniform ways to describe, locate and access services, providing a framework for interactions that are both highly dynamic and adaptive.

Did you know?

Service-oriented architecture (SOA): An approach that envisions applications as loosely coupled assemblies of modular services. Individual services are simple to build, modify, reuse and recombine. As a result, they easily adapt to new business requirements.

For this example, let’s picture a simple service called “Get Customer Value Score” running on a Web server. When called by an authorized application—in this case a call center application—the service generates a query, which it submits to a data warehouse through a standard data access interface—Java database connectivity (JDBC), for instance. It packages the returned value in an extensible markup language (XML) document and sends it to the inquiring application.

Like all Web services, this one employs a few core standards in all of its interactions:

The Web services description language (WSDL) details its service and access interface for publication via universal description, discovery and integration (UDDI), an online service registry and broker.

The simple object access protocol (SOAP) allows for bi-directional message exchange between service providers and consumers.

XML tags data with self-describing metadata to simplify cross-application exchange.

By exploiting these technologies, Web services can provide relatively simple, flexible and inexpensive data integration services between a call center application—or any other front-line operational application—and an EDW.

“Parallel processing becomes a key characteristic to support these workloads, as does the ability to pull data into the data warehouse in real time.”

Application platform services

Complete frameworks of development resources and runtime services for building, deploying and integrating enterprise applications are provided by application platforms. The platforms offer facilities for accessing a variety of enterprise resources, including middleware, application integration, data sources and directory services. Two application platforms divide the market: Sun’s Java Platform and Microsoft’s .NET Framework.

For example, a bank uses a teller application built on the Java 2 Platform, Enterprise Edition. Management wants to add a function that will insert new product sales prompts in the customer service display, based on transaction history analysis performed in the data warehouse. The new business logic can be developed as a Web service using Java Platform resources for messaging, data access and other services. Then it can be deployed on the same application server that currently hosts the teller application.

In banking and beyond, the tools and services provided within the Java Platform can significantly reduce development time and costs, but they demand familiarity with the programming model and development tools. Microsoft’s .NET Framework provides a similar but significantly different set of resources, with its own skill requirements. For obvious reasons, enterprise IT departments tend to favor one environment or the other. For equally obvious reasons, EDW solutions must support both.

Portals and Web 2.0 technologies

Many large organizations use portals to consolidate online services for internal or external users. A portal is essentially a Web site that hosts a collection of related services and applications that users access through a browser. Often these services provide significant levels of personalization related to the user’s identity, role and interests and are presented visually as charts, dials and reports. A typical application for an internal portal might be a sales performance dashboard that displays current sales data by product line, channel and geography and provides drill-down access to historical detail.

This application could use Web services to retrieve the pertinent data from an EDW, as previously described. Or, because this data is subject to frequent change, the application could be developed to use one of the technologies that have emerged to support more interactive integration and which have been collectively labeled Web 2.0. Several of those technologies—such as really simple syndication (RSS)—enable automated content feeds between Web service publishers and subscribing applications. By combining Web services, syndication and a portal application, organizations can automatically push KPI updates out to concerned users whenever those values change in the EDW.

These three examples—Web services, application platform services, and portals and Web 2.0 technologies—begin to illustrate the diversity of SOA environments, but more important is the standards-based order beneath the superficial complexity. By breaking up software into smaller functional components and standardizing the ways they interact, SOA makes it possible to deliver intelligence to anyone, through any operational application, within the enterprise or beyond. The key requirement is that the data warehouse supports the integration standards and is prepared for the larger workloads.

Working together

Application integration has always been a strategic priority in data warehousing because the business value of a data warehouse grows with every additional user, business process and decision. Extending decision-support data to the widest range of users helps focus investments on integrating the data warehouse with the largest possible library of analytical and operational applications. Organizations have collaborated with Teradata to develop innovative solutions to do just that:

  • IBM has developed IBM InfoSphere DataStage Balanced Optimization, a hybrid solution for large-scale data movement between operational and analytical systems. It distributes batch-processing tasks between the high-speed, parallel DataStage system and the data warehouse, leveraging available processing capacity during off-peak hours to quickly load larger and more complete data volumes into the data warehouse system.
  • Microsoft provides business intelligence (BI) technologies that can leverage the data warehouse for high-speed data access, while providing tools such as Microsoft Office System and SQL Server for analysis and reporting. Microsoft technologies include Excel for easy use by virtually any employee, PerformancePoint Services for deep analysis, Reporting Services for formatted reports, Analysis Services for multi-dimensional analysis, and SharePoint for search, document management and portal functionality. The Microsoft BI suite of tools can be expanded through Visual Studio and Microsoft .NET technology to integrate with custom code or third-party applications. The suite can also support a range of dissemination devices, including PCs, browsers and smartphones.
  • TIBCO has produced active data warehouse solutions that combine the company’s expertise in event-driven business integration with the processing and analytical resources of the data warehouse to deliver real-time decision support across an enterprise. These solutions publish events to virtually any system, load large data volumes in real time and integrate BI directly into legacy system workflows.
  • The open-source community has created a sample application development and deployment solution. The Teradata system is based on open-source technologies such as extensible markup language; Java Servlet API; Apache Tomcat; and Java 2 Platform, Enterprise Edition. The Teradata system also supports a variety of Web application servers and operating systems, so users can make their existing resources available to new uses and applications.


Building operational decision support

Operational decision support inevitably imposes new requirements on a data management infrastructure that may have been designed for strategic, analytical applications.

“If you’re pulling information from an operational data warehouse, then it’s part of your application, and it needs to be treated as such,” Gassman says. “It has to be highly available, and unscheduled downtime may be unacceptable. It’s something that has to be negotiated with the business owners of each application.

“You also need to plan for performance and capacity,” he adds. “Someone has to be looking at the workloads and setting the policies so that the data warehouse runs efficiently and the queries are processed promptly. You may not want to schedule monthly reports when you’re expecting a thousand inquiries a second from the call center. Every application has to have an architecture that supports capacity.”

“You can’t support a call center with the same infrastructure you used to support a few analysts,” Vesset observes. “The queries may be simpler, but you need performance to support the large number of users and large amount of data. I think parallel processing becomes a key characteristic to support these workloads, as does the ability to pull data into the data warehouse in real time. You need query optimizers that can evaluate queries as they come in and decide which ones require real-time response and which ones can safely tolerate slightly longer latency. These all become important characteristics of an operational business intelligence [BI] service.”

And inevitably, operational applications raise the stakes for data quality. “You need to be doing things like master data management,” Gassman says. “You need a master schema for the company. Best practices for data management are what really make integration possible in ways that are efficient and cost-effective. If you don’t have those best practices in place, then things will start breaking unpredictably, and your life will become one long fire drill.”

“It’s the organizations that have command of their data that can adapt to large-scale, unpredictable change. Those are the organizations that are highly competitive today.”

Out of the back office

Whatever the requirements of reinforcing the data management infrastructure or integrating the operational applications, improving front-line decision support is obviously paying off for organizations that seize the initiative. “We did a survey with InfoWorld Magazine last year,” Vesset recalls. “We asked 400 IT and business professionals how long their business intelligence solution would have to be down before it would have a material negative impact on the business. Ten percent said ‘immediately,’ and almost 30 percent said ‘within six hours.’ These are pretty aggressive numbers for a back-office technology. And I think that the next time we ask that question, the number who say ‘immediately’ will increase.”

Smart businesses have clearly realized that the right place for BI is right out front.

Your Comment:
Your Rating:

Fuzzy Logix