Tech2Tech
Insider's Warehouse
Perfect match
Business process management and BI form a synergistic relationship.
by David McGoveran
Business process management (BPM) and business intelligence (BI) are natural-born partners. BPM, with its support of standards such as Web services, is increasingly used to design, manage, monitor and optimize business processes. A well-implemented BPM solution will make efficient use of BI resources, from complex analytics and data mining to strategic planning and operational, near real-time decisions. On the other hand, an enterprise-wide BI solution can make excellent use of BPM technology, from data flow and process definition and management to real-time monitoring of activities and key performance indicators (KPIs). As IT organizations redesign processes using Web services managed by BPM, it is the ideal time to add the long-missing element—real-time analytic data, otherwise known as pervasive BI.
What is a business process?
Beginners often assume BPM equals workflow management, but a business process is much more than a workflow. A business process is best understood as an interconnected and ordered set of activities and decision points, often represented visually as a graph. A business process may be complex, comprising multiple synchronous or asynchronous sub-processes and interacting with other business processes. Processes are differentiated primarily by the business goals they are intended to achieve. Activities (tasks or operations) may be manual, semi-automated or automated and are both consumers and producers of data.
Transaction processing by an enterprise application, for example, is an automated activity. Although transactional data output is typically written to a database, a BPM system (BPMS) will transfer it to downstream steps as well. Interconnections among process components convey both the execution order (i.e., control) and the flow of data (including process state), documents, material resources or other objects among activities. Decisions, captured as changes in process state or data, affect routing and control. Implementing such interconnections with an asynchronous messaging and routing subsystem can ensure robustness.
Not all processes are transactional or automated. Strategic, administrative and support processes are common examples. Some may not even have well-defined transaction semantics. These are often characterized as informational processes. Planning and reporting processes commonly fall into this group, as do most IT processes.
Activities such as data extraction, transformation, load, cleansing, analysis and reporting are processes that BI experts know well. Today, BI is evolving beyond its traditional focus on summary or historical data. Pervasive BI (sometimes called “real-time BI”) products provide BI services for extremely small—even transaction-level—units of processing. When used to provide closed-loop decision support, these products fit directly into BPM, which can accommodate near real-time events and data associated with a single event.
Regardless of business process type, decisions are key to providing the responsiveness for which BPM is famous. In essence, decisions are implementations of business rules reacting to data from the process or data warehouse. Business rules can affect both the choice of subsequent activities and their operational parameters, including responses to errors or critical conditions. Decisions that control processing alternatives fan out to multiple possible paths. Thus, such decisions act as a routing switch in a process flow.
Integrate BI and BPM
BPMS products provide access from within processes to relational and other popular data sources. If analytical information is exposed via a Web service, it is easily integrated with a BPMS. Master data management (MDM) products also offer easy ways to add Web services to a BPMS model. BI and BPM can be used together in numerous ways:
1. Storing and analyzing historical business process data
BPMSs generate massive amounts of activity data, process performance data and message data, well beyond that collected in traditional online transaction processing databases. Analysis and data mining of BPMS performance and message data can be complex, so its value is often overlooked by those unfamiliar with BI. BPMS experts often use business activity monitoring (BAM), balanced scorecard tools or dashboards for process intelligence, but these are rarely capable of handling problems on the scale of historical process data. Active data warehouses and BI analytical tools are designed for this purpose, providing improved scalability and performance. So a better approach is to collect that BPMS data and put it into the data warehouse for analysis.
2. Process-centric analysis
The marriage between a BPMS for control of data capture and BI for powerful analysis techniques can lead to a more rapid return than either technology alone. A BPMS is designed to capture real-time business events with appropriate time stamping and other correlations. Accurate predictive trending of many KPIs requires an understanding of such non-uniform time distributions and is critical for root-cause analysis of process latencies. Thus, BI plus BPMS could calibrate forecasts and KPIs based on the frequency, dates and times that process events typically occur.
3. Operational BI
Traditional BI analysis focuses on data, independent of the process in which it was produced. However, data is more valuable in its process context when decision strategies balance near-term volatility with operational and strategic goals. Using a BPMS as a mediator, integration of the active data warehouse, analytics and process execution can be flexible (via a call-out or as a service) and bi-directional. The result is the closed-loop performance management promise of the active data warehouse.
4. Complex data integration (ELT/ETL/MDM) analysis
How do you know if your extract, transform and load (ETL) processes run efficiently or consistently? Eventually, data integration processing should be managed by BPM. This will provide ETL governance, fine-grained monitoring and automated error recovery, while offering better performance insights. BPM also provides “what if” simulation of process changes before implementation to achieve the best improvements. Of course, once managed by BPM, data integration processes can be connected to other processes more easily as well.
5. BPM process lineage
The summary rollup tables in BI derived from process data tell only half the story. With BPM, the designer can create “process metric rollups.” These rollups maintain the process context of time, location, process identification, and people at different levels of summarization. BI tools can then be used to create and manage process metrics lineage, which are aligned with the levels of process abstraction used in BPM. Such alignment makes rollups and aggregations possible and makes it easier to correlate process data with warehouse data and aggregates. Enriching “data lineage” with “process lineage” makes it more meaningful to business managers.
6. Exploiting BPM and BI synergy
The fundamental BPM value propositions are reliable business objective achievement, closed-loop process monitoring and control, process agility, and lower-cost integration. By comparison, the fundamental BI value propositions are higher data quality and more timely and accurate decision making, preferably in the context of business objectives. These value propositions are compatible, interdependent and synergistic. Using BAM technology, BPM focuses on how best to perform those activities to achieve a process objective. BI focuses on enabling and fine-tuning the decisions that mediate activities in a process. Its potential value for identifying and optimizing the business rules that implement those decisions and for providing high-quality data input to those rules is an essential element of closing the management loop.
Reach your objective
While a BPMS permits processes to change within days of a business event, the decision making and planning involved in making such changes depend on clean, reliable data. This is where an active data warehouse comes into play.
New or modified processes require the collection of new data and the analysis of new correlations as an integral part of process design. The BPMS can then draw upon the data warehouse and BI analytics for improved decision making in a process context. When the BPM designer can seamlessly use the data warehouse and BI analytic services to correlate effects of process change on the achievement of strategic goals, a BPM implementation will have gone a long way toward achieving its real potential.
David McGoveran is principal and senior analyst with Alternative Technologies (www.AlternativeTech.com).
Photography by Alamy