“The biggest hurdle is getting the POS and inventory data flowing. Once it does, the amount of data always increases,” says Paul Beduhn, CEO of Vision Chain.


Guest Perspective

Elephants in Your Haystack

Manufacturers take their first crack at mining point-of-sale data.

Recently, manufacturers have started mining point-of-sale (POS) data, following the lead of top retailers that have been doing so for years. Using store inventory and POS data, manufacturers can finally get a complete view of their supply chains and figure out just which shoppers are driving sales and stockouts.

Paul Beduhn, CEO of Vision Chain, the world’s No. 1 provider of demand signal software for consumer goods manufacturers, discussed this trend with Teradata Magazine.

Q: POS data has been in the realm of the retailer for decades. Why would a product manufacturer want to use it?

A: There are three advantages a consumer products manufacturer achieves through using POS data. The first is around sales. These companies will increase sales by improving areas of supply chain inconsistency, such as stockouts or distribution voids, where the consumer would otherwise be tempted to switch at the moment of purchase to a competitor’s brand. So this helps the retailer as well.

Next, the manufacturer will improve promotional effectiveness and sales per product facing (purchases made per view of a product) by understanding and acting on shopper insights found by mining data for patterns and demographics.

And third, the company will have a 360-degree view of its supply chain down to the last mile, especially tying it to demand-forecasting tools it already owns.

Q: How much data is out there, and which retailers are making it available?

A: More than 20 large food retailers share detailed data directly with suppliers outside of the traditional electronic data interchange (EDI). Most life science and pharmaceutical companies have an over-the-counter division selling products in drug stores—these are all sharing direct data. The big box electronics stores mostly share, although that’s coming more slowly. This shared data represents a huge component of the business for manufacturers. They can’t ignore the data anymore; they can’t afford to. These retailers account for 40 to 75 percent of sales for these companies.

There’s an evolution once a retailer starts sharing. The biggest hurdle is getting the POS and inventory data flowing. Once it does, the amount of data always increases. The granularity, or “grain,” becomes deeper, the data becomes closer to daily, and the amount of history or other fields of data increases.

Q: Who is in charge of this data at the manufacturer?

A: As the financial benefits of this data become clear, an organizational issue arises. Dealing with each retailer varies, as they all have different contact people, data frequency, access methods, data security agreements and restrictions. Each retailer’s business program requires management to participate and stay up to date with new data fields and collaboration opportunities.

Each supplier business department (supply chain, finance, trade promotions, et cetera) may not have its own retailer-facing program management process for each retailer. However, central departments of business users—supply chain planning, for example—should leverage as many retailers’ worth of direct data as they can get.

This leads to an obvious need for a single clearinghouse of management, process and information for all key retailers within the supplier. This dynamic will naturally lead to organizations having a “demand data czar” who interfaces with all of the players to enable a smooth flow of benefit. Whether this role will emerge as an IT function or as a business function is not yet clear.

Q: What’s an example of data mining that manufacturers are doing with this data today?

A: There’s definitely a lot more than red light-green light reporting and alerts going on. These POS databases can get huge. Think about it: A large retailer might have a 10TB data warehouse for POS. If a manufacturer is building a data warehouse for 10 retailers, they’re dipping into a pool of 100TBs of POS data. The biggest suppliers will have multi-terabyte databases. With a haystack this big, you’re not going to find the needles with reporting. Needles? You need data mining to find elephants in a haystack that big.

We see a lot of interest in turning shopper insights into customized product sets for each store tied to those consumers. We advocate using a simple proxy, like geography, to transform how demand chains work. In the old approach, stores located near each other are treated as a group, even though shopper tastes and weather events are totally different. Consumers in two nearby regions are likely willing to pay different prices, want different products on the shelves and respond to promotions differently.

"The biggest suppliers will have multi-terabyte databases. With a haystack this big, you're not going to find the needles with reporting."

Some companies are starting to ignore geographic proximity. They want “shopper proximity.” So how do you know which ZIP codes include large numbers of shoppers who like to eat nachos with a sour cream-based dip? You need those details if you want your nacho manufacturer to pay for a promotion so you can make volumes spike on private-label dairy.

Q: Is there a standard format, such as with EDI, that governs how POS data should be formatted and what should be included as it moves from a retailer to a manufacturer?

A: We hear this question a lot. Everyone wants a common standard of data subject areas or the data fields sent from a retailer to the manufacturer. The manufacturers want the retailers to all look the same from a data model perspective so they can have a single extract, transform, load (ETL) routine that fits all retailers in their system.

UCCnet, which provides a global repository for companies to share supply chain information, has done a lot in our space to create a standard for product codes and item master data. It hasn’t happened for POS, and it won’t. Retailers can share a tiny set of fields, like sales and volume, without inventory. Then you have more retailers providing store inventory daily, 52-week forecasts, all kinds of demand signals. At the other end of the spectrum are expansive retailer databases for the supply chain. As a result, among retailers, none of the columns will be the same. They’ve spent time and money creating their own data types, frequency and granularity.

Retailers think about their formats as intellectual property. It is a competitive advantage to share fields that suppliers can use to improve performance. They’ll want to create collaboration initiatives and invite suppliers to participate in them or to use their metrics. However, there’s nothing in it for retailers to have a standard. From their point of view, doing the work to change their method of data delivery, type of data, et cetera, into a standard only benefits the manufacturer.

Q: Are manufacturers actively building these solutions themselves?

A: We see less and less of this. In the 1990s, there weren’t a lot of options, so often legacy systems remain. These are typically hard to manage. It’s also hard to add new retailers since IT needs to start over each time, building ETL code for the unique new data feed. No multi-retailer databases are being handcrafted. A few technology providers have developed and sold solutions to handle all parts of the process. But no supplier I know of is using development tools or hiring integrators to design a system from the ground up. We do see integration of a demand signal repository to internal data, like supply chain shipments or master data.

Your Comment:
Your Rating:

Is there a way for somebody to verify product purchases at the point of sale? Our business is interested in promoting sales of more eco-friendly products, and we would like to verify that those pre-approved products were actually bought at a store somewhere.

5/25/2012 4:10:48 PM
— Anonymous