Immelman crowned Masters champion!

Congratulations to South African Trevor Immelman who secured a maiden major title win today with a three-shot victory at the 72nd Masters Golf tournament at Augusta.

If you are a golf fan it was for sure a quality weekend in front of the TV.

If you were immersed in the golf, waiting for Tiger to yet again make a run for the championship, did you ever find yourself wondering where on earth do the commentators get all those performance statistics from that they continuously feed you during their commentary?

Well check out the article on “How the PGA Tour Manages Its Data” to see how “IT Chief Steve Evans relies on legions of golf-crazed volunteers, high-tech lasers and the input of golf pros to help him identify, manage and display the Tour's most critical data.”

Thomas Wailgum’s article provides interesting insight into the efforts made to capture data real-time from the field, then to translate it into interesting statistics that both the general public can hear about to enhance their golf watching experience, as well as players on the course to not just help them track their progress but also help them evaluate their risk exposure when contemplating their next shot.

The system used is ShotLink, a revolutionary system that “tracks every shot at every event—where a player's golf ball starts and lands, and all the ground covered in between.” ShotLink requires the use of over 1000 volunteers out on the course to help “capture” the required data on over 32,000 shots which is feed into the system.

Impressively in the article Mr. Evans states that through some minor modification to ShotLink they have been able to ensure a very high level of data quality in the statistics that they produce. He states that “Our goal is to have any data corrections made inside of one minute, and we consistently meet that metric.”

How accurate is the data that you use? What data quality manaagement process do you have in place? Do you tend to resolve data quality issues upstream at its source, or do you cleanse within the applications your report from, or "tweak" the actual reports?

All-You-Can-Eat Seats

I learned last week that the Pittsburgh Pirates are joining a growing trend across Major League Baseball (as well as other sports) by offering an All-You-Can-Eat seating section during specific games for the 2008 season. Fans purchasing a $35 advance ticket ($40 on game-day) will receive a wristband providing access to a dedicated concession stand and all the hot dogs, hamburgers, nachos, salads, popcorn, peanuts, ice cream and soda they can eat.

I’m interested in the analytics behind this decision and wonder if the following conversation took place:

Marketing Executive: The fans want a winning team.

Baseball Executive: Are you kidding? Have you seen our lineup? What if we gave them unlimited hot dogs?

Marketing Executive: I’ll start working on the spreadsheet…

The seats are normally $17. At the $35 price, you’d need to stuff yourself with $18 worth of concessions in order to “break even” – not a particularly hard thing to do given current stadium prices.

Nutritionists and public-health officials oppose the plan, calling it a “recipe for obesity” as fans try to get their money’s worth by over-indulging. Team officials say they’re getting rid of tickets and making fans happy.

There are 164 seats in the All-You-Can-Eat section at the Pirates’ PNC Park. An advance sell-out would generate about $3000 more in revenue at $35 than at the regular $17 price, but expose a liability of 164 hungry fans trying things like “Let’s have a hot dog every time a Pirates reliever gives up a hit,” which – given the Bucs’ early-season performance - could result in numerous emergency shipments from Oscar Meyer to the Golden Triangle.

This doesn’t strike me (strike, get it?) as a good deal for the team, and potentially has some problems for the fans as well – this Braves fan did some analysis on Atlanta’s plans to offer a similar promotion.

Do promotions like these ever make business sense? Often they are designed to be loss-leaders, enticing customers with a lower entry price with the hope they’ll spend more later. Perhaps these Pirate fans, tired of shelling peanuts while watching their pitchers get shelled, will buy an over-priced souvenir.

Other promotions are designed to attract first-time customers and turn them into repeat customers. So those who can’t get tickets to the Penguins playoff games might say “What the heck, let’s go across the river, watch some baseball, and see if we can eat 5 trays of nachos before the 7th-inning stretch.”

What analytical techniques have you used to evaluate promotional activities – before, during, and after the promotion?

Early results in the 'Burgh are inconclusive. At last Wednesday’s game against the Cubs, 67 All-You-Can-Eat seats were sold. The total attendance was 9,735 so gluttons comprised less than 1% of the crowd. But the game lasted 15 innings, so they had a really, really long time to eat. And, in an amazing coincidence, the Cubs player with the winning RBI was center fielder Felix Pie.

At Red Sox games they play the Dropkick Murphy’s “I’m Shipping Up To Boston” when the closer enters late in the game. The Pirates may need their own version – “I’m Throwing Up in Pittsburgh” if this All-You-Can-Eat craze takes off…

Picture source: Keith Srakocic, Associated Press

Prediction Markets Explained

Last week, Mark Lorence posted a topic for discussion on prediction markets and questioned if it would have been an effective tool for British Airways in the opening of the T5 terminal in London. I wanted to provide a quick follow up, as the topic of prediction markets was in the NY Times yesterday with references to different organizations that are using them to predict sales forecasts, store openings and project completion dates.

If you have an interest in the software and services companies who provide platforms to facilitate a prediction market, you should check out Inkling Markets, Consensus Point and NewsFutures. Each company provides the technology and infrastructure to setup and manage prediction markets. To give you a sense of the type of problems that a prediction market can be used for take a look at the Inkling Markets home page. They have examples such as "Improve Forecasting of Performance Indicators", "Expose Quality Problems" and "Predict Risk in Your Supply Chain". All real issues that many organizations need to address.

If you want to see how a prediction market may work for something like sports, politics or weather, I would recommend taking a look at TradeSports.com. You could argue this is simply a gambling website since the trading is performed with real money, but if you look a little closer you will find some interesting information. For example, according to the current "2008 Democratic Presidential Nomination", the current trading price for Barrack Obama is 85.5, which means that 85.5% of the market predicts that he will be the democratic nominee. In other words, the "market" strongly believes he will be the nominee. If I were to purchase options at this price and he were nominated, I would earn $14.5 per block of shares ($100 - $85.5) I purchased. If he did not receive the nomination, I would lose my entire investment since the value would be $0 when the market expires, which should be August 28th at the DNC.

Bo Cowgill from Google, who is a speaker at the DIG 2008 conference, runs Google's internal prediction markets. He is also an expert on the topic and operates his own blog on the topic of prediction markets. Also take a look at Midas Oracle. We are looking forward to hear Google's story from Bo at the conference.

One interesting side note is that there is a market on TradeSports.com for Google's Lunar X project and if it will be won on/before 12/31/2012. The current price is 24.3.

In the Mood

How are you getting your engines revved for Vegas the DIG conference?

Consider this an open thread to share book and article recommendations related to data, analytics or enterprise 2.0. The poster with the most compelling suggestion will...be treated to their choice of a soft drink or adult beverage at the Green Valley Ranch by legendary DIG conference chair Pete “Memphis Ruined My Week” Graham.

In Search of BI Mashups

An area that has had a tremendous impact on the consumer aspect of the web is the concept of a “mashup”. The history of the term goes back to DJs and mixers combining different songs together to create new music. The term has evolved to more generically represent an application that is built by combining two or more data sources (if this isn’t the definition of a business intelligence application, I am not sure what is). The Senior Director of Engineering at Adobe put it best when he said

“a lot of talk about Web 2.0, web mashups, Ajax etc., which in my mind are all facets of the same phenomenon: that information and presentation are being separated in ways that allow for novel forms of reuse.” - Sho Kuwamoto

The same statement can be applied to enterprise data…separate the organization’s data from the different ways it can be presented. Where mashups come into play is when enterprises start presenting this data beyond grids and charts. In addition, as I have discussed on this blog, enterprises can combine traditional and non-traditional data sources to provide further context. Thus, the case for BI mashups.

If you perform a quick search for examples of BI mashups, you primarily find sample applications from different BI platform vendors. The first example I came across was from open source BI vendor Pentaho, which combines sales data with Google Maps to plot customer performance. Additional examples from Information Builders and Oracle offer similar examples. The trend with the majority of these examples is that they plot spatial data into geographic maps to show enhanced visualization. Not quite what Tufte would recommend, but certainly an enhancement over traditional BI. Adding non-structured data into the mix such as blogs and customer surveys through RSS feeds would enhance the experience even more.

For the technical audience, here is a very in-depth article by Larry Clarkin and Josh Holmes on mashups including examples, architectural components and key considerations when developing your first enterprise mashup. There is a wealth of information within the article, but one of the key elements applicable to a BI mashup is providing “rich visualization of data” for users that they won’t get from a typical chart or grid of data. If you are considering your first enterprise mashup, I would get familiar with this article as a first step.


There are some great resources available if you are looking for more examples of mashups. The most well known site is Programmable Web, which tracks interesting mashups, Web 2.0 applications and new web platforms. And if you have a short attention span and would prefer to see a video, check out this YouTube video.

Shifting Mindsets on BI

Pete Graham recently wrote a post on Using Business Intelligence in E2.0 that challenged each of us to bring business intelligence (BI) into the business conversation (verses creating a business conversation around BI). It was a prickly role reversal for those of us who like to look at the information value chain in a linear fashion beginning with data: data -> information -> knowledge (picture below of basic analytical information systems strategy). However, he provided a gentle but persuasive reminder that our mental mindsets and diagrams need to shift.

Let me explain. The idea of information and its use within business is an old idea, but its mastery reigns rather elusive. There are three core competencies that need to be achieved: Data IN, Information OUT, & Knowledge AROUND.

Data IN
Every time something happens within a business, there exists the opportunity for us to capture a piece of “data” that records its occurrence. For instance, when someone walks into a retail outlet, their visit can be recorded with a date stamp and time stamp. When the visitor buys a greeting card, the transaction is stored, inventory is marked down, and cash can be credited. If the person happens to pay by credit card, the purchase is tagged with the person’s card number. If the customer scanned their loyalty card, the transaction is immediately tagged with their profile information - and on and on. We could go on to name thousands of activities that are tracked within our organizations. These transactions let us know that something has happened!

This is not surprising. We live in a digital world where many of our actions are recorded. The challenge for businesses is to store this point-in-time data in a timely fashion and in such a way that it can be accessed quickly and easily in the future. I call this exercise, the “Data IN” process. This is the opportunity for our organizations to capture all of the happenings within our business ecosystem. Unfortunately, this raw data is unwieldly to the average business person.

Information OUT
Therefore, an organization is tasked with putting this data into context so that users can see an evolving narrative about their business. This narrative helps us to understand the what, when, and how of our businesses and their performance within the marketplace. We get to see the single occurrence (or piece of data) with the context of the business story. This process of transforming data into “information” is invaluable and gives us the digestible analytics to manage, measure, and improve our businesses.

Getting “Information OUT” is achieved by answering both traditional and current business questions with information about the past or with forecasts about the future.

Knowledge AROUND
The last piece of the information value chain is to seize the Aha! moments and business insights and push them out to the organization. For instance, a store manager who sees a declining trend in her customer base may realize that a profound shift is taking place in her market. With the combination of some analytical reporting and some field observation, she may notice that a local competitor has cut deeply into her customer base. This “knowledge” needs to be shared with her organization so that other store managers can prevent a similar decline and so functional groups within the organization can support or assist with planning a response (or change to the business). Our companies have a need to easily and quickly share insights throughout the organization, or broadcast “Knowledge AROUND”.

Today’s E2.0 tools have brought renewed energy to the business conversation represented by the Knowledge AROUND piece of the value chain. Tools like blogging, microblogging, wikis, prediction markets, etc… are democratizing the voice of the market facing parts of our organizations! This is exciting because it allows the conversation that is happening out in the field – between the people in the field and the market (customers, vendors, etc… ) to more effectively influence the information value chain. To Pete’s point, at the beginning of this post, our organizations need to bring BI into the business conversation. If we do, we have the opportunity to consistently adapt to fulfill the needs of our changing markets.

Let’s keep thinking about the paradigm shifts required to bring BI to E2.o. What do you think? What topics should we be discussing?

Information Quality & Master Data Management?

Master Data Management is the process used to create and maintain a “system of record” for core sets of data elements and their associated dimensions, hierarchies and properties which typically span business units and IT systems.

Master Data, often referred to as “Reference Data”, may in your organization take the form of Charter of Accounts, Product Catalogue, Stores Organization, Suppliers and Vendor Lists but to name a few.

In his article “Demystifying Master Data Management”, Tony Fischer uses Customer as an example of Master data and how, if not understood and managed appropriately, can cause all sort of headaches for a company, in this case the CEO himself!

“Years ago, a global manufacturing company lost a key distribution plant to a fire. The CEO, eager to maintain profitable relationships with customers, decided to send a letter to key distributors letting them know why their shipments were delayed—and when service would return to normal.

He wrote the letter and asked his executive team to "make it happen." So, they went to their CRM, ERP, billing and logistics systems to find a list of customers. The result? Each application returned a different list, and no single system held a true view of the customer. The CEO learned of this confusion and was understandably irate. What kind of company doesn't understand who its customers
are?”

So what are the typical barriers that hinder organizations from addressing their master data management problem? My colleagues and I typically encounter four primary barriers:

Multiple Sources and Targets: Reference data is created, stored and updated in multiple transactional and analytic systems causing inaccuracies. Synchronization challenges between disparate systems

Ability to Standardize: Most organizations cannot agree on a standardized view of master data. There are a lack of audit policies that comply with federal regulations

Organizational Ownership: Disagreement within the organization as to who takes ownership of master data management, business or IT. Assignment of accountability with cross-functional processes is difficult

Centralization of Master Data: Organizational resistance to centralizing master data since there is a sense that control will be lost. Challenges to find a technology solution that supports existing systems and the lifecycle of master data management

Organizations that are addressing such barriers typically have a successful master data management process in place that contains the following components:

Data Quality: Focus on the accuracy, correctness, completeness and relevance of dataIncorporate validation processes and checkpoints. Effort is highest in the beginning of a MDM initiative to correct quality issues.

Governance: Cross functional team formed to establish organizational standards for MDM related to ownership, change control, validation and audit policies. Focus includes establishing a standard meeting process to discuss standards, large changes and organizational issues.

Stewardship: Assignment of ongoing ownership of MDM stewardship. Typically MDM stewards are business users. Accountable for the implementation of standards established through MDM governance

Technology: Create an architectural foundation that aligns with the other three components. Implement a technology that centralizes reference data. Align processes with the technology solution to synchronize master data across source and analytic systems

As we can see, master data management is not a one-time initiative but rather a long-term program that runs continuously within the organization. To be successful organizations need to instill an iterative approach that helps develop a program that continuously monitors, evaluates, validates and creates master data in a consistent, meaningful and well communicated way.

What is your organization doing about Master Data Management? Have you had success in establishing a Data Governance program? Who own the process in your organization, IT or the business?