Worst Practices in Business Intelligence

I came across this white paper from Information Builders on worst practices in BI. Aside from having to register to download the white paper, Kevin Quinn does a good job highlighting the 4 key areas of failure:

Worst Practice #1: Assuming the Average Business User Has the Know-How or Time to Use BI Tools

This issue is very accurate. Even with the advances in user interfaces and hiding of the complexity to aggregate and join information, the tools are still overwhelming. What is interesting is what people are usually migrating from is desktop productivity tools like MS Access or Excel (see worst practice #2).

Worst Practice #2: Allowing Excel to Become the Default BI Platform

Excel becomes the default BI platform inherently because of worst practice #1, meaning the tools are too complicated. The simplicity, flexibility and ease of use along with the overall ubiquitous familiarity of Excel makes it the default tool of choice. Typical organizations at some maturity point will move from something like Excel to a standard tool (see worst practice #4). Nothing can be more frustrating then purchasing a BI platform and having to fall back to the legacy solution, which can be error prone and unmanageable.

Worst Practice #3: Assuming a Data Warehouse Will Solve All Information Access and Delivery

Data Warehouses are meant to store data, in whatever preferred design approach that makes sense (hub and spoke, data marts, enterprise warehouse, federated, etc). The BI application implementation should leverage this single view of the organization's data. The point that is made in the white paper is that the information needs typically go beyond what is housed inside the warehouse. The BI platform should have the flexibility to consume these non-warehouse sources. As much as I agree with this point, it can be inherently dangerous to use sources of "unknown nature". That's not to say that all data outside the warehouse is not trusted, but consistency in the usage of this information across the enterprise needs to be assessed.

Worst Practice #4: Selecting a BI Tool Without a Specific Business Need

Absolutely (see worst case #1 and #2). From my experience, the most successful selection and implementation processes have happened when both the business users and the IT group run the selection process together. I have also seen where one group runs the process independently from the other, the "opposing" group rejects the selection. So it is not just the business users that need to make the selection.

What additional worst practices have you encountered? The list is endless, but it would be great to start sharing other experiences individuals have.

Making Business Intelligence more “Social”

A couple of months back, I had written a post on the idea of Using Business Intelligence in E2.0. In the post I discussed how to bring BI into the social aspects of Enterprise 2.0. George Veth also posted on the topic of Social Objects, which is the concept of the “topic” that two people have a conversation about, opposed to the discussion just being about two people talking. There are many examples of this, both in personal and business settings. Wine is a good example of a social object that spurs a lot of dialog. My wife and I had dinner on Saturday night at a Portuguese restaurant, where a good amount of the discussion with our friends was on the topic of the sangria that we had ordered.

So, in the continued quest to figure out how to enhance the value of Business Intelligence I am looking for suggestions, comments and ideas on how to make BI more “social”. Much of today’s BI systems are what I would consider one-way, meaning a majority of solutions produce content to consumers and it stops there. There is no doubt that people are socializing what they are seeing, but not in a way that we are seeing other social computing being applied inside the enterprise. It can thus be said that BI is like Web 1.0 in the sense that a majority of the content is produced by a small group of “users” and consumed and that’s it.

So how can we make BI social?

The Bloglines are open and there are no bad ideas here. I am hoping to spur some discussion and ideas to advance the cause. Maybe we can get to BI 1.5 by the end of the summer.

Business Intelligence:”I’m not dead.”

If you are a fan of Monty Python, you will recognize this line from “Monty Python and the Holy Grail”.

I had a bit of a positive experience today with a client that caught me a bit off guard. It may be because there are times in this industry where you feel business intelligence has become a commodity. Seems like every organization owns at least one, and in most cases, multiple BI tools. There is pressure in the consulting industry from offshore resources, which I am personally not convinced about for BI applications. So needless to say, at times I am a bit jaded.

So that brings me to today. I am working at a client to build out a new reporting architecture. It’s a pretty significant size initiative and will replace numerous disconnected reporting packages that are managed through a hodge podge of manual processes. We are using the Cognos 8 reporting platform, specifically Report and Query Studio. As part of design, we have been prototyping a reporting package that is currently being done in Excel and packaged into PowerPoint. There are a lot of charts, formatting and commentary that goes into the package. Thus far, I have been thoroughly impressed with the Cognos tools to replicate the current set of reports. But the shocker was the level of excitement from business users who are currently saddled with creating these reports manually. And on top of that, the flexibility to expose the dimensional model would support a majority of the ad-hoc requests that the group receives.

So the point is I guess people are still not getting what they can from their investments in business intelligence and corporate performance management technologies. A majority of organizations are still in the 80/20 position of spending 80% of their time gathering data, creating highly formatted reports manually. If that is the case, it is hard to be called a commodity. I guess the software could be since the tools are there, but organizations haven’t yet realized the value from the investments.

There is another scene from the Holy Grail that draws parallels to the current state of BI.

King Arthur: Go and tell your master that we have been charged by God with a sacred quest. If he will give us food and shelter for the night, he can join us in our quest for the Holy Grail.
French Soldier: Well, I'll ask him, but I don't think he will be very keen. Uh, he's already got one, you see.

(Replace King Arthur with software sales rep and Holy Grail with BI tool)

Back in the Saddle

After taking a couple of days off, I have found that I have a bit of writer’s block. Between responding to the mountain of emails that piled up last week and getting back into the blogging groove, I am a bit stuck. That being said, I came across a couple of articles that I wanted to post some thoughts on.

There seems to be a lot of chatter around mobile BI. I came across 3 separate articles in the last week related to analytics being delivered to mobile devices. The first was an announcement by IBM that they will deliver business solutions through new BlackBerry devices. This includes the Cognos 8 Go software. The second article was from Intelligent Enterprise and had three case examples of businesses that are using mobile devices to deliver BI. The interesting thing I found with this was the simplicity of the applications, which seems critical when dealing with a small interface and only needing specific KPIs. The final announcement is a clinical specific application built by Vaultus Partners and Covisint that delivers patient dashboards to mobile devices.

One trend in all three of these articles is that BlackBerry is the device of choice. Not surprising since RIM has a strangle hold on the enterprise mobile market.

If you read through these and are wondering “How the heck can I be thinking about mobile BI?” when you can’t even get desktop-based reporting, don’t worry. Eventually, you will get to the point where these type of solutions will be possible.

The Data Integration Challenges and BI (Part Two)

In Part One of this topic Brian introduced some of the key data integration challenges for a typical BI engagement and left off by highlighting some of the specific data integration challenges that included:

(i) Transformation of data that does not meet expected rules (contents of data elements and the validation of referential integrity relationships for example)

(ii) Mapping of data elements to some standard or common value

(iii) Cleansing of data to improve the data content (for example to cleanse and standardize name and address data) that extends the data transformation process a step further

(iv) Determining what action to take when those integration rules fail

(v) Ensuring proper ownership of the data quality process

In this second part of the article he takes a little deeper into several of these components.

Data transformations may be as simple as replacing one attribute value with another or validating that a piece of reference data exists. The extent of this data validation effort is dependent on the extent of the data quality issues and may require a detailed data quality initiative to understand exactly what data quality issues exist. At a minimum the data model that supports the data integration effort should be designed to enforce data integrity across the data model components and to enforce data quality on any component of that model that contains important business content. The solution must have a process in place to determine what actions to take when a data integration issue is encountered and should provide a method for the communication and ultimate resolution of those issues (typically enforced by implementing a solid technical solution that meets each of these requirements).

As Organizations grow via mergers and/or acquisitions, so too does the number of data sources and eventually lack of insight into overall corporate performance. Integration of these systems upstream may not be feasible and so the BI application may be tasked with this integration dilemma. A typical example is the integration of financial data from what used to be multiple Organizations or the integration of data from different geographical systems.

This integration is a challenge. It must consider (i) the number of sources to be integrated, (ii) commonality and differences across the different sources, (iii) requirements to conform attributes [such as accounts] to a common value but retain visibility to the original data values and (iv) how to model this information to support future integration efforts as well as downstream applications. This task is indeed a challenging one. All attributes of all sources must be analyzed to determine what is needed and what can be thrown away. Common attribute domains must be understood and translated to common values. Transformation rules and templates must be developed and maintained. The data usage must be clearly understood especially if the transformation of data is expected to lose visibility into any data that is transformed (for example if translating financial data to common charts of accounts).

Making Information Accessible to Downstream Applications

With this data integration effort in place, it is important to understand the eventual usage for this information (downstream applications and data marts) and to ensure that downstream applications can extract data efficiently. The data integration process should be designed to support the requirements for integrating data, that is to support the data acquisition and data validation/data quality processes (validation, reporting, recycling, etc), to be flexible to support future data integration requirements and to support historical data changes (regardless of any reporting expectations that may require a subset of this functionality requirement). The data integration process should also be designed to support the push or pull of data in addition. With that in mind the data integration model should provided metadata that can assist downstream processes (timestamps for example that indicate when data elements are added or modified), partition large data sets (to enable efficient extraction of data), reliable effective dating of model entities (to allow simple point in time identification) and be designed consistently.

The data integration process may at first seem a daunting process. But by breaking the BI architecture into it’s core components (data acquisition, data integration, information access), developing a consistent data model to support the data integration effort, establishing a robust exception handling and data quality initiative and finally implementing processes to manage the data transformation and integration rules, the goal of creating a solid foundation for data integration can be met.

DIG Bits & Bytes

I had a couple of interesting articles come across the “ether” that I felt were newsworthy enough to post and comment on. They each hit on the themes of DIG: Data IN, Information OUT and Knowledge AROUND (The articles are not listed in that order):

Outsourcing your data warehouse

In this article on TWeb by Jannie Strydom, the idea of outsourcing an organization’s data warehouse is proposed. The primary drivers are around lack of skills to properly maintain and keep the warehouse relevant to the business. As much as I agree with outsourcing the components of a warehouse that are repetitive and process oriented (loading data, maintaining production processes, fixing errors), it is a slippery slope to outsource aspects that are critical to meet the business needs. A strong understanding of an organization’s business model and needs should be weighed heavily against the value gained (typically cost savings) by outsourcing certain aspects of a data warehouse, especially those that can help facilitate better management of performance.

More on Enterprise Mashups

I made a post a few weeks back on BI and enterprise mashups. This news story came out of the O’Reilly Web 2.0 Expo that caught my eye because of the mention of integration with Excel. In particular, the article discusses going beyond the geographic mashups being done with Google Maps and starts “mashing” multiple external data sources for enhanced analytics inside of Excel. For example, pulling competitor data directly into your own organization’s performance (I am working at a client where modeling this in their data mart has become a bit of a challenge). Two software companies that are mentioned in the article that are providing these type of mashup services and software are Kapow Technologies and JackBe Corporation. I have not kicked the tires on these two products but they sound extremely valuable to a business user trying to consume multiple and different types of data sources into a single “view”. I would like to understand how these products might fit into an overall information architecture from a consistency and “one version of the truth” perspective.

Business Intelligence and My Carbon Footprint

This one builds on the “everything must be environmental” and “green movement”. At the WTTC Global Travel & Tourism Summit in Dubai, Travelport announced their new Carbon Tracker reporting tool. It is designed for travel agencies and corporations to track their carbon footprint when it comes to corporate travel. It provides different analytic views using standard environmental calculations. The reporting tool includes travel budget and environmental impact analysis and comparison to other modes of travel (car, bus, train, flying). There is a slick product overview with screenshots on the Travelport website. I am considering using this tool to calculate the carbon footprint of DIG in Las Vegas versus another location in the US. I may need to recommend that the speakers ride bicycles to the event to reduce our environmental impact. I know Mark Lorence would be up for it (Mark is an avid bicyclist enthusiast that continues to educate me on the nuances of professional cycling…we have a prediction market already established on his first post that links Lance Armstrong to Business Intelligence). I may need to start referring to the first theme of DIG as "An Incovenient One Version of the Truth".

The Data Integration Challenge and BI (Part One)

This week I've asked a collegue of mine, Brian Swarbrick, to provide insight into some of the typical Data Integration challanges faced when developing Business Intelligence solutions. Brian, an expert in large scale data warehouse and data integration initiatives was so enthusiastic on the subject that we have decided to split his blog into two parts!! Next week I will publish Part Two. Thank you Brian!

The Data Integration Challenge and BI
The goal of any BI solution should be to provide accurate and timely information to the User organization. The User must be shielded from any complexities related to data sourcing and data integration. It is up to the development team to ensure that they deliver a robust architecture that meets these expectations.

The most important aspect of any BI solution is the design of the overall BI framework that encompasses data acquisition, data integration and information access. There are challenges in designing each of these components correctly but often data integration is the one that is the most complex yet important component of the BI solution that must be developed. A solid architecture is required to support the data integration effort (see Claudia Imhoff’s article on why a Data Integration Architecture is needed).

So what are some of key the challenges and considerations that should be addressed when thinking about data integration?

First, unless your project is tasked with building "one off’’ or departmental type solutions, it is important to separate the integration component of the architecture from the analytical component (this is the point where some readers may disagree, but separation of these components allows for a more flexible and scaleable architecture over time – a must for any Enterprise solution today). With this rule in place, the data integration team can focus on what they do best (data integration) and the analytical team can focus on what they do best (designing for reporting and analytics).

With this structure in place the data integration team has some tough challenges ahead of them that must be addressed:

(i) Identifying the correct data sources of information

(ii) Identifying and addressing data quality and integration challenges

(iii) Making information accessible to downstream applications

Identifying the Correct Data Sources of Information
Before data can be integrated it must be identified and sourced. As simple as this sounds it in not unusual for an Organization to have multiple sources of the same data. It is important to identify the data source that is the true ‘system of record’ for that information, contains the elements that support current information requirements and can extend to support future information requirements. Choose the data source that makes the most sense and not the one that is the easiest to get to.

Once the appropriate sources of information have been identified, the integration team must then determine how best to access that information. The team must identify how often the data needs to be extracted (once a day, week, etc) and how the data will be extracted (push or pull, direct or indirect). The frequency should be based on future as well as current requirements for information. It is easier to build based on what is required for today than for what may be planned or needed tomorrow. Data volumes should be a consideration when determining the optimum acquisition method and often a more frequent data sourcing process may be beneficial irrespective of the final reporting expectations (this is a good example of where separation of integration and analytics has merit since the data integration layer can be designed for optimum integration without impact to the requirements of the analytic environment).

Getting at the data itself is often more politically challenging that technically challenging. Source data may exist in internally developed as well as packaged and externally supported applications.

Pull paradigms are good when:

(a) Tools are available that can connect directly to the source systems (that’s a given) and when needed provide options for change data capture mechanisms

(b) Access to the systems is allowed; just because you can connect to a source system does not mean that the IT organization will allow that to happen – these solutions can be invasive and direct access may not be welcomed or allowed (so make sure you consider this)

(c) Source volumes are small and all data is being extracted in full or there is a means to identify new or changed records. The latter is a definite consideration when data volumes are large but there must be a means to identify these changes and it must be reliable and efficient else source invasiveness becomes a concern (especially if the source system must perform tuning to support these downstream processes)

Push paradigms (even when enterprise tools for pulling data are available) are good
options when:

(a) Data with the desired granularity, frequency and content is readily available in a different format and can be leveraged

(b) Direct access to source systems is not an option and/or the IT prefers to source the data that is needed. In this scenario a solution for change data capture may need
to be developed

(c) It is easier for IT to identify the data to be pulled and provide it instead of downstream applications pulling the data directly

Before determining the best choice for your project you also need to consider the limitations of the tools available within your environment

Identifying and Addressing Data Integration Challenges
Once the method for data acquisition has been addressed, data must be cleansed, transformed and integrated to support downstream applications such as data marts. So what does this mean and what are the potential challenges?

The size of the data integration effort is dependent on several factors: (i) the number of data sources being integrated and the number of source systems from which data is provided (ii) quality of data within each of those systems, (iii) quality of data and integration across those source systems, (iv) the Organization’s priority for improving data quality in general. When integrating data the Organization has the choice of enforcing data quality during the integration process or ignoring it.

So what are some of the key challenges for a typical data integration effort? These typically include:

(i) Transformation of data that does not meet expected rules (contents of data elements and the validation of referential integrity relationships for example)

(ii) Mapping of data elements to some standard or common value

(iii) Cleansing of data to improve the data content (for example to cleanse and standardize name and address data) that extends the data transformation process a step further

(iv) Determining what action to take when those integration rules fail

(v) Ensuring proper ownership of the data quality process

So what are some of the challenges and considerations within each of these areas? Tune in to Part Two of this article when we will address some of these considerations as well as addressing the need for making information easily accessible downstream of the integration process.

Can I get the Consumer Reports for these Appliances?

I wanted to pull together a quick summary of the Data Warehouse and Business Intelligence Appliance space. It is a continually maturing space with a set of strong vendors still fighting for market share. Teradata, DATAllegro, Netezza, NeoView from HP and Dataupia all provide solutions that combine hardware, operating system and database software into a single unit. Calpont, Kognitio, Vertica and ParAccel provide software only and platform independent solutions. The benefits of these appliance solutions include a reduced total cost of ownership, increased performance through massively parallel systems, reduced administration and database administrators, and high availability and scalability. Where these solutions typically sell is through a proof of concept where a customer has a very specific performance issue that the vendor can show proven results. Industries that collect massive amounts of transaction data such as retailers or web clickstream data are inherit sweet spots for DW appliances.

If you aren’t familiar with the DW Appliance space, I would recommend taking a look at a series of articles from Krish Krishnan (intro, part 1, part 2) on the topic. I also came across this blog posting fact or fiction that unwinds some of the misconceptions on the DW appliance space.

Another interesting area that has followed the DW Appliance trend is in Business Intelligence. I have come across fewer vendors here, but Celequest (acquired by Cognos) and Ingres Icebreaker are two that provide a bundled hardware, operating system, database software and reporting tools. Business Objects has also partnered with Netezza to provide a single point solution in data warehousing and business intelligence. All the solutions adhere to standards which allows for integration with a majority of the BI vendor software that are SQL based tools.

In Search of BI Mashups

An area that has had a tremendous impact on the consumer aspect of the web is the concept of a “mashup”. The history of the term goes back to DJs and mixers combining different songs together to create new music. The term has evolved to more generically represent an application that is built by combining two or more data sources (if this isn’t the definition of a business intelligence application, I am not sure what is). The Senior Director of Engineering at Adobe put it best when he said

“a lot of talk about Web 2.0, web mashups, Ajax etc., which in my mind are all facets of the same phenomenon: that information and presentation are being separated in ways that allow for novel forms of reuse.” - Sho Kuwamoto

The same statement can be applied to enterprise data…separate the organization’s data from the different ways it can be presented. Where mashups come into play is when enterprises start presenting this data beyond grids and charts. In addition, as I have discussed on this blog, enterprises can combine traditional and non-traditional data sources to provide further context. Thus, the case for BI mashups.

If you perform a quick search for examples of BI mashups, you primarily find sample applications from different BI platform vendors. The first example I came across was from open source BI vendor Pentaho, which combines sales data with Google Maps to plot customer performance. Additional examples from Information Builders and Oracle offer similar examples. The trend with the majority of these examples is that they plot spatial data into geographic maps to show enhanced visualization. Not quite what Tufte would recommend, but certainly an enhancement over traditional BI. Adding non-structured data into the mix such as blogs and customer surveys through RSS feeds would enhance the experience even more.

For the technical audience, here is a very in-depth article by Larry Clarkin and Josh Holmes on mashups including examples, architectural components and key considerations when developing your first enterprise mashup. There is a wealth of information within the article, but one of the key elements applicable to a BI mashup is providing “rich visualization of data” for users that they won’t get from a typical chart or grid of data. If you are considering your first enterprise mashup, I would get familiar with this article as a first step.


There are some great resources available if you are looking for more examples of mashups. The most well known site is Programmable Web, which tracks interesting mashups, Web 2.0 applications and new web platforms. And if you have a short attention span and would prefer to see a video, check out this YouTube video.

Shifting Mindsets on BI

Pete Graham recently wrote a post on Using Business Intelligence in E2.0 that challenged each of us to bring business intelligence (BI) into the business conversation (verses creating a business conversation around BI). It was a prickly role reversal for those of us who like to look at the information value chain in a linear fashion beginning with data: data -> information -> knowledge (picture below of basic analytical information systems strategy). However, he provided a gentle but persuasive reminder that our mental mindsets and diagrams need to shift.

Let me explain. The idea of information and its use within business is an old idea, but its mastery reigns rather elusive. There are three core competencies that need to be achieved: Data IN, Information OUT, & Knowledge AROUND.

Data IN
Every time something happens within a business, there exists the opportunity for us to capture a piece of “data” that records its occurrence. For instance, when someone walks into a retail outlet, their visit can be recorded with a date stamp and time stamp. When the visitor buys a greeting card, the transaction is stored, inventory is marked down, and cash can be credited. If the person happens to pay by credit card, the purchase is tagged with the person’s card number. If the customer scanned their loyalty card, the transaction is immediately tagged with their profile information - and on and on. We could go on to name thousands of activities that are tracked within our organizations. These transactions let us know that something has happened!

This is not surprising. We live in a digital world where many of our actions are recorded. The challenge for businesses is to store this point-in-time data in a timely fashion and in such a way that it can be accessed quickly and easily in the future. I call this exercise, the “Data IN” process. This is the opportunity for our organizations to capture all of the happenings within our business ecosystem. Unfortunately, this raw data is unwieldly to the average business person.

Information OUT
Therefore, an organization is tasked with putting this data into context so that users can see an evolving narrative about their business. This narrative helps us to understand the what, when, and how of our businesses and their performance within the marketplace. We get to see the single occurrence (or piece of data) with the context of the business story. This process of transforming data into “information” is invaluable and gives us the digestible analytics to manage, measure, and improve our businesses.

Getting “Information OUT” is achieved by answering both traditional and current business questions with information about the past or with forecasts about the future.

Knowledge AROUND
The last piece of the information value chain is to seize the Aha! moments and business insights and push them out to the organization. For instance, a store manager who sees a declining trend in her customer base may realize that a profound shift is taking place in her market. With the combination of some analytical reporting and some field observation, she may notice that a local competitor has cut deeply into her customer base. This “knowledge” needs to be shared with her organization so that other store managers can prevent a similar decline and so functional groups within the organization can support or assist with planning a response (or change to the business). Our companies have a need to easily and quickly share insights throughout the organization, or broadcast “Knowledge AROUND”.

Today’s E2.0 tools have brought renewed energy to the business conversation represented by the Knowledge AROUND piece of the value chain. Tools like blogging, microblogging, wikis, prediction markets, etc… are democratizing the voice of the market facing parts of our organizations! This is exciting because it allows the conversation that is happening out in the field – between the people in the field and the market (customers, vendors, etc… ) to more effectively influence the information value chain. To Pete’s point, at the beginning of this post, our organizations need to bring BI into the business conversation. If we do, we have the opportunity to consistently adapt to fulfill the needs of our changing markets.

Let’s keep thinking about the paradigm shifts required to bring BI to E2.o. What do you think? What topics should we be discussing?

Evolution of On-Demand BI

So I have been seeing a lot of the “On-Demand Business Intelligence” marketing term being tossed around. Just to validate the infectious nature of the term, right or wrong, try doing a quick Google search on the phrase “On Demand BI”. For me it returned 9,120 results in less then 0.27 seconds. When I looked at the top 50 results, they ranged from written articles to software vendors to some topics inappropriate for the DIG blog space. (As a side note, I think I found a new feature for Google to add, or I have missed it. I would like to be able to go to the end of the search results. I was curious to see what was at the bottom of the list).

So I filtered out the software vendors, since the purpose of my search wasn’t to evaluate the technologies available, although it is something I would like to do in a future post. Instead, I wanted to see what the “experts” were saying about business intelligence on-demand.

The first observation is that less then 15% of the results were from the past year. I can conclude that one of three things occurred. One, the marketing dollars are starting to dry up and there is only so much that can be spent on certain buzz words. Second, the industry has moved on to a different set of marketing terms that will help create hype cycles along with fear, uncertainty and doubt. Or third, and what can only be the real reason in the drop of web content over the past 12 months, we have finally arrived. BI on demand is a reality and there is no reason to discuss it anymore (insert sarcasm here).

So what is “on demand BI”? Simply put, it is hosted business intelligence solutions or “Software as a Service”, SaaS for short. The most successful and well-known SaaS vendor is Salesforce.com and their AppExchange platform.

The first question I asked myself was “So what are the benefits of on-demand business intelligence”? I started back to the earliest Google results I could find from 2006. Here are some of the highlights that I found in different articles selected at random.

In a Computer World posting back in October of 2006, Jerri Ledford made the following points on the topic:

“…it's business intelligence when and where you need it, without all of the difficulties of building the solution yourself, or hosting it yourself, or even maintaining it yourself.”

“on-demand BI is gaining ground, because it's appealing to smaller companies that can't afford to invest in a full-blown BI solution”

“…the benefits become clear (usually). For example, on-demand BI usually means you have the answers to your BI questions when you need them (and where you need them) without having to devote an entire army of IT professionals to pulling those answers from mountains of data.”

Another posting on TechLink by Amit Kesarwani in December 2006 had this to say:

“On-Demand BI provides much more benefits than operational Software as a Service due to the complexity and high cost of BI systems. On-Demand BI providers can easily offer economies of scale and shared cost using multi-tenant architecture but without jeopardizing security”

Mr. Kesarwani goes on to compare “Traditional BI Deployments” to “On-Demand BI Deployments”. Here are a couple of the comparison points.

“Traditional BI Deployments: Complex to use and deploy, Expensive Customization, Long Project Life Cycle, Lack BI best practices, High Risk”

“Usability for On-Demand BI Deployments: Easy to use and deploy, Low Cost Customization, Rapid time to market, Provides best practices, Low Risk”

And finally, this post from Darren Cunningham on the SuccessForce Community Blog:

“I think the renewed focus on simplicity in the BI market is accelerating the shift from on-premise to on-demand solutions and this will ultimately help drive greater end-user adoption and improve organizational decision making.”

So, let’s jump ahead to some more recent postings in 2007 and 2008. The first was from a research report by the Aberdeen Group on what is driving BI user adoption. Here are a couple of the highlights:

“This benchmark study finds that organizations falling into the ‘Best-in-Class’ category are addressing the skill set shortage by establishing training programs, contracting with 3rd party consultants and solution providers, or implementing On-Demand options such as hosted BI, SaaS (Software as a Service), and BI appliances.”

And finally, I found the following on CIO.com. This article was posted 4 days ago!

“The oft-cited concerns regarding on-demand and SaaS applications (integration, customization, security) typically don't emanate from the business side of an organization. Typically, they come from IT groups already under intense pressure from project backlogs and a lean number of staffers, who most likely don't have BI development skills.”

“With easy-to-install on-demand applications, IT's role as gatekeeper is minimized, say analysts. By 2012, Gartner's Schlegel predicts that emerging technologies such as on-demand and SaaS BI tools will make users ‘less dependent on central IT departments to meet their BI requirements.’”

“One major sticking point for IT usually involves the security of corporate data as it moves outside of IT's control. But executives and analysts say that the potential business benefits of quicker access to BI data, coupled with the robustness of third-party providers' security mechanisms, may outweigh concerns.”

So what can we glean from these articles? What are some common themes? What has changed over the course of three years?

There are some pretty big promises being made with a lot of the positive rhetoric. Promises like fast deployments, higher user adoption and elimination of your IT organization. Okay, I made that last one up but I was just checking to see if you were paying attention. If you peel away the SaaS vision and happy talk, there are some key themes that should be takeaways, for both on-demand and on-premise BI deployments.

First, user adoption is a significant issue for BI applications. So what is driving low user adoption? The issue that continues to bubble to the top is complexity of the tools. Does SaaS solve this problem? Possibly, but BI systems still comprise of complex tools to answer complex questions. The advantage of SaaS is the total cost of ownership will be significantly less, since the per user license cost is lower and if you only have a handful of people actually using it, then you would hope you only pay for those users. This will be much cheaper then predicting you will have 1000 users for on on-premise solution and paying all of those users before you have the software installed.

Second, the simple and easy deployments seem to popup consistently. I do agree that removing the complexity of hardware and software procurement will reduce some upfront angst, but do we really think that the primary reasons why BI applications go amuck (i.e. poor data, inconsistent agreement to business requirements to name a few) will magically disappear? I am not a pessimist by nature, but instead a realist. These issues need to be address no matter how you physically deploy your BI solutions.

Finally, security of data is a consistent concern associated with hosting a BI application. It was pointed out in 2006 and 2008 and I will predict it will be a concern in 2012. That being said, how many organizations have had data compromised when it was behind their own firewalls? Maybe it will be safer if someone else takes care of it for you.

So who should consider on-demand BI? The quick answer is everyone. It is a viable approach and a majority of the software industry is moving to SaaS architectures. But to be more specific, I would say there are three considerations when selecting the type of BI deployment. The first is the size of your organization. Smaller organizations that currently have no BI solution and the economics make more sense should consider an on-demand deployment. For those firms that have limited business intelligence skills in IT, should also consider BI SaaS. Finding talented BI skills has been a lingering issue for years. And finally, consideration should be made on the security of your data being hosted externally to your organization. As I said earlier, it may be more secure when you don’t host it!

To read some additional thoughts beyond my rants, check out Timo Elliot's BI blog posting on the topic.

What are you thoughts? Are you using or considering on-demand BI? If so, what made your deployment successful versus an on-premise deployment (assuming it was successful)?

Using E2.o in Business Intelligence or Business Intelligence in E2.o?

I recently did a Google search to see if I could find anyone who has written on Business Intelligence and Enterprise 2.0. I came across a series of articles written by Colin White on the b-eye Network on just such a topic. I commend Mr. White in doing the difficult job associated with building out a framework for E2.0 and BI. I now have the advantage of commenting and building upon what he has written. Hopefully my comments in this post will not be taken in anyway disparaging what Mr. White has written. I am responding with hopefully semi-coherent thoughts, since the article created some level of emotion that I felt compelled to make this post. As I said, I have the easy job of writing the blog post!

The article starts out stating “This series of articles examines the use of Enterprise 2.0 in business intelligence (BI)”. This may be semantics, but I see the need to make the statement in reverse, “the use of Business Intelligence in Enterprise 2.0”. When I envision an Enterprise 2.0 platform, I see business intelligence as a plug-in to the E2.0 platform. It is analogous to how Facebook works. The Facebook platform provides the foundation for the social network that exists between people. Applications are then distributed and shared across the “social graph” as Facebook calls it. Relate this back to E2.0 and BI. Business Intelligence “objects”, such as a report or metric, can be distributed and shared in the same fashion. This is why I see the statement as the use of business intelligence in Enterprise 2.0.

Mr. White continues on to discuss the current challenge that BI deployments face, which is user adoption. He points out that this is driven by two primary issues: complexity of the BI tools and the ability for business users to understand the data. I agree that these two issues are #1 and 1A in why BI applications fail. So how can E2.0 address these two issues? The first is that enterprise 2.0 technologies…wikis, blogs and social networks…are inherently easier to understand and use. This is because the tools are built around driving collaboration and discussion. Having a conversation is easy, assuming the two people talking speak the same language. Enterprise 2.0 is really putting that dialog in a massively collaborative environment that is a many to many discussion. Where E2.0 can really start to drive adoption of BI is around the second issue of understanding data. By providing a platform that allows users to provide context and meaning to data, you start to address the understanding gap that exists. Simply viewing a report with data will certainly cause confusion without context. What better way to do this then by using a collaborative environment where users can provide commentary and textual descriptions.

Another concept that Enterprise 2.0 can bring to business intelligence is the “Amazon Effect”. For example, users who viewed this report also look at these other reports. Or users rated this metric a 4 out 5 when it comes to value in understanding performance, and here is why. These may be silly examples, but the concept of tapping into the collective intelligence of an organization to drive better decision making can have a powerful impact.

A challenge I foresee is how can we link the E2.0 world of unstructured data and the BI world of structured data? This is quickly becoming a challenge that is being addressed in a couple of different ways. There are niche software vendors that are providing information access and search platforms that integrate structured and unstructured data. Endeca, Attivio, Fast Search & Transfer and Northern Light, to name a few, all provide technology platforms that address this subject area. Glyn Heatley, a TalkDIG contributor, happened to provide a post on the topic of structured and unstructured data this morning. I also did a post a couple of weeks ago on the topic of building out an enterprise semantic layer through interpretative technology that can deduce the objects and their relationships based on the natural language inside of text.

On the topic of structured and unstructured data, the article discusses issues associated with accuracy, stating that “Information is never 100 percent accurate”. For business intelligence applications and deployments, success will be driven by the overall trust in the data being presented. Structured data such as revenue will need to be accurate. In an E2.0 environment, different challenges will exist. Since there is no governance in place for the unstructured data, focus will need to be on the validity associated with the context being provided.

The article concludes with Mr. White’s seven components of Enterprise 2.0 for Business Intelligence. These components include information collaboration, exploration and analysis, integration, syndication and delivery, a rich user interface, a web-oriented architecture and adopting open source solutions. Mr. White will discuss each of these components in separate articles during the series and has already completed two additional articles in the series.

Finally, the article discusses how the BI vendors are addressing the adoption of E2.0. Mr. White points out that the major BI vendors have been slow to consider and adopt the E2.0 technologies into their respective platforms. In my opinion, this is a good thing and let me explain. If you go back to how I started this post, I made the statement that I believe BI is an application service that plugs into an E2.0 platform. If you agree with that philosophy, then the BI vendors need to be looking at other enterprise 2.0 platform providers. The issue to date is that there are very little standards that would provide the plug and play of applications into an E2.0 platform, although Open Social is starting to gain some momentum. But if you take another look at what has made Facebook wildly popular and successful, it was the recent decision to open up the platform to outside developers to build applications to run inside Facebook. This is what is needed for E2.0 to elevate within the enterprise and start to see the user adoption expectations associated with business intelligence.

So what are your thoughts on the topic? I have in no way answered all the questions. If anything, I have probably only left more questions to be answered. Hopefully that is the case and we can have a dialog on the topic.