Now is the perfect time to assess the value of your business intelligence investments, experts say. That may sound counterintuitive given the pressure to roll out systems to meet user demands, but when funding is tight, "wringing the most from your investments and ongoing operations" is prudent, says Dorothy Miller, president of Redstone360 Management Services, Tucson, Ariz.

"Better business information is a foundation and prerequisite for business activity and decision making," Miller says. Yet she takes a hard-nosed approach to assessing and improving the value of BI products and operations. Too many business and IT leaders put BI software infrastructures and products in place and then assume everything is OK. "How do we know that six of our managers made bad competitive decisions related to the style of a product because they did not have the analysis information necessary?" she asks to illustrate her point. "No one is rigorously measuring [BI] value, and I think that must be improved."

Miller has created a comprehensive analysis program to audit and evaluate BI investments and returns, and she recommends that most businesses conduct an audit review before making additional BI investments. Miller has also written several books on the topic including: Measuring Business Intelligence Success (D.M. Morrisey 2007) and Improving Business Intelligence: The Six Sigma Way (Holt, Thompson & Associates, 2008).

Mark Smith, CEO and executive vice president, research, at Ventana Research, agrees that CIOs need to be sure their BI investments are paying off. "If I spend [IT budget dollars] and deploy tools, does that mean that all of the software delivered to users is in use?" CIOs need to drill down to see how many users log on, for how long, and what information they're using," Smith says.

Business-Side Responsibility

The business side also has to take more responsibility for measuring BI use and effectiveness, Smith says. Business users need to evaluate whether they are reducing costs and accelerating processes as a result of BI. They should ask: "Has BI cut the time it takes to fix problems and to find new business opportunities?" "Are we making fewer last-minute decisions?" "Are we comparing results in real time?" Assessing these soft metrics will lead to increased margins and hard-dollar revenue, he says.

Ideally, Redstone's Miller says, IT and business should jointly define their goals before any spending or product deployment begins; meeting customer expectations is key to success. In addition, assessing BI operations and calculating tangible BI product values are critical to obtaining the most value from an investment. Most BI infrastructures have been in place for years, she notes, and there is a tendency to accept the idea that we should make large investments for BI products where the value is intangible.

Miller suggests that CIOs "set aside time and resources for a BI assessment and improvement program." Additionally, she says to "define the customers for each business intelligence product and isolate key performance measures."

In addition to a BI operations assessment, Miller also recommends product assessments based on standardized, detailed surveys and focus groups. BI products can be improved if users identify and measure what Six Sigma calls critical-to-quality (CTQ) factors. Some CTQ factors Miller offers for measuring end-user or customer satisfaction are:

• Performance: Response time is usually the most relevant performance factor for BI – for example, the time it takes from an information request/query to receipt of the business information.
• Reliability: Is the business information always there when needed?
• Quality: Is the information accurate, timely and clear?
• Does the information meet business requirements?
• Can the product and/or application be easily expanded and scaled to meet business requirements?

Most Significant Concerns about Managing Data across Business and IT

Source: Ventana Research Data Governance Benchmark Research, October 2009, 341 global respondents

Ventana's Smith says BI effectiveness is best determined in tangible but qualitative ways, for instance, by seeing how long it takes to close a deal; trends and patterns that help determine pricing, volume or sales; and cost-avoidance. "You have to make sure that the data is of high quality and is integrated across the business," Smith says. To best determine BI value, he advises three key pre- and post-deployment actions:

• Take a role-based approach and provide the best tools for each task and each user. This may mean simple tools for power users and dashboards and scorecards for managers and executives.
• At bigger companies, you can also look at user competencies and generational issues. For example, Gen X users are more aggressive than others. They want to use data collaboratively and don't want static reports.
• Before adding any new tools or expanding current systems, determine usage and maximize current capabilities. You may decide to implement a "use-it or lose-it" policy to encourage use and cut losses from underutilized systems.

Additional Resource:A discussion about measuring the benefits of BI takes place on Peter Thomas' blog here