Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Sponsored by Produced by
2
TABLE OF CONTENTS
Respondent Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
BI and ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Figure 4: When is the last time you implemented a new BI application (including an upgrade of a current product)? . . . . . . . . . . . . . . .6
Figure 5: How many people regularly access your business intelligence applications? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Figure 6: Did you have to perform a cost-justification prior to implementing your BI solution? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Figure 7: Has your company conducted an ROI analysis for your investments in business intelligence? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Figure 8: Have you ever calculated the total cost of ownership for your BI implementation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Introduction
Measuring the return on investment for technology acquisitions is seen as a key management process. Nonetheless, for a variety of
reasons, many companies do not conduct formal return on investment studies. ROI studies can be complex. New IT investments
generally have direct, indirect, and related expenses, so calculating the appropriate investment can be tricky. Calculating benefits and
return is also a challenge. Once again, IT projects generally have both tangible and intangible benefits. And many of the benefits
may not be easily assigned a monetary value.
Measuring the ROI for business intelligence projects is even more challenging than for other kinds of IT investments. The overall
objective of business intelligence is to improve company performance by putting the right information into the right hands at the
right time. Many BI implementations are intended to allow business analysts to identify significant trends in enterprise operations
and to provide senior management with better information with which better decisions can be made in a timely fashion. Over the
past several years, business intelligence implementations have also been intended to provide key metrics to front-line personnel,
including the sales force, call center operators and even companies customers.
Of course, many different factors have an impact on overall company performance, so precisely measuring the influence of a busi-
ness intelligence implementation can be difficult, if not impossible. In the same way, it is hard to assign a measurable advantage to
providing improved access to better information for a companys customers.
To understand the role of ROI analysis in IT generally and the key metrics for measuring ROI for business intelligence specifically,
Unisphere Research conducted a Web-based survey on behalf of Noetix. Email solicitations were sent to readers of Unisphere Media
publications as well as to a list of IT professionals supplied by Noetix. In total, 415 people participated in the survey. Of that num-
ber, approximately 80 percent of the respondents indicated that at least one business intelligence/business analytics tool was in use in
their enterprises.
Most companies in the study conduct ROI studies for at least some of their IT projects.
A team of IT and line-of-business staff are generally responsible for conducting ROI studies.
A majority of respondents in the study have not conducted ROI studies for their BI investments.
The key cost metrics for an ROI study for BI is the direct cost of the software licenses and the
cost of needed hardware.
The key benefit metric for BI implementation for senior management and line-of-business staff is
improved access to data.
The key benefit metric for IT staff is the time it takes to develop a report.
The key metric to judge the impact on decision-making is the quality of information delivered.
A majority of respondents felt that the intangible benefits of BI were more important than the
tangible benefits.
The difficulty in accurately measuring both costs and benefits are the chief hurdles in conducting
ROI studies for BI.
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Respondent Pool
Respondents came from a wide range of companies of different sizes participating in many different industries and held a variety a
job titles. Approximately 64 percent of the respondents indicated they have primarily IT-related responsibilities and close to 23 per-
cent have a combination of IT and line-of-business-related responsibilities.
The respondent pool, however, was weighted towards large companies. Approximately 54 percent of the respondents said they were
at companies with more than 1,000 employees (nearly 32 percent are at companies with more than 5,000 employees). Judging size
by revenue, the spread was more even. About 27 percent of those surveyed said they work at companies with more than $1 billion in
revenue while 21 percent work at enterprises with less than $50 million in annual revenue. And while technology companies repre-
sented the largest single market sector with around 26 percent of the survey participants, manufacturing, financial services, govern-
ment/non-profit, healthcare, and retail were also well represented.
In terms of job titles, database administrators and systems administrators made up the largest classification, generating about 26 of
the respondents. Directors and managers of IT/IS were the next largest group, accounting for more than 18 percent of the respon-
dents. A complete breakdown of the respondent pool can be found in Figures 20 through 23.
Survey respondents routinely conduct ROI analyses of their IT investments. As Figure 1 shows, nearly 90 percent of respondents
analyze ROI at least from time to time for what are considered significant IT projects. However, only a little more than a quarter of
those surveyed (26.9%), assess ROI for all IT projects.
Figure 1: How frequently does your company conduct ROI analysis for
significant IT projects?
Never 10.91%
From time to time 28.43%
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Not surprisingly, the term significant is interpreted differently in different organizations. In almost one-third of the enterprises
(29.5%), ROI studies are conducted for all budgeted IT projects. In about a quarter of the enterprises (23.9%), corporate manage-
ment determines the project for which return on investment will be calculated. And in around 10 percent of the organizations, ROI is
only used in special situations. (See Figure 2.)
Figure 2: For which kinds of IT projects does your company conduct ROI analysis?
All projects with an initial cost of more than $50,000 only 9.73%
All projects with an initial cost of more than $100,000 only 11.21%
All projects with an initial cost of more than $500,000 only 3.24%
0 20 40 60 80 100
Charts do not add up to 100 percent due to rounding.
As Figure 3 shows, ROI studies are conducted either by a team consisting of both IT staff and line-of-business staff (30%) or by IT
managers (28.9%).
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
BI and ROI
The use of BI and analytics was widespread among the respondents. Around 80 percent had at least one business intelligence or ana-
lytics application in operation and another six percent planned to implement a BI solution within a year. As Figure 4 shows, around
half of the companies had put a new BI application online or upgraded an existing implementation within the last year. As measured
by the number of users, the reach of the BI application varied greatly among the enterprises. On the one hand, more than 17 percent
of the end-users said more than 500 users accessed their BI applications. On the other hand, almost 14 percent said less than 10 peo-
ple in their organization used BI. (See Figure 5.)
Figure 4: When is the last time you implemented a new BI application (including an
upgrade of a current product)?
Figure 5: How many people regularly access your business intelligence applications?
100-499 20%
10-49 29.21%
50-99 19.37%
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
As Figure 6 shows, most enterprises do justify costs before implementing a BI application. However, fewer than half of the compa-
nies surveyed formally calculate their return on investment for the BI implementation (Figure 7). And even fewer companies have
ever calculated the total cost of ownership of their BI applications (Figure 8).
No 31.55%
Yes 68.45%
Figure 7: Has your company conducted an ROI analysis for your investments
in business intelligence?
Yes 45.24%
No 54.76%
Figure 8: Have you ever calculated the total cost of ownership for your
BI implementation?
Yes 23%
No 77%
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Calculating the return on investment for IT projects is a formidable activity and in some ways calculating the ROI for business intel-
ligence is more challenging than for other kinds of IT applications. The ROI equation has two parts determining the real costs
associated with a project and then being able to measure the real benefits.
To accurately assess ROI for business intelligence, both direct and indirect costs must be measured. But assigning costs to a specific
project often calls for judgment. For example, if more storage is not purchased specifically for a business intelligence project but
will be required in the future to manage the data associated with BI, how should those costs be treated in terms of calculating ROI?
Figure 9 ranks the most important direct costs to measure when determining ROI for BI. Not surprisingly, the initial cost of software
and new servers required as well as the consulting services needed to implement the project are the most important direct costs to
take into consideration.
Figure 9: Rank how important it is to measure each of the following direct costs
for BI applications. (Scale of 1 to 5, with 5 being very important)
0 1 2 3 4 5
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
But the calculation of direct costs is only one step in determining the cost side of the ROI equation. BI implementations also gener-
ate indirect costs and other, related costs that must be measured. As could be anticipated, in general, respondents found that the cal-
culation of indirect and related costs was less important than calculating direct costs. Nevertheless, the most important indirect cost
to take into consideration is the cost of upgrading related software (Figure 10), while the most important related cost is time for
training end-users, both business analysts and other end-users (Figure 11).
Figure 10: Rank how important it is to measure each of the following indirect costs
in calculating ROI for BI applications. (Scale of 1 to 5, with 5 being very important)
0 1 2 3 4 5
Figure 11: Rank how important it is to measure each of the following related costs
in calculating ROI for BI applications. (Scale of 1 to 5, with 5 being very important)
0 1 2 3 4 5
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Calculating the benefits from an investment can be even more challenging than calculating the costs. Some of the anticipated bene-
fits include the potential to increase productivity among several different communities, including IT staff, senior management and
line-of-business personnel. Moreover, business intelligence is intended to improve the quality of the decisions taken.
The question is, what are the best measures to determine the potential benefits produced by the use of business intelligence? As
Figures 12 and 13 show, the most important metric for improving the productivity of both line-of-business personnel and senior man-
agement regarding the use of business intelligence is the time it takes for each community to improve the access to data for making
decisions.
For line-of-business personnel, the second most important criterion is the number of reports end-users can create on their own. For
senior management, the second most important criterion is time senior managers must spend on analysis.
Figure 12: Rank how important it is to quantify each of the following benefits for staff in
calculating ROI for BI applications. (Scale of 1 to 5, with 5 being very important)
0 1 2 3 4 5
Figure 13: Rank how important it is to quantify each of the following for senior man-
agers in calculating ROI for BI applications? (Scale of 1 to 5, with 5 being very important)
0 1 2 3 4 5
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
The productivity measures for IT staff differ from those for senior management and line-of-business personnel. The most important
metric for IT staff is the time it takes to develop a report followed by the staff time required to respond to users requests. (See Figure
14.)
Figure 14: Rank how important it is to quantify each of the following benefits for IT
staff in calculating ROI for BI applications. (Scale of 1 to 5, with 5 being very important)
0 1 2 3 4 5
While developing metrics to gauge increased productivity from all three communities of users involved IT staff, senior manage-
ment and line-of-business personnel is important for analyzing ROI, the promise of business intelligence is the potential to improve
the quality of decisions made. The most important criterion for measuring the quality of decisions made, according to survey
respondents, is the quality of the information available to decision-makers, followed by the quality of information available to front-
line personnel. The least important variable to measure that is associated with the quality of decision-making is the quantity of infor-
mation available to front-line or line-of-business staff (see Figure 15).
Figure 15: Rank how important it is to quantify each of the following general benefits
in calculating ROI for BI applications? (Scale of 1 to 5, with 5 being very important)
0 1 2 3 4 5
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
As Figures 16 and 17 show, other factors could also have a material impact on calculating ROI, including making BI more available
to a wider community of users and vendors' improvement of the ease of use of BI tools.
No 22%
Yes 78%
Figure 17: Would improving the ease-of-use of your BI solution have a material
impact on the ROI calculation?
No 10%
Yes 90%
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Calculating the ROI for BI implementations is a complicated task. As Figure 18 shows, the greatest hurdle companies face in calcu-
lating ROI is that both real costs and tangible benefits are hard to quantify.
Figure 18: What are the most significant challenges to developing a meaningful ROI
analysis? (Multiple responses allowed)
0 20 40 60 80 100
The challenges involved in conducting ROI assessments have led some people in the industry to argue that the intangible, non-meas-
urable benefits are more important than the tangible benefits. Survey respondents generally agreed with this view. (See Figure 19.)
Figure 19: Do you agree or disagree with the idea that the intangible benefits of BI
are more important than the tangible, measurable benefits?
No opinion 30.4%
Agree 53.8%
Disagree 15.8%
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Conclusion
Assessing return on investment is a key IT management discipline. Almost 90 percent of the respondents indicated that their compa-
nies engage in ROI analysis for at least some of their IT projects. However, ROI analysis is often not conducted for BI projects. In
fact, nearly 55 percent of the respondents in this survey indicated they do not study ROI for their BI investments. Interestingly, 95
percent of large companies (those with over 5,000 employees) conduct ROI studies for at least some of their IT investments and 40
percent conduct ROI studies for every IT project. But even in this group, only around 57 percent conduct ROI studies for BI projects.
Companies do not conduct ROI analysis for BI perhaps in part because a majority believes that the intangible benefits of BI are more
important than the tangible benefits, which are easier to measure. In conducting ROI studies for BI, respondents said they believe
that the most significant cost metrics are the direct costs of software licenses and hardware acquisition. The key benefit metrics are
improved access to data and improved quality of a data for decision-making. The greatest hurdles to conducting ROI assessments for
BI is that real costs and tangible benefits are hard to quantify and those hurdles seem consistent for both large and small companies.
1 to 99 13.64%
0 20 40 60 80 100
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
0 20 40 60 80 100
Charts do not add up to 100 percent due to rounding.
Government/Education/Non-Profit/Military 12.77%
Other 15.5%
0 20 40 60 80 100
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
Programmer 8.28%
Other 9.51%
0 20 40 60 80 100
Key Metrics for Determining ROI for Business Intelligence Implementations is sponsored by Noetix and produced by Unisphere Research. Unisphere Research is the
market research unit of Unisphere Media, LLC, publishers of Database Trends and Applications magazine and the 5 Minute Briefing newsletters. To review abstracts of our past reports,
visit www.dbta.com/research. Unisphere Media, 229 Main Street, Chatham, NJ 07928. Tel: 973-665-1120, Fax: 973-665-1124, Email: Tom@dbta.com. Web: www.dbta.com
The information in this report has been gathered through Web-based surveys of member and prospective member lists provided by the OAUG, through interviews with
knowledgeable participants in the computer industry and through secondary research of generally available documents, reports and other published media, as well as from earlier studies
conducted by Unisphere Research. Unisphere Research has relied on the accuracy and validity of all information so obtained. Unisphere Research assumes no liability for inaccurate or
omitted information.
Unisphere Research shall not be liable for errors contained in the document, its data and analysis, and/or for incidental or consequential damages in connection with the use of this document
and its information.