Finding the Right Business Intelligence Tool

Which is the best business intelligence (BI) tool? What makes it the best one? What do existing customers think of their BI tools? Is the best business intelligence software also the right one for my business? Which criteria are the most important when comparing BI tools? Is superior functionality the main factor to consider? Or are aspects like vendor and implementer support equally important? And how do independent analysts rate different business intelligence solutions?

Comparing BI tools is no easy task. Especially since there are so many products on the market, all promising they will deliver all sorts of business benefits while at the same time solving all manner of problems.

Naturally, companies searching for a new business intelligence software tool can become overwhelmed with all the choices and promises from vendors.

To make the comparison and search for the best tool for your company easier, this article sets out to provide you with a balanced view of what BI users – as well as BARC analysts – have to say about the leading business intelligence products on the market.

By combining our in-depth knowledge of current BI software tools with one of the largest databases of user reviews of business intelligence software, we aim to guide you through the first steps to finding the best tool for your company.

We believe a combined view of user feedback and in-depth analyst perspective is necessary in order to fully appreciate how business intelligence software tools compare against each other, and to understand which are the best ones for your company.

In this article you will find out:

  • which are the best BI software tools as rated by users (BI User Review Matrix)
  • how BARC analysts compare the products (BARC Score Business Intelligence)
  • the most important lessons learned from surveying more than 30,000 respondents about business intelligence software usage and selection over the last 17 years

The BI Survey: Head-to-Head BI Tools Comparison

The interactive dashboard let’s you compare two business intelligences tools. The comparison is based on the aggregated KPI results from The BI Survey 17. See how the different BI products stack up against each other by choosing a peer group and then two business intelligence software products of your choice.
For an overview comparison of the different business intelligence software products see the BI User Review Matrix further down the page. There you’ll also find more information on the methodology.
User Comparison of Business Intelligence Software

The BI Survey: The User Comparison

No one knows more about how a business intelligence tool performs in the real world than the customers already using it. All too often, they find that products don’t live up to expectations, or that the vendor does not support its product properly. Therefore, when comparing BI software, there is almost nothing more valuable than user reviews.

This is where The BI Survey comes in. Collecting feedback from business intelligence software users every year for the last 17 years, The BI Survey provides detailed analysis of which BI software products get the best user ratings. This year’s BI Survey is based on the analysis of the real-world experience of 3,066 respondents with 2,564 answering a series of detailed questions about their usage of a named business intelligence software tool.

It is the largest and most thorough fact-based analysis of the BI market and software currently available, using over 17 years of experience to analyze market trends and challenge some of the myths surrounding the business intelligence software industry.

This edition features a broad range of software tools, not only from the well known business intelligence giants, but also specialist products from much smaller vendors and open source vendors that ordinarily don’t get much press but which, in many cases, offer outstanding value to their customers.

Altogether, The BI Survey compares 42 business intelligence tools (or groups of products) in detail. To be included in the detailed analysis a tool needs at least 30 user reviews.

Since not all business intelligence tools are alike, we use peer groups to ensure similar products are compared against each other (see below for more information on the peer groups). The groups are essential to allow fair and useful comparisons of solutions that are likely to compete.

The BI User Review Matrix rates software based on the business value it provides (x-axis), customer experience (y-axis), customer satisfaction (color) and competitiveness (circle size). To see the comparison in action, please select a peer group.

The KPIs

The BI Survey 17 examines BI product selection and usage among users in categories (KPIs) including business benefits, project success, business value, recommendation, customer satisfaction, customer experience, innovation and agility. There are 29 KPIs in total.
Different readers will have their own views on which of these KPIs are important to them. For example, some people will regard fast query performance as critical, whereas others may consider recommendation or innovation to be more important.
Consequently, we think reducing the KPIs to only one aggregated score is too simplistic to be helpful when seeking out the best business intelligence software to match your needs. Reducing all KPIs to just one overall score doesn’t provide much value for the reader. In our view, there are at least four crucial KPIs when it comes to comparing BI tools from a user perspective. They are made up from 17 individual root KPIs:
‘Business value’ is possibly the most important KPI, focusing on bottom line benefits of BI projects. Business benefits are the real reason for carrying out any BI project. Business intelligence that does not deliver broad business value is superfluous.

The ‘Business value‘ KPI shows how a successful BI software product can provide these benefits in the real world. The KPI combines the ‘Business benefits’, ‘Project success’ and ‘Project length’ KPIs.

Delivering a superior user experience is more important than ever. BI professionals don’t want to have to spend a lot of time figuring out how a BI product works, attempting to learn interfaces or waiting around for a query to finish. With the current vogue for agility and self-service BI capabilities and the increasing need for users to be able to access a variety of data sources, the user experience of a BI tool is an important consideration for many organizations.

To calculate the quality of user experience of a BI tool, we combine the ‘Ease of use’, ‘Self-service’, ‘Query performance’, ‘Performance satisfaction’, ‘Data volume’, ‘Flexibility for users’ and ‘Sales experience’  KPIs.

Happy customers are a strong indicator of a superior BI software. We combine the ‘Price-to-vale’, ‘Recommendation’, ‘Product satisfaction’, ‘Vendor support’ and ‘Implementer support’ KPIs to calculate an aggregated ‘Customer satisfaction’ KPI. These three factors are clearly related: If one is lacking, then the importance of the others is accentuated.
The ‘Competitiveness’ KPI gives insights into how BI tools perform in a competitive selection process as well as the strength of a product’s market presence. It combines the ‘Considered for purchase’ and ‘Competitive win rate’  KPIs.

Recognizing which business intelligence software to compare entails understanding which BI tools have fared well in other organizations’ product selections. This enables users to eliminate ‘losers’ at an early stage in the selection process.

The KPI rules

Only measures that have a clear good/bad trend are used as the basis for KPIs.
KPIs may be based on one or more measures from The BI Survey.
Only products with samples of at least 20 – 30 (depending on the KPI) for each of the questions that feeds into the KPI are included.
For quantitative data, KPIs are converted to a scale of 1 to 10 (worst to best). A linear min-max transformation is applied, which preserves the order of, and the relative distance between, products‘ scores.

The BI Survey User Review Matrix

Check Out the Planning User Review Matrix

Comparing BI Software with the help of peer groups

Not all BI tools are alike. Some are strong on reporting, while others are more suitable for dashboards or analytics.

Results from The BI Survey show the types of capabilities respondents use with their BI tools. Standard and enterprise reporting is top of the list followed by ad hoc query. The least popular BI capability this year is advanced analysis (e.g. predictive analytics or data mining), although it has the highest ‘planned’ rates, with 48 percent of respondents intending to start using advanced analytics in the future.

The peer groups are primarily based on the results from The Survey. They take into account how customers say they use the business intelligence software, which can vary widely across the different products. But we also include the experience and judgment of the BARC analysts in deciding on the groupings.

Peer groups are simply a guide to the reader to help make BI tools easier to compare and to show why individual products return such disparate results. They are not intended to be a judgment of the quality of the tools.

Some products appear in more than one peer group. We use these peer groups in a consistent way in our analysis, as well as in The BI Survey Analyzer.

The point to the peer groups is to make sure that the comparisons of the BI tools we make in The Survey make sense. The products are grouped together as we would expect them to appear in a shortlist for business intelligence software selection.

To make a proper choice, a buyer should first segment the market into the tool types that fit his requirements. The peer groups are intended to help with this task.

The segmentation of the peer groups is based on two key factors:

  • Usage scenario – functional peer groups – mainly data-driven and based on how customers say they use the product.
  • Regional focus – is the vendor a large international vendor with a truly global presence or does it focus on a particular region? We also take into account the location of BI Survey respondents.

The regional focus is of particular importance since organizations require local contacts, resources and knowledge from implementers to conduct projects in their own country. Furthermore, locally competing vendors and markets are better reflected and easier to compare with this method of market segmentation. To compare vendors with global coverage, a peer group featuring only large international vendors was created.

The peer groups

Includes products from vendors that have a significant presence in - and focus on - the Americas region.

Includes products that provide integrated functionality for BI and performance management, especially planning and budgeting.

Includes products equipped with functionality for enterprise deployments that focus on a broad range of BI use cases.

Tasks where BI is used

Tasks Business Intelligence Software is used for

Tasks where BI is used (n=2,568)

Want to rate your BI software?

Participate in the world‘s largest and most comprehensive survey of BI software users

Take part in The BI Survey
Analyst Comparison of Business Intelligence Software

BARC Score Business Intelligence: The Analyst Comparison of leading BI tools

BARC Score Business Intelligence compares the strengths and challenges of all the leading vendors in the business intelligence market as well as many smaller vendors that often have less visibility but equally offer outstanding value to their customers.

Based on countless data points and many analyst interactions, business intelligence software vendors are rated on a variety of criteria, from product capabilities and architecture to sales and marketing strategy, financial performance and customer feedback

Calculating the individual ratings for all criteria and all vendors produces two scores per company: the ‘portfolio capabilities’ score and the ‘market execution’ score, which are plotted on each axis resulting in the vendor’s dot on the BARC Score graphic.

‘Portfolio Capabilities’ includes a series of weighted criteria based on three major areas: functionality, infrastructure and product-related evaluation criteria.

‘Market Execution’ takes into account a blend of product, sales and marketing strategy as well as certain organizational, financial and geographical considerations.

There are two separate inclusion criteria to be reviewed in the BARC Score: the first is associated with a software vendor’s products and the other is linked to the financial results relating to their BI tools. To be evaluated in this BARC Score, a vendor has to have a strong focus on providing BI functionality and supply four out of six technologies from the following functional portfolio:

  • Formatted and Ad Hoc Reporting
  • Dashboards
  • Analysis
  • Data Mining
  • Planning
  • Self-Service BI and Data Discovery

In addition, the vendor has to generate a minimum of 15 million EUR in license revenue per year with the above product set, spread across at least two separate geographies. As individual geographies we consider:

  • Europe, Middle East and Africa
  • North America
  • Latin America
  • Asia/Pacific

Vendors with an open source business model are compared by their total revenue because those companies do not charge a license for their products, but an annual subscription fee.

The BARC Score Business Intelligence 2016

Business Intelligence Software Comparison by BARC Analysts

Score Regions

Dominators are vendors that drive both technology and market adoption in a highly influential manner. They possess both a broad portfolio of market-leading and dominating products with a strong brand as well as a robust commercial prowess through best-in-class sales and marketing programs, an extensive ecosystem of business partners and alliances, and a rock-solid financial position. Dominators are considered a contender in virtually every planned implementation.
Market Leaders are well established vendors that drive strong market adoption, supported by technology innovation and strategic acquisitions and by leveraging robust account management and a solid track record. Their portfolio enjoys high brand awareness in the market, covers an extensive range of technologies and services with only few gaps. Market Leaders typically have a large market share, making them a viable contender in almost all implementation scenarios.
Challengers come in various shapes and sizes. They can be large vendors tapping into a new market by acquisition and pushing their way in with force, small innovative companies with a promising portfolio but limited sales and marketing resources, or vendors that attempt to disrupt a market with a new technology approach or different business model.
Specialists are smaller vendors with a portfolio focused on a specific market segment. Vendors can be either limited in their technical capabilities by concentrating on certain features and functions, or the company isn’t a global enterprise and focuses on select geographic regions.
Entrants are usually startups that have limited reach and visibility in the market. Their product capabilities are incomplete when compared to competitors, and the vendor’s long-term market potential is still unproven.
Don‘t miss out – Get the latest BI product insights, research, surveys and more!
Lessons Learned for BI Comparison

What We’ve Learned About Comparing Business Intelligence Software

Why the choice and comparison of business intelligence products is important when it comes to finding the best tool

Business intelligence projects are about creating business value by enabling better business decisions. An important – though not the only – part of this value creation is derived from the BI software products that gather, enrich, store and present data to end users, business analysts, or even data scientists.

Choosing a software product that doesn’t match your requirements can pose a serious threat to the success of your project. Survey results show major differences in the business benefits achieved with different BI tools.

The Business Benefits Index (BBI) is a unique measure used in The BI Survey to determine the value of BI projects. It takes both quantitative and qualitative benefits into account as well as different levels of possible benefit achievement. Here you can find more detailed information on how we calculate the BBI and how we classify best-in-class companies.

Business Benefits of the best and worst BI products
Percentage of businesses doing a competitive BI software comparison

Percentage of businesses doing a competitive BI software evaluation (n=2189)

Business benefits by selection type

Business benefits by selection type (n=2189)

How to select a BI product

Selecting the best business intelligence software is key to a successful project, and many software problems can be traced back to poor software selection processes.

The software is often used for ten years or more so the total cost of ownership (TCO) can be substantial. Investing time in choosing the best business intelligence software tool will have positive long-term effects.

Before purchasing and committing to any BI tool, we always recommend that organizations conduct detailed, formal evaluations of multiple products on the market.

However, we are aware that not everyone follows this advice. The data collected in The BI Survey allows us to draw some definite conclusions on this topic.

The findings from our survey reflect those stretching back to 2002 in that business benefits improve when a company performs a product evaluation and benefits are further improved if enterprises extend the formal assessment process to a multi-vendor comparison.

What are the right criteria for evaluating BI software

The criteria best-in-class companies use for a selecting their BI software confirm that successful projects are based on selecting the tool rather than the vendor.

This is one of the most consistent findings over The BI Survey’s seventeen-year history.

  • Best-in-class companies take more selection criteria into account than laggards.
  • Best-in-class companies are much more interested in functionality, ease of use and fast query performance; all key considerations in enabling self-service BI.
  • Laggards, on the other hand, are more concerned with non product-related criteria such as ‘vendor listed as corporate standard’ or ‘bundled with another product’.
Reasons to buy BI software

Reasons for buying BI software tool (n=2434)

Implementation time for BI software

Median implementation time (n=2260)

Why implementation speed matters for a BI tool

Countless IT projects experience delays and data warehousing and business intelligence projects are by no means immune to this perennial issue.

Implementation time depends largely on which product is being implemented. Naturally, the complexity of the BI solution and/or the project will impact on project length and certain BI products lend themselves better to more complex projects, thus increasing their average implementation time.

Clearly, other factors such as the number of data sources, the volume of data and the number of users or departments served also have an influence here.

Fast implementation times are key to a successful BI project. Projects taking more than three months to complete lead to significantly lower business benefits. Project implementation times appear to have a direct impact on the level of all business benefits; with benefits decreasing as project length increases.

Aim for a three-month implementation window for the first application. Best-in-class companies prove that it can be done in just two months on average.

If possible, try an incremental approach by breaking the project into a series of smaller projects.

Why good support is especially important

The standard of service provided by consultants varies considerably. The BI Survey asks business intelligence users about their experiences with external implementer support as well as vendor support. We found a clear correlation between customer ratings of implementer and vendor support and business benefits achieved.

Seeing such a big difference in business benefits between different types of consultants does not come as a surprise. Good support from an implementation consultant is an important aspect of project success.

Finding the right vendor in combination with the right implementer is crucial to project success. In previous editions of The BI Survey, we used to ask about the type of consultants companies used. Results revealed that small, specialized consulting firms frequently scored better than large general purpose consulting companies.

The success of business intelligence projects depends upon organizations’ and vendors’ abilities to deliver successful and relatively rapid implementations, along with supportive follow-up and maintenance.

Many BI solutions end up as shelfware within enterprises, due to difficulties in setting up software, difficulty in using tools, or lack of ability to communicate the benefits of the tools and platforms to end users.

The ability to seamlessly incorporate business intelligence software and platforms into corporate decision-making processes and to closely align those solutions into business results is crucial.

Support for BI tool analyzed by business benefits
Percent of users having no problems with BI tool

No significant problems with the BI tool by
best-in-class companies (n=2586)

Avoiding problems

Survey results clearly reveal that a carefully conducted comparison and selection of business intelligence tools pays off. The organizations that achieve the greatest level of business benefits with business intelligence (best-in-class companies) are much more likely to have conducted a competitive product evaluation, and they experience far fewer problems with their BI software.