Finding the Right Business Intelligence Tool
Which is the best business intelligence (BI) tool? What makes it the best one? What do existing customers think of their BI tools? Is the best business intelligence software also the right one for my business? Which criteria are the most important when comparing BI tools? Is superior functionality the main factor to consider? Or are aspects like vendor and implementer support equally important? And how do independent analysts rate different business intelligence solutions?
Comparing BI tools is no easy task. Especially since there are so many products on the market, all promising they will deliver all sorts of business benefits while at the same time solving all manner of problems.
Naturally, companies searching for a new business intelligence software tool can become overwhelmed with all the choices and promises from vendors.
To make the comparison and search for the best tool for your company easier, this article sets out to provide you with a balanced view of what BI users – as well as BARC analysts – have to say about the leading business intelligence products on the market.
By combining our in-depth knowledge of current BI software tools with one of the largest databases of user reviews of business intelligence software, we aim to guide you through the first steps to finding the best tool for your company.
We believe a combined view of user feedback and in-depth analyst perspective is necessary in order to fully appreciate how business intelligence software tools compare against each other, and to understand which are the best ones for your company.
In this article you will find out:
- which are the best BI software tools as rated by users (BI User Review Matrix)
- how BARC analysts compare the products (BARC Score Business Intelligence)
- the most important lessons learned from surveying more than 30,000 respondents about business intelligence software usage and selection over the last 18 years
The BI Survey: The User Comparison
No one knows more about how a business intelligence tool performs in the real world than the customers already using it. All too often, they find that products don’t live up to expectations, or that the vendor does not support its product properly. Therefore, when comparing BI software, there is almost nothing more valuable than user reviews.
This is where The BI Survey comes in. Collecting feedback from business intelligence software users every year for the last 18 years, The BI Survey provides detailed analysis of which BI software products get the best user ratings. This year’s BI Survey is based on the analysis of the real-world experience of 3,045 respondents with 2,569 answering a series of detailed questions about their usage of a named business intelligence software tool.
It is the largest and most thorough fact-based analysis of the BI market and software currently available, using over 18 years of experience to analyze market trends and challenge some of the myths surrounding the business intelligence software industry.
This edition features a broad range of software tools, not only from the well known business intelligence giants, but also specialist products from much smaller vendors and open source vendors that ordinarily don’t get much press but which, in many cases, offer outstanding value to their customers.
Altogether, The BI Survey compares 37 business intelligence tools (or groups of products) in detail. To be included in the detailed analysis a tool requires at least 30 user reviews.
Since not all business intelligence tools are alike, we use peer groups to ensure similar products are compared against each other (see below for more information on the peer groups). The groups are essential to allow fair and useful comparisons of solutions that are likely to compete.
The BI User Review Matrix rates software based on the business value it provides (x-axis), customer experience (y-axis), customer satisfaction (color) and competitiveness (circle size). To see the comparison in action, please select a peer group.
The KPIs
‘Business value’ is possibly the most important KPI, focusing on bottom line benefits of BI projects. Business benefits are the real reason for carrying out any BI project. Business intelligence that does not deliver broad business value is superfluous.
The ‘Business value‘ KPI shows how a successful BI software product can provide these benefits in the real world. The KPI combines the ‘Business benefits’, ‘Project success’ and ‘Project length’ KPIs.
Delivering a superior user experience is more important than ever. BI professionals don’t want to have to spend a lot of time figuring out how a BI product works, attempting to learn interfaces or waiting around for a query to finish. With the current vogue for agility and self-service BI capabilities and the increasing need for users to be able to access a variety of data sources, the user experience of a BI tool is an important consideration for many organizations.
To calculate the quality of user experience of a BI tool, we combine the ‘Ease of use’, ‘Self-service’, ‘Query performance’, ‘Performance satisfaction’, ‘Data volume’, ‘Flexibility for users’ and ‘Sales experience’ KPIs.
Recognizing which business intelligence software to compare entails understanding which BI tools have fared well in other organizations’ product selections. This enables users to eliminate ‘losers’ at an early stage in the selection process.
The KPI rules
The BI Survey User Review Matrix
Comparing BI Software with the help of peer groups
Not all BI tools are alike. Some are strong on reporting, while others are more suitable for dashboards or analytics.
Results from The BI Survey show the types of capabilities respondents use with their BI tools. Standard and enterprise reporting is top of the list followed by ad hoc query. The least popular BI capability this year is advanced analysis (e.g. predictive analytics or data mining), although it has the highest ‘planned’ rates, with 48 percent of respondents intending to start using advanced analytics in the future.
The peer groups are primarily based on the results from the survey. They take into account how customers say they use the business intelligence software, which can vary widely across the different products. But we also include the experience and judgment of the BARC analysts in deciding on the groupings.
Peer groups are simply a guide to the reader to help make BI tools easier to compare and to show why individual products return such disparate results. They are not intended to be a judgment of the quality of the tools.
Some products appear in more than one peer group. We use these peer groups in a consistent way in our analysis, as well as in The BI Survey Analyzer.
The point to the peer groups is to make sure that the comparisons of the BI tools we make in The Survey make sense. The products are grouped together as we would expect them to appear in a shortlist for business intelligence software selection.
To make a proper choice, a buyer should first segment the market into the tool types that fit his requirements. The peer groups are intended to help with this task.
The segmentation of the peer groups is based on two key factors:
- Usage scenario – functional peer groups – mainly data-driven and based on how customers say they use the product.
- Regional focus – is the vendor a large international vendor with a truly global presence or does it focus on a particular region? We also take into account the location of BI Survey respondents.
The regional focus is of particular importance since organizations require local contacts, resources and knowledge from implementers to conduct projects in their own country. Furthermore, locally competing vendors and markets are better reflected and easier to compare with this method of market segmentation. To compare vendors with global coverage, a peer group featuring only large international vendors was created.
The peer groups
Includes products from vendors that have a significant presence in - and focus on - the Americas region.
Products that focus on creating advanced and highly sophisticated dashboards.
Includes products that focus on visual data discovery and advanced data visualization.
Includes reporting and analytics products that can be embedded in other business applications.
Includes products from vendors that have a significant presence in - and focus on - the EMEA region.
Includes products that provide integrated functionality for BI and performance management, especially planning and budgeting.
Includes products equipped with functionality for enterprise deployments that focus on a broad range of BI use cases.
Includes products from companies with annual revenues of $200m+ and a truly international reach.
Includes products that focus on analysis in dimensional and hierarchical data models.
Includes products that focus on self-service reporting and ad hoc analysis.
Tasks where BI is used
Tasks where BI is used (n=2,568)
Want to rate your BI software?
Participate in the world‘s largest and most comprehensive survey of BI software users
Take part in The BI Survey
BARC Score Business Intelligence: The Analyst Comparison of Leading BI Tools
BARC Score Business Intelligence compares the strengths and challenges of all the leading vendors in the business intelligence market as well as many smaller vendors that often have less visibility but equally offer outstanding value to their customers.
Based on countless data points and many analyst interactions, business intelligence software vendors are rated on a variety of criteria, from product capabilities and architecture to sales and marketing strategy, financial performance and customer feedback
Calculating the individual ratings for all criteria and all vendors produces two scores per company: the ‘portfolio capabilities’ score and the ‘market execution’ score, which are plotted on each axis resulting in the vendor’s dot on the BARC Score graphic.
‘Portfolio Capabilities’ includes a series of weighted criteria based on three major areas: functionality, infrastructure and product-related evaluation criteria.
‘Market Execution’ takes into account a blend of product, sales and marketing strategy as well as certain organizational, financial and geographical considerations.
There are two separate inclusion criteria to be reviewed in the BARC Score: the first is associated with a software vendor’s products and the other is linked to the financial results relating to their BI tools. To be evaluated in this BARC Score, a vendor has to have a strong focus on providing BI functionality and supply four out of six technologies from the following functional portfolio:
- Formatted and Ad Hoc Reporting
- Dashboards
- Analysis
- Data Mining
- Planning
- Self-Service BI and Data Discovery
In addition, the vendor has to generate a minimum of 15 million EUR in license revenue per year with the above product set, spread across at least two separate geographies. As individual geographies we consider:
- Europe, Middle East and Africa
- North America
- Latin America
- Asia/Pacific
Vendors with an open source business model are compared by their total revenue because those companies do not charge a license for their products, but an annual subscription fee.
The BARC Score Business Intelligence 2016
Score Regions
What We Have Learned About Comparing Business Intelligence Software
Why the choice and comparison of business intelligence products is important when it comes to finding the best tool
Business intelligence projects are about creating business value by enabling better business decisions. An important – though not the only – part of this value creation is derived from the BI software products that gather, enrich, store and present data to end users, business analysts, or even data scientists.
Choosing a software product that does not match your requirements can pose a serious threat to the success of your project. Survey results show major differences in the business benefits achieved with different BI tools.
The Business Benefits Index (BBI) is a unique measure used in The BI Survey to determine the value of BI projects. It takes both quantitative and qualitative benefits into account as well as different levels of possible benefit achievement. Here you can find more detailed information on how we calculate the BBI and how we classify best-in-class companies.
Percentage of businesses doing a competitive BI software evaluation (n=2189)
Business benefits by selection type (n=2189)
How to select a BI product
Selecting the best business intelligence software is key to a successful project, and many software problems can be traced back to poor software selection processes.
The software is often used for ten years or more so the total cost of ownership (TCO) can be substantial. Investing time in choosing the best business intelligence software tool will have positive long-term effects.
Before purchasing and committing to any BI tool, we always recommend that organizations conduct detailed, formal evaluations of multiple products on the market.
However, we are aware that not everyone follows this advice. The data collected in The BI Survey allows us to draw some definite conclusions on this topic.
The findings from our survey reflect those stretching back to 2002 in that business benefits improve when a company performs a product evaluation and benefits are further improved if enterprises extend the formal assessment process to a multi-vendor comparison.
What are the right criteria for evaluating BI software
The criteria best-in-class companies use for a selecting their BI software confirm that successful projects are based on selecting the tool rather than the vendor.
This is one of the most consistent findings over The BI Survey’s eighteen-year history.
- Best-in-class companies take more selection criteria into account than laggards.
- Best-in-class companies are much more interested in functionality, ease of use and fast query performance; all key considerations in enabling self-service BI.
- Laggards, on the other hand, are more concerned with non product-related criteria such as ‘vendor listed as corporate standard’ or ‘bundled with another product’.
Reasons for buying BI software tool (n=2434)
Median implementation time (n=2260)
Why implementation speed matters for a BI tool
Countless IT projects experience delays and data warehousing and business intelligence projects are by no means immune to this perennial issue.
Implementation time depends largely on which product is being implemented. Naturally, the complexity of the BI solution and/or the project will impact on project length and certain BI products lend themselves better to more complex projects, thus increasing their average implementation time.
Clearly, other factors such as the number of data sources, the volume of data and the number of users or departments served also have an influence here.
Fast implementation times are key to a successful BI project. Projects taking more than three months to complete lead to significantly lower business benefits. Project implementation times appear to have a direct impact on the level of all business benefits; with benefits decreasing as project length increases.
Aim for a three-month implementation window for the first application. Best-in-class companies prove that it can be done in just two months on average.
If possible, try an incremental approach by breaking the project into a series of smaller projects.
Why good support is especially important
The standard of service provided by consultants varies considerably. The BI Survey asks business intelligence users about their experiences with external implementer support as well as vendor support. We found a clear correlation between customer ratings of implementer and vendor support and business benefits achieved.
Seeing such a big difference in business benefits between different types of consultants does not come as a surprise. Good support from an implementation consultant is an important aspect of project success.
Finding the right vendor in combination with the right implementer is crucial to project success. In previous editions of The BI Survey, we used to ask about the type of consultants companies used. Results revealed that small, specialized consulting firms frequently scored better than large general purpose consulting companies.
The success of business intelligence projects depends upon organizations’ and vendors’ abilities to deliver successful and relatively rapid implementations, along with supportive follow-up and maintenance.
Many BI solutions end up as shelfware within enterprises, due to difficulties in setting up software, difficulty in using tools, or lack of ability to communicate the benefits of the tools and platforms to end users.
The ability to seamlessly incorporate business intelligence software and platforms into corporate decision-making processes and to closely align those solutions into business results is crucial.
No significant problems with the BI tool by
best-in-class companies (n=2586)
Avoiding problems
Survey results clearly reveal that a carefully conducted comparison and selection of business intelligence tools pays off. The organizations that achieve the greatest level of business benefits with business intelligence (best-in-class companies) are much more likely to have conducted a competitive product evaluation, and they experience far fewer problems with their BI software.