Finding the Right Business Intelligence Tool is Not So Easy
Which is the best business intelligence (BI) tool? What makes it the best one? What do existing customers think of their BI tools? Is the best business intelligence software also the right one for my business? Which criteria are the most important when comparing BI tools? Is superior functionality the main factor to consider? Or are aspects like vendor and implementer support equally important? And how do independent analysts rate different business intelligence solutions?
Comparing BI tools is no easy task. Especially since there are so many business intelligence products on the market, all promising they will deliver all sorts of business benefits while at the same time solving all manner of problems.
Naturally, companies searching for a new business intelligence software tool can become overwhelmed with all the choices and promises from vendors.
To make the comparison and search for the best BI tool for your company easier, this article sets out to provide you with a balanced view of what BI users – as well as BARC analysts – have to say about the leading business intelligence products on the market.
By combining our in-depth knowledge of current BI software tools with one of the largest databases of user reviews of business intelligence software, we aim to guide you through the first steps to finding the best BI tool for your company.
We believe a combined view of user feedback and in-depth analyst perspective is necessary in order to fully appreciate how business intelligence software tools compare against each other, and to understand which are the best BI products for your company.
In this article you will find out:
- which are the best BI software tools as rated by users in 2017 (BI User Review Matrix)
- how BARC analysts compare the products (BARC Score Business Intelligence)
- the most important lessons learned from surveying more than 30,000 respondents about business intelligence software comparison and selection over the last 15 years
Table of Content
The BI Survey: The User Comparison
No one knows more about how a business intelligence tool performs in the real world than the customers already using it. All too often, they find that products don’t live up to expectations, or that the business intelligence vendor does not support the BI tool properly. Therefore, when comparing BI software, there is almost nothing more valuable than user reviews.
This is where The BI Survey comes in. Collecting feedback from business intelligence software users every year for the last 15 years, The BI Survey provides detailed analysis of which BI software products get the best user ratings. This year’s BI Survey is based on the analysis of the real-world experience of 3,137 respondents with 2,611 answering a series of detailed questions about their usage of a named business intelligence software tool.
It is the largest and most thorough fact-based analysis of the BI market and software currently available, using over 15 years of experience to analyze market trends and challenge some of the myths surrounding the business intelligence software industry.
This edition features a broad range of BI software tools, not only from the well-known business intelligence giants, but also specialist products from much smaller vendors and open source vendors, that ordinarily don’t get much press but which, in many cases, offer outstanding value to their customers.
Altogether, the BI Survey compares 37 business intelligence tools (or groups of products) in detail. To be included in the detailed analysis a BI tool needs at least 30 user reviews.
Since not all business intelligence tools are alike, we use peer groups to ensure similar products are compared against each other both in fairness to the vendor and for the benefit of readers (see below for more information on the peer groups). The groups are essential to allow fair and useful comparisons of BI tools that are likely to compete.
The BI User Review Matrix rates BI software based on the business value the software provides (x-axis), customer/user experience (y-axis), customer satisfaction (color) and competitiveness (circle size). To see the comparison in action, please select a peer group.
The KPIs
The ‘Business value‘ KPI shows how a successful BI software product can provide these benefits in the real world. The KPI combines the ‘Business benefits’, ‘Project success’, ‘Price-to-value perception’, ‘Project length’ and ‘Innovation’ KPIs.
To calculate the quality of user experience of a BI tool, we combine the ‘Ease of use’, ‘Self-service’, ‘Performance satisfaction’, ‘Flexibility for users’ and ‘Recommendation’ KPIs.
Recognizing which business intelligence software to compare entails understanding which BI tools have fared well in other organizations’ product selections. This enables users to eliminate ‘losers’ at an early stage in the selection process.
The KPI rules
The BI Survey BI User Review Matrix
Comparing BI Software with the help of peer groups
Not all BI tools are alike. Some are strong on reporting, while others are more suitable for dashboards or analytics.
Results from The BI Survey show the types of capabilities respondents use with their BI tools. Standard and enterprise reporting is top of the list followed by ad hoc query. The least popular BI capability this year is advanced analysis (e.g. predictive analytics or data mining), although it has the highest ‘planned’ rates, with 45 percent of respondents intending to start using advanced analytics in the future.
The peer groups are primarily based on the results from The Survey. They take into account how customers say they use the business intelligence software, which can vary widely across the different products. But we also include the experience and judgment of the BARC analysts in deciding on the groupings.
Peer groups are simply a guide to the reader to help make BI tools easier to compare and to show why individual products return such disparate results. They are not intended to be a judgment of the quality of the BI tools.
Some products appear in more than one peer group. We use these peer groups in a consistent way in our analysis, as well as in The BI Survey Analyzer.
The point to the peer groups is to make sure that the comparisons of the BI tools we make in The Survey make sense. The products are grouped together as we would expect them to appear in a short list for business intelligence software selection.
To make a proper choice, a buyer should first segment the market into the BI tool types that fit his requirements. The peer groups are intended to help with this task.
The segmentation of the peer groups is based on two key factors:
- Usage scenario – functional peer groups – mainly data-driven and based on how customers say they use the product.
- Regional focus – is the vendor a large international vendor with a truly global presence or does it focus on a particular region? We also take into account the location of BI Survey respondents.
The regional focus is of particular importance since organizations require local contacts, resources and knowledge from implementers to conduct projects in their own country. Furthermore, locally competing vendors and markets are better reflected and easier to compare with this method of market segmentation. To compare vendors with global coverage, a peer group featuring only large international vendors was created.
The peer groups
Includes products from vendors that have a significant presence in - and focus on - the Americas region.
Products that focus on creating advanced and highly sophisticated dashboards.
Includes products from vendors that have a significant presence in - and focus on - the EMEA region.
Includes products that provide integrated functionality for BI and performance management, especially planning and budgeting.
Includes products equipped with functionality for enterprise deployments that focus on a broad range of BI use cases.
Includes products from companies with annual revenues of $200m+ and a truly international reach.
- IBM Cognos Analytics
- IBM Cognos TM1
- Infor BI
- Information Builders WebFOCUS
- Microsoft Excel
- Microsoft Power BI
- Microsoft SQL Server Reporting Services (SSRS)
- MicroStrategy Analytics Platform
- Oracle BI
- Qlik Sense
- QlikView
- SAP Business Explorer (BEx)
- SAP BusinessObjects Analysis (Office)
- SAP BusinessObjects Design Studio
- SAP BusinessObjects Web Intelligence
- SAS Enterprise BI Server
- Tableau Server and Tableau Desktop
- TIBCO Spotfire
Includes products that focus on analysis in dimensional and hierarchical data models.
Includes products that focus on self-service reporting and ad hoc analysis.
Includes products that focus on visual data discovery and advanced data visualization.
Tasks where BI is used
Tasks where BI is used (n=2618)
Want to rate your BI software?
Participate in the world‘s largest and most comprehensive survey of BI software users
Take part in The BI Survey
BARC Score Business Intelligence: The Analyst Comparison of leading BI tools
BARC Score Business Intelligence compares the strengths and challenges of all the leading vendors in the business intelligence market as well as many smaller vendors that often have less visibility but equally offer outstanding value to their customers.
Based on countless data points and many analyst interactions, business intelligence software vendors are rated on a variety of criteria, from product capabilities and architecture to sales and marketing strategy, financial performance and customer feedback
Calculating the individual ratings for all criteria and all vendors produces two scores per company: the ‘portfolio capabilities’ score and the ‘market execution’ score, which are plotted on each axis resulting in the vendor’s dot on the BARC Score graphic.
‘Portfolio Capabilities’ includes a series of weighted sub-criteria based on functional and technological capabilities, as well as the overall architecture and product satisfaction from an end-user point of view.
‘Market Execution’ takes into account a blend of product, sales, and marketing strategy as well as certain organizational, financial and geographical considerations.
There are two separate inclusion criteria to be reviewed in the BARC Score: the first is associated with a business intelligence software vendor’s products and the other is linked to the financial results relating to those BI tools. To be evaluated in the BARC Score Business Intelligence, a vendor has to supply four out of five technologies from the following platform portfolio:
- Formatted Reporting
- Dashboarding
- Ad hoc Query and Reporting
- Analysis
- Data Mining
In addition, the vendor has to generate a minimum of 15 million EUR in license revenue per year with the above product set, spread across at least two separate geographies. As individual geographies we consider:
- Europe, Middle East and Africa
- North America
- Latin America
- Asia/Pacific
Vendors with an open source business model are compared by their total revenue because those companies do not charge a license for their products, but an annual subscription fee.
The BARC Score Business Intelligence
The categories
What We’ve Learned About Comparing Business Intelligence Software
Why the choice and comparison of business intelligence products is important when it comes to finding the best BI tool
Business intelligence projects are about creating business value by enabling better business decisions. An important – though not the only – part of this value creation is derived from the BI software products that gather, enrich, store and present data to end users, business analysts, or even data scientists.
Choosing a software product that doesn’t match your requirements can pose a serious threat to the success of your project. Survey results show major differences in the business benefits achieved with different BI tools.
The Business Benefits Index (BBI) is a unique measure used in The BI Survey to determine the value of BI projects. It takes both quantitative and qualitative benefits into account as well as different levels of possible benefit achievement. Here you can find more detailed information on how we calculate the BBI and how we classify best-in-class companies.
Percentage of businesses doing a competitive BI software evaluation (n=2189)
Business benefits by selection type (n=2189)
How to select a BI product
Selecting the best business intelligence software is key to a successful project, and many software problems can be traced back to poor software selection processes.
The software is often used for ten years or more so the total cost of ownership (TCO) can be substantial. Investing time in choosing the best business intelligence software tool will have positive long-term effects.
Before purchasing and committing to any BI tool, we always recommend that organizations conduct detailed, formal evaluations of multiple BI tools on the market.
However, we are aware that not everyone follows this advice. The data collected in The BI Survey allows us to draw some definite conclusions on this topic.
The findings from this year’s Survey reflect those stretching back to 2002 in that business benefits improve when a company performs a product evaluation and benefits are further improved if enterprises extend the formal assessment process to a multi-vendor comparison.
What are the right criteria for evaluating BI software
The criteria best-in-class companies use for a selecting their BI software confirm that successful projects are based on selecting the tool rather than the vendor.
This is one of the most consistent findings over The BI Survey’s fourteen-year history.
- Best-in-class companies take more selection criteria into account than laggards.
- Best-in-class companies are much more interested in functionality, ease of use and fast query performance; all key considerations in enabling self-service BI.
- Laggards, on the other hand, are more concerned with non product-related criteria such as ‘vendor listed as corporate standard’ or ‘bundled with another product’.
Reasons for buying BI software tool (n=2434)
Median implementation time (n=2260)
Why implementation speed matters for a BI tool
Countless IT projects experience delays and data warehousing and business intelligence projects are by no means immune to this perennial issue.
Implementation time depends largely on the product being implemented. Naturally, the complexity of the BI solution and/or the project will impact on project length and certain BI products lend themselves better to more complex projects, thus increasing their average implementation time.
Clearly, other factors such as the number of data sources, the volume of data and the number of users or departments served also have an influence here.
Fast implementation times are key to a successful BI project. Projects taking more than three months to complete lead to significantly lower business benefits. Project implementation times appear to have a direct impact on the level of all business benefits; with benefits decreasing as project length increases.
Aim for a three months implementation window for the first application. Best-in-Class companies prove that it can be done in just two months on average.
If possible, try an incremental approach by breaking the project into a series of smaller projects.
Why good support is especially important
The standard of service provided by consultants varies considerably. The BI Survey asks business intelligence users about their experiences with external implementer support as well as vendor support. We found a clear correlation between customer ratings of implementer and vendor support and business benefits achieved.
Seeing such a big difference in business benefits between different types of consultants does not come as a surprise. Good support from an implementation consultant is an important aspect of project success.
Finding the right vendor in combination with the right implementer is crucial to project success. In previous editions of The BI Survey, we used to ask about the type of consultants companies used. Results revealed that small, specialized consulting firms frequently scored better than large general purpose consulting companies.
The success of business intelligence projects depends upon organizations’ and vendors’ abilities to deliver successful and relatively rapid implementations, along with supportive follow-up and maintenance.
Many BI solutions end up as shelfware within enterprises, due to difficulties in setting up software, difficulty in using tools, or lack of ability to communicate the benefits of the tools and platforms to end-users.
The ability to seamlessly incorporate business intelligence software and platforms into corporate decision-making processes and to closely align those solutions into business results is crucial.
No significant problems with the BI tool by
best-in-class companies (n=2586)
Avoiding problems
Survey results clearly reveal that a carefully conducted comparison and selection of business intelligence tools pays off. The organizations that achieve the greatest level of business benefits with business intelligence (Best-in-class companies) are much more likely to have conducted a competitive BI tool evaluation, and they experience far fewer problems with their business intelligence software.