How to select BI software
Every company is unique and has its own particular software requirements. But despite this variety, companies tend to make the same mistakes during the software selection process.
This article series sets out guidelines and best practices to support the process of selecting and evaluating business intelligence software; recommending criteria to be used in software selection and how a multi-step process is key to the successful choice of a product fit for purpose.
Note that this series of articles does not recommend which product to select, nor does it describe the relative merits and weaknesses of particular products.
Multi-step BI software selection process
Selecting the right software is key to a successful project and many software problems can be traced back to poor software evaluation processes. Software is often used for ten years or more so the total cost of ownership (TCO) can be substantial. Investing time in choosing the right product will have positive long-term effects.
On a basic level, selecting and comparing BI software solutions is no different to buying any other durable consumer product such as a vacuum cleaner or washing machine. However, the challenge of buying the right product can be complicated by the multitude of different cross-departmental needs requiring consideration.
Large software projects often suffer from changing requirements during their lifespan. A well-defined software evaluation process along with robust best practices for documenting results will help maintain project focus in later stages.
Project challenges when selecting software
Costs associated with the software selection process are dependent on the extent of research and analysis carried out by the project team and whether input from an external consultant is required.
The software market is large and the needs of each organization different. Because of these factors it can be difficult to attribute accurate costs to the software evaluation process.
However, it is important to remember when justifying selection process costs that choosing the wrong software can lead to a significant reduction in the return on investment or in extreme cases, the failure of the entire project.
Decisions based on ‘strategic’ factors (such as those listed in the next paragraph) often lead to difficulties in later projects. Although it often makes sense to base decisions on long-term considerations, the word ‘strategic’ is all too often misused as an excuse to go through with poorly conceived plans while avoiding detailed analysis of the issue at hand.
The software evaluation process is carried out at an organizational level, rather than at the individual level, often giving rise to inter-departmental political issues. Listed below are some examples of factors that drive decision-making:
- Personal preferences of project leaders, influential external advisors or company management;
- Software vendors or systems integrators who are well established in the company;
- Time and budget constraints in the selection process itself.
The project team for evaluating software
One individual alone rarely possesses the breadth of knowledge and competency to choose a software option capable of meeting all an organization’s requirements. Software selection should always involve agreed representatives from user and IT groups.
Communication channels across organizations can be fraught. Differing departmental objectives and a lack of collaboration can lead to strange product choices, resulting from bizarre shortlists based on contradictory business and technical needs. Such projects are prone to failure and often end users will avoid the new software, instead falling back on those tools it sought to replace.
Self-service BI is the way forward for many business users attracted by the convenience of performing tasks without the support of IT.
However, many users fail to take a holistic view of organizational needs when expressing their wishes. Absolute end user autonomy can have chaotic results when a company is striving to achieve data consistency across a department, organization or group of organizations.
Successful autonomy, provided in a self-service approach, must be underpinned by solid data governance; defining roles, responsibilities and general guidelines on data usage.
Business users are prone to making product selections based on unimportant and seldom-used highly interactive visual analysis components or highly specialized analysis features. Users also fail to judge their data management requirements and capabilities correctly.
The introduction of an unbiased neutral party into the software selection process can help strike the right balance between end-user and IT requirements.
This could be a high-level sponsor with no preference as to which software is used or an independent external consultant who specializes in selection and evaluation, does not offer implementation services and is unaffiliated with any software company.
Want to rate your BI and analytics software?
Share your experience in the world‘s largest and most comprehensive survey of BI software users
Take part in The BI & Analytics Survey
The 4 essentials steps in the BI software selection process
The best practice to evaluate BI software involves a multi-step process comprising the following steps.
- Definition of strategy and goals
- Requirements analysis (workshop, weighting, common list of requirements)
- Software evaluation
- Implement the product
Recommended steps and best practice for selecting BI software
Definition of strategy and setting goals
A project team should define clear business goals in the initial phase of the selection process. This will ensure the level of success or failure of the project can be properly measured.
Software selection projects are complex and usually involve two separate dialogs; technical and business. Defining a successful business intelligence strategy and related project goals should comprise specification of the technical, business and organizational aspects of the project.
Defining a Business Intelligence strategy
Best practice
Requirements analysis
Identifying requirements is a key stage in the software selection process. A list of criteria is one of the main outputs of the project. Defining and working with the criteria is a good way of involving key players in the process and improving their acceptance of the final product selection.
The process of defining criteria can often simplify projects, reducing the cost and increasing the chances of success. It also identifies just who is backing the project, and concentrates their minds if they know that the effects will be measured and not just assumed.
List of criteria
The list of criteria when selecting BI software should include criteria relating to architecture, data sources, administration, information delivery, user interface, report development, performance, user-defined content, analysis and planning. When defining the requirements of a project, individuals and departments should weight the criteria according to their needs.
When creating a catalog of requirements, technical, business and organizational goals and requirements should all be considered carefully.
The whole project team should be involved in the process, which must be documented to ensure that nothing is missed. Where possible, decisions on the criteria should be approved by everyone on the team.
Once the basic requirements and criteria of the project have been established, ‘must-have’ features should be identified and prioritized over ‘nice-to-have’ features before creating a final requirements list.
As the project progresses the criteria may change. This can be problematic if the implementation has already begun; in fact changing criteria can cause a project to fail.
But it is not necessarily a bad thing if changes arise during the software evaluation process. As project members become more familiar with the products on the shortlist, their attitude and understanding of their own requirements may very well change.
Best practice
The output of this process is a weighted list of criteria that allows the project team to reduce the number of vendors to a shortlist for final tool selection.
Software Evaluation – Analyzing the market and making a shortlist
Once project goals have been defined and requirements identified and weighted, the actual evaluation of the product can begin.
The BI software market is complex, with several hundred vendors to choose from and new vendors continually appearing with new solutions. On average, each vendor has 2-3 products in their portfolio.
The market is characterized by the big international players but also by a multitude of smaller players, often focusing on specific geographical regions or specializing in certain application areas.
The software evaluation process has three different steps. Initially, a general screening of the market will identify possible suitable vendors before these contenders are narrowed down to a shortlist of 3-5 products. In the final step the selection can then be made from this shortlist.
Steps for a software evaluation
In each step, the level of detailed analysis increases but the number of software products reduces, as shown above.
Market Analysis
The first step in the product selection process is a wide-ranging market analysis providing an overview of the market.
At this stage, the market analysis should already be focused on the type of application required (eg. analysis, reporting, planning etc.). The goal of this step is to identify products that look like a reasonable fit against criteria defined by users.
Our consultants find many BI evaluations end up comparing apples with oranges because the selection team has created a shortlist too quickly without first understanding the business issues.
In particular, buyers often allow vendors to educate them about the market; a dangerous route to follow as vendors will invariably have their own interests at heart.
Shortlist
The most common problem during software selection is a poorly defined shortlist.
Our peer groups should help you ensure that the products on your list are comparable. It is also worth exploring independent sources of market information such as our Vendor Performance Summaries of each of the business intelligence products featured in The BI Survey, which provide an unbiased and insightful source of information underpinned by expert analysis.
In our peer groups, business intelligence products are grouped according to their functionality and regional focus. The following tabs show how business intelligence products in The BI Survey 16 were divided and allocated into peer groups.
Includes products that focus on ad hoc query, data navigation and analysis.
BI & Analytics specialists are software vendors who focus solely on BI and/or analytics. Often, they have just one product in their portfolio.
Business software generalists have a broad product portfolio that is not limited to BI and analytics, including most (or all) types of enterprise software for a variety of business requirements (e.g., ERP).
Includes reporting and analytics products that can be embedded in other business applications (e.g., ERP systems).
Includes products from companies with annual revenues of $200m+ and a truly international reach (partner ecosystem, on-site locations, global installations and revenues).
Products in this peer group are typically (but not exclusively) used in large scenarios and/or enterprise-wide implementations with a large number of users and data volumes.
Products in this peer group are typically (but not exclusively) used in small and midsize scenarios and/or departmental implementations with a moderate number of users and data volumes.
Includes products that focus on the creation and distribution of standardized and governed content such as dashboards or reports.
Best practice
In step two, narrow your list down to 3-5 tools by concentrating on ‘must-have’ and key criteria.
Evaluate these remaining tools in detail against all criteria and test them in a proof of concept.