How do BI projects differ from traditional system development projects?
This article discusses the required project approach to the competencies and roles needed to merge the Business Intelligence processes, the architecture, the tools, and the applications into a lasting, working entity. Business Intelligence projects differ from ‘traditional’ system development projects in several ways, namely:
They often go beyond the boundaries of departments, processes and even business units;
- They contain a mixture of strategy, business operations and technology and are usually very political;
- Business and IT need to cooperate very closely;
- Business Intelligence systems are for the most part derived from the operational systems and relate not only to the operational level but also to the tactical and strategic levels within organizations;
- They contain a relatively large number of risks and therefore a high probability of failure;
- They require a combination of two methods: the traditional ‘waterfall’ method and the modern iterative method (incremental development). The first method is needed for both information analysis and a balanced architecture, whereas the latter realizes the different increments;
- BI projects require a multi-disciplinary project team.
These aspects make BI projects difficult. For that reason, we require a separate and specific project approach: the Business Intelligence project cycle.
What makes or breaks a BI project?
Once the organization has defined and prioritized its projects, it is important that we know exactly how we can make the Business Intelligence project – the first one in particular – successful. What makes or breaks a BI project?
What steps do we need to take (and in which order) to achieve success? How do we navigate the project? This is where we define the BI project cycle: a cycle that has proven itself in practice in many organizations, particularly those that started using Business Intelligence seriously after an initial trial period. The project cycle takes into account the most important risks and exploits new opportunities as they arise. It also ensures that the organization continues to learn from its BI projects.
BI projects often provide unexpected opportunities that we have to take advantage of. After all, it’s all about flexible reporting and analysis. Flexibility comes with risks – for example when users get bogged down in the many possibilities BI has to offer – but also offers the organization many opportunities to increase the return on registered data. This fact alone is a plea to reserve sufficient (additional) time and money in order to exploit these unexpected chances, for example in future increments.
Figure: The BI project cycle is made up of 12 phases: F.D stands for Functional Design.
1. From awareness to blueprint (phase 1 to 8): the project cycle begins with awareness: what are the benefits of BI? We determine the business case, the scope and a project plan. After this, we describe the (performance) indicators and the associated architecture and we select the Business Intelligence tools. The blueprint has an integral character and is intended to manage and feed multiple increments (sub-projects).
2. The increments (phase 9 to 11): during these phases, we design, build and test. We create a part of the blueprint and we build and test the system. The activities of the increments are divided into three elements:
– Data warehouse and ETL: extracting data from the source systems, integrating the data into a data warehouse and aggregating the data in data marts;
– reporting and analysis: designing and developing dashboards, reports, and analyses based on the data in the data marts;
– distribution: ensuring that reports and analyses are in fact distributed throughout the organization.
For a large part, these three elements can be executed simultaneously (in parallel), whereby the first element often requires the most effort and the most risk.
Hint: if most of the logic (or knowledge) is already present in the data warehouse, then it is usually relatively easy to build reports without first having to create a detailed design. This even applies to more complex reports.
3. Embedding and evaluation (phase 12): the project organization will ensure that the project results are embedded in the business operations and encourage – or where necessary try to enforce – effective usage of these results, to achieve the highest possible return on the project results. Additionally, the project organization evaluates the project and determines the content of the following increments. If necessary, the architecture and tools are also evaluated.
The BI project cycle concentrates mainly on the specific activities and products of Business Intelligence projects, thereby using elements from existing standard project management methods – such as working with business cases from Prince II and DSDM.
Business Intelligence and data warehouse projects often fail
This is partly because the aforementioned aspects make them high-risk. Various studies prove that Business Intelligence projects show an average return of investment (ROI) of between 17% and 2000% (IDC, 2002; Wu, 2000). However, at the same time, many Business Intelligence projects (50-70%) fail or do not live up to expectations. They don’t produce the desired results.
How can we explain this contradiction? There are, in fact, two explanations:
- The figures don’t include the projects that have failed, either completely or almost completely,
- If a project does succeed, the returns can be enormous. The question therefore is not so much whether we should invest in Business Intelligence, but how we can influence the success factors and monitor the probabilities of failure.
The high failure rate is often because no proper business case was drawn up, even though there may have been one, had we looked for it. It is thus important that we properly prepare projects, use a good project approach, identify risks and success factors and put together a balanced and knowledgeable project team. This actually applies to nearly all projects. For that reason, we will elaborate these issues further in the next articles, but in this case specifically for Business Intelligence projects.
Our practical Business Intelligence training course covers KPIs, analytics, BI Project Cycle Management, Big Data, change management, data visualization, and BI success factors. This training will let you take the next step in BI Project cycles and take on a leading and advisory role. Read more…
Business Intelligence awareness
The Business Intelligence project cycle begins with awareness: becoming aware of the benefits and the ‘why of BI’.
Business Intelligence systems and the associated concepts and technologies actually differ significantly from traditional information systems supporting the operational level of the organization. The latter are not very flexible: mostly we can only choose one single dimension or indicator (or a combination) simultaneously.
BI systems, on the other hand, provide end users with the option to determine ‘on-the-fly’ which views and indicators they wish to combine in a report or analysis. This allows users to ‘juggle’ with the data – if the response time is fast enough – and to gain insight into the business operations quickly.
Hence, Business Intelligence systems essentially differ from conventional information systems (with respect to design and usage). Organizations that wish to start using Business Intelligence but have not yet gained experience in this area are wise to attend supplier demonstrations, to study demo versions, to experiment and to visit colleagues in the industry who have already gained knowledge and experience with Business Intelligence.
In practice, we quite often see that future users – for different reasons, often political – tend to position or label the data warehouse as a ‘stupid box’ (data-driven approach). These users are usually ‘afraid’ to relinquish control and often wish to perform their own calculations and analysis. Lastly, but just as important, is the fact that Business Intelligence can create a shared single version of the truth and increase transparency. The awareness of the essence of Business Intelligence, its advantages and the costs and benefits, must be further developed – and preferably quantified – in the next phase of the project cycle.
How to manage Business Intelligence projects?
How to manage Business Intelligence projects? Business Intelligence projects require a specific project approach because they significantly differ from traditional system development projects on a number of aspects.
Be aware of the nature of BI
The BI project cycle starts with awareness: we need to be aware of the nature and character of Business Intelligence. Once this is clear, we start looking for the business cases and we determine the scope. The blueprint of the BI system is created by using the basic principles that govern the BI architecture whilst defining the indicators and dimensions (information analysis) based on which we create a functional design that ultimately leads towards both a balanced architecture and a suitable data model.
Data quality audit
The organization will also perform a data quality audit (in order to determine the quality of data) and select the required tools that both fit the architecture and match the user requirements. During the first eight phases, we use the waterfall methodology. In the phases that follow – detail design, build and test – we use the incremental method. Ultimately, the Intelligent Organization will ensure that the project results are embedded in the business and evaluated.
Start with a business case
We must have a well written business case: stating what the project contributes to the improvement and development of the organization. When we start using BI, we must determine a scope that allows us to create a business case after the first or the second sub-project has been completed and put into production. Such a scope should contain roughly 20% of the total information needs of the organization. The main purpose of information analysis: what information do we need in order to improve the results and the processes and to achieve our goals faster?
Matrix of all indicators and dimensions
The functional design includes a matrix and the definitions of indicators and dimensions. It also describes the correlations between indicators and dimensions. Building a prototype is recommended whilst designing the architecture. The selection criteria are subdivided into ‘need to have’ and ‘nice to have’ so to reduce the chances that good products are rejected just because they score poorly on aspects are not essential. In the detailed design, we describe the transformation and loading processes, which we will then realize during the build phase and test during the test phase. After approval by the users, the BI system is rolled out and embedded in the organization after which it can be put into production.
High risk projects
BI projects are characterized by an extremely high risk factor and many obstacles. These obstacles are mainly related to the fact that Business Intelligence projects typically go beyond the boundaries of departments, processes and even business units; contain a mix of strategy, business operations and technology; are often highly political.
Make or break a BI project
In addition, the Business Intelligence systems use data from the operational information systems. The obstacles refer to the most common and most important powers and risks that can make or break a Business Intelligence project. The obstacles refer to four ‘faulty’ behavioral cultures, the complexity of ETL, the ‘technology’-push and to the amount, the quality and the absence of data. The most critical success factors of Business Intelligence mainly deal with how we use information for analysis and proper action, and whether we regularly evaluate and adjust standards and targets.
Effective BI teams
Effective BI project teams are characterized by their multidisciplinary nature. For a multidisciplinary project team to function properly, it is necessary that adjacent roles have sufficient knowledge of each other’s problems, experience, ambitions, plans and expertise. We should particularly focus on the different ideas and visions that project members may have with respect to the architecture.
Small BI projects
In BI projects, we distinguish between basic roles and additional roles. Small projects can usually suffice with the basic roles. Most successful BI projects are managed by a project manager who has knowledge of both business and technology. These projects are characterized by the continuous interaction between IT and business operations. BI projects that are solely data-driven or technology-driven are more likely to fail.
Contact us for advice about BI Projects
If you want to learn more about how to properly set up a BI Project cycle, feel free to contact us. We’re also available as BI consultants, or you can take a look at our BI training course or BI Tools Survey.