The adaptive capacity of an organization depends largely upon how well the organization processes information. It is therefore important that the handling process– the second process of the large Business Intelligence cycle – is well designed and supported within the organization. We define a number of steps within this process. These are shown in following figure.
Figure: The BI cycle consists of 15 process steps where data is converted to information and knowledge.
The handling process of BI can be roughly divided into two parts:
- The first part contains the processing steps 1 to 5 and concerns the automatic conversion of registered signals and data into meaningful management information. After step 5, the probability that users will encounter actual useful information in the data will have drastically increased.
- The second part concerns the non-automated handling process (done by people) and contains steps 6 to 15. During these steps, information is being interpreted, internalized, revised, checked, enriched, shared and so on. Nowadays, this part of the handling process is usually supported using specific Business Intelligence tools like WebFOCUS, Birst and TIBCO, but is usually not fully automated (with the exception of closed-loop applications).
Look before you leap
The following expression: “look before you leap” is true for each process step. In other words: be convinced that KPIs and other indicators do in fact form the compass you use for steering. The indicators we defined previously do not only determine the scope of the process steps (where are we focusing), they also partly determine the final quality of the handling process itself. After all a KPI should be much easier to interpret than just some data containing little or no information.
Getting the first part right can be amply recouped later
With that in mind, the first part of the handling process can ensure that the second part runs faster and smoother with far fewer errors. In short: the quality of the decision-making increases. In most cases, the first part of the process requires the most (visible) effort and energy and is often very complex frequently causing Business Intelligence projects to come to a standstill. According to Gartner 70 to 80% of the funds required to set up an effective Business Intelligence system are spent in the first part. Every hour spent on getting this part right can be amply recouped later – in the second part.
Achieve effective Business Intelligence in 15 steps
The process steps are elaborated further on this website from an architecture perspective and the Business Intelligence tools that are available. We will then also address the question of how organizations use these tools in order to achieve effective Business Intelligence. We will discuss the first five steps more thoroughly later and for that reason we will only describe them briefly here.
The handling process is divided into fifteen process steps:
- Collect: signals that are stored as data in various registration systems are collected and copied to a separate system.
- Filter: only data that is relevant for the information needs and for improving performance is let through. Data, which is out of date or of poor quality will not be selected but rejected, or cleansed.
- Combine: the collected data (from the various registration systems) is combined and integrated so creating a single version of the truth and a common source for decision-making. This step also includes various calculations and transformations in order to create indicators.
- Aggregate: the detailed data is now aggregated and then summarized to enable users to access the information they need quickly.
- Visualize: data is visualized in such a manner that interpreting and internalizing information can be carried out much faster. In the section ‘Visualization and simulation’ we will elaborate further on this process.
- Interpret: What does this piece of data mean? What does it refer to? What is it about? These questions (and others) arise during the interpretation process and relate to the context of data: to be able to interpret – how we conceive things – you must know the place, time, measurements, quantity and size to which a piece of data refers. A line chart shows a market share line that suddenly drops in the penultimate time period. Is this because of a rising or falling sales volume? By how much does the market share drop? How long is the exact time period? How much money is involved? What is the scale of the Y-axis? Which products are included in the calculations? and, which market is this anyway? The answers to all these questions together form the context of this piece of data. The more complete the context, the easier we can interpret that piece of data. Moreover, we should beware not to exaggerate and add all sorts of context to data when we may assume that the meaning is already very clear to everyone.
- Internalize: once we understand the context of data, we start the process of internalization. This means that we try to understand (internalize) the message behind the data. In doing so, the data becomes actual information. During this process we read, and read again and we look and keep looking at the ‘picture’ until we grasp its meaning and ultimately embed it in our memory. Fast as lighting, we will run through existing structures and connections in our minds so that new knowledge and information can be connected to what we already know. We are, as it were, trying to fit this new information into our brain, which either happens comparatively easily or with some considerable amount of effort During the internalization process, people should ideally be aware of which fixed beliefs and assumptions they, unknowingly or in fact knowingly, use. Unfortunately, various studies show that we tend to see what we want to see. (See also ‘The 10 Biggest Biases every decision maker should be aware of’)
- Revise: during this step we revise, change or delete existing information and knowledge based on new information. For example: when a new product recently introduced into the market instantly generates the highest revenue, the top 10 bestselling product list should be altered. In this case, the newcomer should replace the existing number one on the bestselling product chart. Newly available information may lead to rejecting old (or older) ideas and (assumed) certainties.
- Check: new insights and knowledge should be checked, for example by comparing a negative development of market share with sales development. If the sales development is positive but the market share drops, the market may have increased. It is also possible that a mistake has been made during the process of transforming the original data into new information in the Business Intelligence system.
- Enrich: both newly gained insights and changed insights are documented. An explanatory text that accompanies the development of, for example, market share indicates that their decline is almost entirely due to two new players in the rapidly growing market and not so much the result of falling sales. Enriching also includes combining external information with internal information to support new insights. In the previous example, an organization will carry out further research into these new players in the market and combine the results with market share reports.
- Share: sharing and communicating new insights and knowledge is hardly possible if an organization does not go through the previous ten steps properly or skips one or two. Information can only be communicated well and quickly if it is – in a manner of speaking – completely experienced and chewed upon. In Bacon’s theories (1620) dialogue is an important tool in preventing false assumptions and interpretations. Kessel (1997) applies this principle consciously to organizations. Dialogue creates certain nuances and confronts the communicator with people who may have a different interpretation (Social BI). It goes without saying that a culture of power in which managers are mostly surrounded by ‘yes-men’ does not belong in an Intelligent organization. However, the complete opposite – a culture that lacks decisive action, where everyone is being listened to and in which decisions keep being postponed – does not belong to the Intelligent organization either. Communication also serves as a means of additional enrichment and good communication requires knowing which people possess which existing knowledge and where they can be reached.
- Remember: once the new signals have been interpreted, internalized, revised, checked and enriched, they need to be stored. After all, a signal may require further analyses because it is not yet entirely clear or because it is not useful now but might be in the near future when new signals enter the organization. Therefore, these signals should be parked, or in other words: we delay the decision and the corresponding action. The decision to launch a new service program because two customers walk out due to poor service is not made straightaway. However, when a news story states that the competition delivers much better service, the organization will feel the need to take action.
- Decide: once information is sufficiently clear and no further analysis is needed, the final decision can be made. However, this is almost never the case. Usually we need more information to make informed decisions or we need more time to first discuss the consequences; and sometimes we simply must await developments. Decision-making may for example relate to the discontinuation of a poorly selling product; focusing your activities on the top 100 customers; or, moving parts of the business processes abroad. Withdrawing a poorly selling product from the market, without sufficient though and without further analysis, is not wise, it may be that your most loyal and profitable customers are purchasing this specific product. Many decisions are actually often barely visible: how often have you actually come across a manager making decisions (alone)? Most decisions are suddenly presented as if they are obvious or they are just introduced with little fuss, but in reality a whole process of weighing pros and cons, discussing and testing preceded this decision. It is only afterwards that we can see that the organization made a decision (and not that specific manager) and has changed course. Political behavior, as shown by managers in particular, and the fact that strategies and opportunities sometimes present themselves naturally, often form the basis of this phenomenon. Managing based on intuition also plays an important role. One cannot always explain things. Simple as that.
- Distribute: Newly gained insights should be distributed throughout the organization so that all stakeholders can actually use them. In this process stage, the new insights have been validated for accuracy and are suitable for decision-making. Also at this stage, it should be clear to whom – the places and positions within or outside the organization – the new insights should be distributed and for what reason. Additionally, possible decisions should, ideally, be communicated to all staff concerned.
- Anticipate: research into the ‘pros and cons’ of a decision – whether or not expressed in specific costs and benefits – is often part of the decision making process itself. When making important and far-reaching decisions an organization will undoubtedly wish to investigate the potential implications for the business processes and anticipate changes. Hence, the organization creates an approach in which the consequences are worked out and which describes the process of getting out of the current, undesirable situation and into a new desirable situation.
The more conscientiously the steps are carried out; the greater the return of investment
Quite often organizations do not execute the abovementioned steps or take them unknowingly. The steps are in fact – just as decisions – not clearly visible. All in all the process is rather diffuse. This does not alter the fact that each step contributes to a better response to the original signals and the more conscientiously the steps are carried out; the greater the return of investment in Business Intelligence. Unfortunately, managers often seem to lack the time or the priority. Here is an important role for business analysts; they can support and advise managers in getting things together. This naturally assumes that business analysts should be able to delve into the world of their managers.
A certain natural sequence
The process steps as outlined above have a certain natural sequence, which in practice can vary. Logically speaking, internalization takes place after interpretation and checking should be done after something is internalized. Sometimes certain steps should be revisited. During step 9 (checking) certain errors made during the interpretation process may surface. In that case, the user must repeat step 6 and 7. If certain errors frequently reoccur, an organization would do well to adjust the Business Intelligence system (or to replace the manager in question).
Additionally, some data may be irrelevant at the time and therefore should be stored for future reference. Other information might not comply with our set of requirements and consequently the process comes to a standstill. Finally, some information is not relevant at all and therefore disruptive.