How to test Business Intelligence systems? | Testing strategy BI
Passionned Group is a leading analyst and consultancy firm specialized in Business Analytics and Business Intelligence. Our passionate advisors assist many organizations in selecting the best Business Analytics Software and applications. Every two years we organize the election of the smartest company.

If we do not predetermine what we test and how we test the Business Intelligence system, then this testing process can be very time consuming.

BI allows flexible reporting

Precisely because Business Intelligence allows flexible reporting, we are initially tempted to test all possible combinations of indicators and dimensions. What we forget is that, even in small Business Intelligence systems with for example ten indicators that are linked to eight dimensions, the number of possible combinations can reach ten million!

Not all combinations can be tested

It is clear that it is not possible to test all these combinations separately. For this reason, it is wise to determine beforehand what combinations must be tested. Creating a thorough test plan that includes a sophisticated test strategy, different testing scenarios and concrete test cases, is important in order to actually round off Business Intelligence projects. It enables us to perform qualitative tests, yet in a pragmatic manner.

Large amounts of data

While testing it is not sensible to start using large amounts of data, simply because it is time consuming: loading a month’s data into the data warehouse, for example, can take several hours. If we discover crucial errors (during testing), all data will needs to be reloaded – after the error has been resolved –, which again takes time.

Start with a small test set

All in all, it is better to start with a small ‘restricted’ test set to save time on the loading and reloading of data. Generally, the major errors will surface and they can be solved quickly. Once these major errors are resolved, we can load larger test sets in order to run performance tests.

Figures might differ from the general ledger

It is quite possible that the acceptance test shows that there are differences between the figures in the Business Intelligence system and those in the general ledger, for example. It often happens that the revenue, for instance, does not exactly match.

Adjust the data warehouse and rerun the tests

While analyzing the loading processes in the data warehouse, we may conclude that these processes have not been developed in accordance with the specifications in both the functional and the technical design. If this is the case, we ‘simply’ adjust the data warehouse and repeat the test.

Open up the interface to the general ledger

However, it is also possible that the load processes are fine and do match the specifications. In that case, we must ‘open up’ the interface between the source system and the general ledger, which may be very interesting! We may discover that the interface – for some reason – does not include certain transactions, or that revenues are manually booked or corrected in the general ledger, outside the operational source system. In short: testing the data warehouse and the Business Intelligence system is difficult.

Additional budget sometimes needed

Therefore reserving additional budget in the planning is to be recommended in case the Business Intelligence system has to be (re-)aligned with other systems such as the general ledger.

When the BI system is rejected

At the end of the testing phase, the users either reject or accept the Business Intelligence system giving reasons either way. The BI system may be rejected due to crucial errors such as incorrect figures, poor performance, poor data quality, poorly designed reports or the fact that the system does not meet the criteria for maintainability and scalability. If the system is rejected, we will need to return to an earlier stage of the project. Which stage that is depends on the reason for rejection.

Go back to the drawing board

In some cases, it is even necessary to go back to the drawing board in order to revise the blueprint. In any case, we will need to carry out a thorough evaluation. Lastly, if only a few –less crucial – key performance indicators are not yet correct, it is wise to persuade the users into at least accepting parts of the Business Intelligence system. This prevents us from remaining in the test phase for weeks on end without the BI system being put into production. However, if we do this, we must be a hundred percent sure that the indicators we do accept are in fact correct and that there are no fundamental errors in the Business Intelligence system.

Comment on this post by Daan van Beek

Your email address will not be published. Required fields are marked *

A selection of our customers

Become a customer with us now

Do you also want to become a customer of ours? We are happy to help you with how to test business intelligence systems? (testing strategy bi) or other things that will make you smarter.

Daan van Beek, Managing Director

DAAN VAN BEEK MSc

Managing Director

contact me directly

Fact sheet

Number of organizations serviced
3180
Number of training courses
3181
Number of participants trained
3182
Overall customer rating
8.9
Number of consultants & teachers
3183
Number of offices
3
Number of years active
14