The BI Architect & the explosion of trends
The Business Intelligence world is changing quickly. Big Data and AI are on the rise and data warehouse automation is increasing in maturity. The role of the BI architect is changing along with it. Traditionally, the BI architect is the one who formulates the BI principles and translates them into requirements and information needs. The goal of this is to establish a future-proof architecture. The question, however, is: is this role still useful, now that BI is undergoing an explosion of trends and developments?
A role that’s becoming increasingly important. Or is it?
On the one hand, the role of the Business Intelligence architect is more important than ever. They follow developments in the field and ensure that the right choices are being made. And they decide how data should be stored in the future in order to make it available, in time, to the entire organization. That’s the BI architect’s job, so their role is becoming increasingly important.
Architectural choices by systems themselves
But on the other hand, thanks to data warehouse automation, choices become simple and entirely automated. And the presence of Big Data is giving rise to data lakes that store all data, so everyone has access to it at all times, and if necessary it can be structured. Self-service BI solutions promise that every user can simply and quickly fulfill their own information needs. Then you won’t really need a BI architect anymore. The systems make the architectural choices independently.
BI architects are indeed still important, but only if they can see the entire BI playing field. The call for a future-proof BI architect is getting louder.
The world of the BI architect is changing quickly
The job and function requirements of the BI architect are changing just as quickly as the technological environment. Important to consider:
- How do you determine the success of a BI architect?
- Which knowledge should a BI architect have to be able to handle the future?
- Which skills should you expect a BI architect to have?
- Where do you find the right BI architect, who can give future-proof advice?
- How can you become a future-proof BI architect?
BI architecture that works for you
Business Intelligence is a rapidly-developing field. The BI department or architect is no longer the one who decides what’s good for the organization. By using self-service BI and automating BI processes, the organization itself introduces new solutions. These are usually not recommended by the architects, but purchased by the business itself.
How do you deal with this changing landscape? And how do you ensure that previous investments, that have to last for years, aren’t undone with one decision, putting your organization back at square one?
New technologies have to be fit inside existing BI environments without it leading to disinvestment. The technologies also have to fulfill the business need to get quick and agile insight into existing and new data sources.
A future-proof BI architect is required to introduce an architecture that’s future-proof as well as quickly implementable and affordable. Until recently this couldn’t be done by siding with traditional BI vendors. That implied the choice of a certain architecture. Nowadays you have to think more often about the type of BI architecture and whether a partial solution still fits within that.
The motto of the BI architect should be: let the architecture work for you instead of letting the architecture determine how the organization should work.
The trends in data architecture & analysis
Self-service BI and Big Data are trends that have been mentioned above, but they’re certainly not all the trends. The trends mentioned here directly influence the function of the BI architect:
- New sources and their transience: traditionally, data warehouse environments are stable and the amount of mutations in the unlocked sources is relatively low. With Big Data and the availability of more and more (public) sources, the total amount of sources and the data volume can grow exponentially. But the downside is that the reliability of the data strongly decreases. How do you ensure a stable DWH environment that can handle the volatile transience of these new data sources?
- Data Warehouse automation is taking off and end-users are doing more programming themselves. Until recently, programming was supposed to be unnecessary when it comes to Business Intelligence. But there are more and more tools that require some amount of programming for end-users of Big Data solutions. DWH automation is automating the ETL process, especially, alongside the generation of the central DWH database.
- Smart algorithms and machine learning determine success. Data is everywhere, and easily accessible. Algorithms have become essential to quickly interpreting data. The rise of machine learning has enabled much more, and quicker, insight into (existing) data. These algorithms and machine learning environments are integrating better with existing environments as time goes on.
- The rise of the Internet of Things (IoT). Sensors are being added at an exponential rate; the internet of things. These are producing increasing quantities of data are are usually a core part of the primary company process. The trick is turning all this data into information.
- Explosion of new tools. The kinds of solutions for the processing and interpretation of data is growing quickly. Tools like Python and R are taking off, but are usually only able to be used in a stand-alone situation. More and more BI vendors are challenging the positions of large companies like IBM and SAP. What is the right choice, not only now, but especially for the future?
- Cloud and scalability. Where until recently the BI solutions often worked in their own environments, Cloud solutions seem to be becoming the new standard for BI. Many companies, like Microsoft, offer solutions for data storage and processing that aren’t available in traditional environments. Given the strong growth of data and sources, the future definitely seems Cloud-based.
This all is only part of the story. The BI domain is undergoing a revolution. The tools for data governance and manipulation are becoming better and better. The functionality of existing BI solutions is increasing. Our Business Intelligence Tools Comparison guide is showing us that the tools are gaining in functionality. We’re also seeing that the new vendors are challenging the traditional vendors with increasing success.
Which skills and knowledge should a BI architect possess?
There are high demands placed on the knowledge and skills of a Business Intelligence architect. In order to build a solid architecture, the architect needs to be up on the latest developments in their field. They especially need to be able to translate it into the practice of their organization. The most important skills they should possess currently are:
- The ability to translate the company’s long-term goals into a future-proof BI architecture.
- Creativity and the ability to think outside the box.
- Strong communicative skills so they can present their ideas to stakeholders convincingly using practical and conceptual arguments.
- Knowledge and experience with the latest developments in BI, like Big Data and the development of algorithms.
- The ability to support developments like data governance and privacy demands with current architecture.
- But especially: they need to be able to resist the short-term goals of others in the organization, without escalating conflicts.
There are more factors, of course. The primary importance is that the BI architect is steadfast when it comes to BI architecture choices. They shouldn’t follow the latest hype. They also shouldn’t get in the way of the development of the business with architecture rules and choices. The BI architect should be a thinker who helps the organization, not a police officer who enforces the rules. This is a tricky balance to strike, and it can often feel like walking on egg shells. But that’s the only way that the BI architect can add the most value to your organization.
Do you want to hire a Business Intelligence architect?
BI architects can be hired anywhere. But Passionned Group only works with highly experienced BI architects. Architects who are able to develop future-proof architectures and communicate them to everyone in the organization. Our unique formula enables us to quickly introduce you to an experienced BI architect.
When you make a request with us, we do our best to contact you within 24 hours. We then create a clear profile of the BI architecture that you’re looking for for your organization. Of course, we also look at what BI challenges you’re facing.
After clarifying the request, we usually introduce two candidates. If you choose one of our candidates, we can usually close out the process within 2 days. The right BI architect can then quickly get started in your organization.
The extras when you hire one of our BI architects
All of our BI architects are very experienced when it comes to business analytics & Big Data. They also possess strong communicative skills. They strive to develop architectures and solutions that let your organization take the next step towards a more data-driven organization.
Passionned Group has been closely following the BI domain for over 10 years. We continuously perform independent research and publish the results in our own knowledge base. We also regularly write about new developments in the BI field. Our entire knowledge base, and all our other research material, is available to your organization.
Passionned Group is 100% independent. That’s why we can give you completely independent, objective advice at all times. The BI architect can always fall back on other senior consultants who have a wide range of experience in other areas. Whether it’s about ETL tooling, BI solutions, or how to deal with Big Data in an organization.
How to become a good BI architect
Developments in the field of BI are happening quickly and demand new insights and skills. Passionned Academy helps (future) BI architects to develop their knowledge and skills further.
If you want personalized advice, you can contact us. We’d love to help you.
Are you a BI architect looking for work?
Passionned Group is always on the lookout for experienced (interim) BI architects for its clients. Do you think you have what it takes, and are you ready for a change? We’d love to invite you for a conversation. If Passionned Group appeals to you and if you want to get to know us better, leave us your contact information here.
Still have questions?
If you have any questions about hiring a BI architect, or if you want to increase your knowledge but you’re not sure how, feel free to contact us.
About Passionned Group
Passionned Group is a leading firm in designing and implementing Business Analytics. Our senior Data Scientists and consultants assist both large and small companies towards a smarter enterprise. Every two years we organize the election of the smartest company.
Our BI Architects
ir. RICK VAN DER LINDEN
Business Intelligence Architect & Industry analyst