The latest BI trends, big data trends and AI trends: algorithms, ransomware and deep fakes dominate
After another challenging year of faltering supply chains and skyrocketing commodity prices, material shortages and exploding energy prices, the specter of inflation “suddenly” haunts us. But for those who look on the bright side, like the techno-optimists, this artificially perpetuated crisis offers plenty of opportunity. If wages get too high, algorithms and robots will gradually be able to take over our jobs. But that will rapidly accelerate the process of dehumanization and desocialization. And then a picture of a dystopia looms: a nasty society in which technocrats are in charge and pessimists rule the roost. Passionned Group remains positive as always and identifies the most important BI trends, big data trends & AI trends of 2022. IT trends that should make life more enjoyable and work better.
In this article 🕑 11 minutes
- Trend 1: Remote First strategy remains the norm for now
- Trend 2: AI grows into a system technology and leaves the lab
- Trend 3: Decision Intelligence gives AI and BI more relief
- Trend 4: The traditional data warehouse is partly losing its shine
- Trend 5: The digital vulnerability of organizations is on the rise
- Trend 6: Responsible AI is taking off
- Trend 7: Data-driven working emerges as a survival strategy
Trend 1: Remote First strategy remains the norm for now
Remote-First is a business strategy that makes working remotely the norm for most or all employees. By definition, “Remote-First” means that few (if any) people still work from a central office. Employees instead work from another location, such as a home office, a hotel, or a flex office. And it no longer matters if you work from New York, Amsterdam or Istanbul.
The distributed enterprise is a fact
Remote-First is certainly not a new phenomenon. For decades, various interest groups in the Netherlands and abroad, such as trade unions and political parties, have been trying to stimulate so-called home working. Each time it turned out to be old wine in new bottles. Teleworking, telecommuting, working from home, the new way of working, mobile working, nomadic working, hybrid working and even a contradiction in terms such as “free working” alternately dominated the social discussions. Under the guise of fighting traffic jams and protecting the environment, attempts were made to create sufficient support for remote working. With varying degrees of success. In the end, it was Covid 19 that really gave the Remote-First strategy a boost. “Work at home unless you really have no choice,” even became the urgent and not quite imperative advice. Gartner speaks of the “distributed enterprise.”
Dehumanization, loneliness and desocialization
From different perspectives (technological, sociological, and social), one can make reservations about the Remote-First strategy. Young ambitious workers sometimes resist a Remote-First strategy because they are less visible. All that working from home can also lead to dehumanization, isolation and desocialization. Vigilance is required and “looking out for each other” is the motto.
Hooks and eyes from the IT perspective
A Remote First strategy also has its pitfalls from an IT perspective. For example, organizations are (rightly) wary of just giving employees access to often privacy-sensitive data from any location. A data breach can never be ruled out. A CFO, for example, would not want his unpublished price-sensitive quarterly or annual figures to be viewed in draft form from a bar or restaurant.
The Dutch National Cyber Security Centre (NCSC) offers some tips for implementing a secure Remote-First strategy. Provide the necessary (network) capacity to serve the increased number of home workers. Consider both the IT infrastructure and the telecom infrastructure. Install the latest updates for hardware and software. Consider a possible increase in reports regarding phishing emails and fake emails. Ensure that the organization’s guidelines are up to date and known to employees. Think about information security including the (home) use of hardware and software and the possible use of private IT facilities.
Health and safety tips for employers
If Remote-First remains the norm, employers will not be able to avoid providing a home working allowance. New remote working legislation enforced by national Health and Safety Authorities also imposes certain requirements on the furnishing of a workplace at home. For example, employers have a duty of care when it comes to setting up ergonomically sound workplaces.
Trend 2: AI grows into a system technology and leaves the lab
Artificial Intelligence (AI) is not just another technology, according to the Dutch Scientific Council for Government Policy (WRR). It can best be compared to the rise of the steam engine, electricity, the combustion engine, and the computer. These are system technologies that have an unpredictable impact on both the economy and society.
AI is still too abstract for many people
AI is the combustion engine of the 21st century, according to the Dutch WRR in its 500+ page (!) report on AI. Demystification, contextualization, engagement, regulation, and positioning of AI are the abstract magic words that the advisory body pulls out of its hat with which the government and society should get a better grip on the unruly AI matter. However, with a plea for so-called algorithm registers, a ministerial subcommittee and an AI coordination center, new forms of bureaucracy and political errors quickly lurk.
The consultants and teachers of Passionned Group keep far away from these theoretical discussions and woolly reflections. Of course, they follow them closely, but they themselves have both feet firmly on the ground when it comes to AI. With a Master’s degree, a training AI in 1 day and an AI book with practical tips, they help organizations daily with the conception, design and implementation of data-driven, smart technologies such as AI. But also with the necessary change processes and improvement initiatives that make the difference. Lifelong learning is therefore the motto for managers and employees who want to work with AI as a system technology. Have a look at all the courses of the Passionned Academy.
Data labs, AI labs and innovation centers flourish
Moreover, in the context of the Dutch BI & Data Science Award, Passionned Group has intensive contact with organizations that are working hands-on with AI. It is therefore no coincidence that this year’s three finalists, Pon Holdings, the National Police and SpendLab, are true frontrunners when it comes to the practical application of algorithms in their daily operations, according to the jury report. In their data labs, AI labs and innovation centers, they are tinkering with the systems technology and business models of the future. They are living proof of the megatrend that the WRR also identifies “AI leaves the lab and enters society”.
In search of new milestones
This trend however, you describe it, is of course a simplification of reality. Laboratory, the space of research, and society cannot be radically separated. Laboratories are also part of society, and ideas, people and practices are constantly moving back and forth between the two, as the WRR also knows. Yet the movement from lab to society is indeed a useful way to make sense of the current momentum in the field of AI. The most famous AI Lab in the world is undoubtedly DeepMind, Google’s research lab that developed the AI program AlphaGo that in 2016 defeated a human champion in the game Go. A milestone. No doubt there will be a number of more milestones, and not just from Google’s labs. Exciting things are also happening with algorithms, as proven by Pon’s data lab, the Dutch National Police and SpendLab.
Give the humans sufficient attention as well
One comment about the boom in so-called AI labs (Google’s search engine quickly returns over a million hits with the keyword “AI Lab”) is in order. Even if an innovative technology or algorithm works completely well in the lab, that doesn’t mean it will work in production. It takes time to adapt processes, to develop (new) business models, to familiarize people with the system and, irreverently put, to train the data-illiterate among us properly. So, in addition to investing in AI, you also need to set aside budget for investing in your people. In short: practice and (system) technology such as AI have to adapt to each other and that takes time.
Trend 3: Decision Intelligence gives AI and BI more relief
It is common knowledge that the IT world is very adept at coining new terms and abbreviations. Just as organizations are becoming more familiar with Business Intelligence and Artificial Intelligence as key technologies to help organizations function better, yet another new term is popping up: Decision Intelligence. The new term is somewhat confusing. Is Decision Intelligence perhaps a new term for BI or a new branch on the AI tree?
Beware of over-optimism: manage expectations
Although Gartner presents Decision Intelligence as one of the “Top 12 Strategic Technology Trends for 2022,” the term doesn’t come completely out of the blue. Dr. Lorien Pratt wrote a book titled “Link: How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World”. An overoptimistic book published just before the Covid19 crisis. Decision Intelligence should be able to eradicate hunger, poverty, disease, unemployment, conflicts, and inequality. For the time being, that seems to be a utopia.
Automate everything that’s loose and stuck
The Gartner analysts are less pretentious. According to them, Decision Intelligence is primarily a practical approach to improving decision-making within organizations. You model each decision as a series of processes where intelligence and data analytics are used to inform users, allow them to learn from their decisions and then refine those decisions. Decision intelligence can support, improve and potentially automate human decision making through the use of augmented analytics, simulations and machine learning models. In his “Data Science for Decision Makers and Data Professionals” author Daan van Beek has been preaching the gospel of decision management for years. After all, the main goal of Business Intelligence has always been to make faster, more frequent, and better-informed decisions. Decision Intelligence is still the common thread in the seventh edition of his book and in the BI and AI projects Passionned Group carries out.
Decision making process less and less a black box
Nobel Prize winner Daniel Kahneman has aptly demonstrated that individual and collective decision-making in organizations is less and less a black box. In his most recent book “Noise: A Flaw in Human Judgment” he explains in great detail why we as individuals or groups so often make wrong decisions and how we can prevent this.
In short: Decision Intelligence will undoubtedly emerge more often as a (fashionable) term in and after 2022, but what’s in the name. Remember that ultimately it’s about the practical content: how are you going to connect data with actions and outcomes? Our consultants help with the implementation.
Trend 4: The traditional data warehouse is partly losing its shine
According to analysts, the market for Business Intelligence & Analytics platforms is still on the rise, but the double-digit growth rates of the second decade of this century have disappeared. Nevertheless, we are still talking about a billion-dollar market worldwide, depending of course on how you define the various market segments and whether or not you add them together.
Immerse yourself in the “old” BI world and the “new” AI world
Within these market segments, a clear trend can be seen running from the “old” BI world to the “new” AI world. Daan van Beek, boss of Passionned Group, gives his vision. He also discusses a misconception that has traditionally plagued organizations in business operations: the stubborn pursuit of one version of the truth. ‘Take it seriously, but don’t fixate on it,’ he says. ‘You’re much better off putting your efforts into a proper supplier comparison. For an up-to-date overview, consult our online guides, such as the Business Intelligence & Analytics Guide™ 2022 and the ETL & Data Integration Guide™ 2022.
Speaking of vendors, newcomers to Gartner’s overview of BI platforms are Amazon Web Services, Google, Infor, and Alibaba Cloud (the latter had been added earlier). At the beginning of this year, TIBCO acquired the well-known Information Builders, and Google bought in its BI & Analytics knowledge via the acquisition of Looker again two years ago. Meanwhile, each vendor is looking for its own niche.
BI vendors struggle with their positioning
After the high-profile acquisitions in recent years, the remaining BI & Analytics vendors are now struggling with their positioning. What label do you put on your business now? Differentiation and positioning do matter, but it should all come across as a bit modern and sexy. For example, the company ThoughtSpot, which recently raised $100 million in growth capital from investors, is advertising itself as the “Modern Analytics Cloud Company.” Remarkably, this vendor predicts that the data warehouse will be permanently displaced from the throne by 2022. Data mesh, data fabric and data lakehouse are said to be the future. We take a brief look at this seemingly bold prediction and draw a preliminary conclusion.
Tribal warfare is of all times
So ThoughtSpot predicts a holy war. A holy war? Really? Discussions and tribal battles have always raged within the BI world. Think back to Bill Inmon’s first book in 1992, “Building the Data Warehouse,” Ralph Kimball’s book “The Data Warehouse Toolkit” and Linstedt’s “Building a Scalable Data Warehouse with Data Vault 2.0.” Since then, new ideologies and technologies compete for attention.
Some important concepts
ThoughtSpot’s Chief Data Strategy Officer Cindi Howson outlines the characteristics of a few concepts, including the historical context:
- The data warehouse is a tried-and-tested concept. For a long time, the data warehouse proved to be a solid, but relatively expensive concept. In the on-premises world, it took months to build a centralized data warehouse.
- The term data lake was coined in 2010 by James Dixon, CTO of Pentaho. Faster access to granular data and lower costs were the selling points. Soon, however, data lakes turned into far too slow, swampy bogs.
- The data lakehouse combines the best of both a data warehouse and a data lake, providing converged workloads for data science and analytics use cases. Databricks uses this term in its marketing statements.
- Data Fabric originated in 2004 within NetApp. Since then, it has evolved significantly conceptually. Data fabric places a strong emphasis on metadata and AI to discover related data in cloud and on-premises datasets.
- Data mesh is both an architectural approach and organizational concept developed by ThoughtWorks and defined in Zhamak Dehghani’s new book “Data Mesh: Delivering Data-Driven Value at Scale.” The end goal of a data mesh is a data product. Data is organized and controlled by domains and should not be moved.
A data mesh, a data fabric, a data lakehouse or just a plain “old fashioned” data warehouse, after all it seems to matter little to users according to a recent LinkedIn poll.
Are you confused by all these definitions? The consultants at Passionned Group would be happy to think with you. If you want to delve deeper into the subject first, buy our Data Science Book.
Whether the data warehouse will be dethroned in 2022 remains to be seen. There is no denying that the data warehouse has lost some of its lustre, but the same is already true of the new stars in the firmament. In the meantime you just need to make sure your datawarehouse doesn’t become a “data where house” and of course the very experienced BI consultants of Passionned Group will be happy to help you on your way.
Trend 5: The digital vulnerability of organizations is on the rise
Digital transformation in businesses and government agencies has steadily increased in recent years. Many business leaders expect more business processes to become automated. A recent Fujitsu survey revealed that 44% of respondents from offline organizations predicted that more than half of their current business processes that were not yet automated would be automated by 2025.
The digital transformation and its excesses: Apache Log4j
However, this progressive digitization, also referred to rhetorically as the digital transformation, also has an inky dark side. Cybercrime such as ransomware, phishing, identity fraud, DDoS attacks, hacking, data leaks, espionage and last but not least privacy violations have been rampant for years. Every day, many businesses and citizens receive fake emails, fake requests, and links to fake websites. At the end of this year, businesses were still confronted with the Apache Log4j vulnerability and in the new year, the so-called Khonsari family and its accomplices, probably will leave a trail of destruction and be a source of great frustration. There, a legal obligation to report ransomware attacks, however well intentioned, doesn’t do anything. The same goes for a ban on ransomware. However well-intentioned, this will unfortunately not provide any solace.
More and more organizations are required to take appropriate measures
Where do the challenges lie in the digital transition in 2022? As in previous years, security seems to be the biggest concern for many organizations. How do they keep their systems and their data safe? The number of sectors that will be legally required to take appropriate measures to secure their network and information systems and to report serious cyber incidents will only continue to grow. These include, for example, larger parties operating in food production and distribution, manufacturing, and postal and courier services. An agreement on this has already been reached in Brussels at the Telecom Council in 2021.
Currently, providers of essential services (such as banks and the drinking water and energy sectors) are designated by the central government under the current directive. Digital service providers, such as cloud services and online marketplaces, are also already covered by the directive. They must take measures for their digital security and have an obligation to report serious cyber incidents, including data breaches. This is also subject to supervision.
Trend 6: Responsible AI is taking off
As always, we identify trends in BI, big data, and AI early. At the end of 2019, for example, we already signaled the trend of Generative Adversarial Networks (GANs). GANs enable two neural networks, among other things, to generate new data based on an existing training set. This “data synthesis” makes available to machine learning models large amounts of big data with which to feed and train the models. Simply put, these are two deep learning networks that will compete and try to fool each other.
There is a clear element to it: gaming. GANs are widely used in generating images, videos, and voices. But generative AI also opened the door wide to some high-profile deepfakes.
Machine learning, but fair
In the coming years, practical AI-based applications will continue to be developed using the latest techniques, but at the same time the call for responsible AI, also referred to as Fair Machine Learning, will become louder and louder. This is partly due to some debacles at government agencies with the misapplication of algorithms. But the use of facial recognition software is also controversial and has previously led to parliamentary questions. The prevention of various cognitive biases, including selective bias, will become increasingly central to algorithm development and training.
Expert AI specialists are scarce
Responsible AI and supervision of the use of algorithms will become necessary preconditions in companies, hospitals and (government) institutions in the coming years. Without a decent foundation and accountability, the AI party will not happen. In fact, Gartner expects that by 2023, all personnel hired for AI development and training work will have to demonstrate expertise in the application of responsible AI. How exactly they would have to do that, unfortunately, remains unsaid, and by 2023 no one is going to be checking to see what has come of that prediction. But for the ideation and social discussion of what course we should take with AI, this prediction certainly doesn’t hurt. Passionned Group provides those expert AI specialists on an interim basis.
Good intentions and what they are worth
Algorithms will be monitored for transparency, discrimination, and randomness. An algorithm regulator is to monitor this. Governments will set a good example by not collecting and sharing more data than necessary and will develop rules for data ethics in the public sector. People will have their own online identity and control over their own data. These are some good intentions for 2022.
“In addition to deployable personnel, the police must have adequate information facilities,” we can read in the 2022 budget for the Dutch Ministry of Justice and Security. This is necessary to be able to work in a more data-driven way and partly because of that to detect and combat crime in the digital domain more robustly,” the budget documents state. And with “data-driven work” we have arrived at the last trend that we only touch on briefly because it is so natural for Passionned Group and is woven into our DNA.
Trend 7: Data-driven working emerges as a survival strategy
Passionned Group was and is a trendsetter in the field of data driven working. In all kinds of sectors and at customers of Passionned Group the ideas have caught on. Data-driven working is not only a promising way of working, but also a survival strategy in these uncertain times. It requires vision, out-of-the-box thinking, perseverance and – last but not least – a feeling for data. But once you get the hang of it, you will automatically experience a feeling of euphoria, because working with data is addictive.
Data is leading the decision-making process
The associate partners and consultants from Passionned Group are eager to show how you can use data-driven improvement circles, such as PDCA, to bring your company’s services to an ever higher level. Whether it is a municipality, a housing corporation, an IC unit in a hospital, a financial service provider, a hotel, a laundry or an online store, there are improvement opportunities everywhere if you take data seriously and use it to guide decisions.
The benefits of data-driven working
Daan van Beek, boss of Passionned Group, describes the advantages of working in a data-driven way and outlines a simple step-by-step plan. He also convincingly demonstrates that data-driven working is the most fair, democratic way of working. Data-driven working brings passion and fun back to the workplace. And that is desperately needed in 2022. For more information: contact us now.
The management, consultants, teachers, and employees of Passionned Group fully realize that they are still allowed to do their work from a relative position of luxury, although we are of course also affected by the pandemic. At this point we would like to thank everyone who has put their trust in us in the past year.