This is our blog about designing and implementing Intelligent organizations. Within this area of expertise we write often about the following subjects BI, analytics, the tools, decision management, data visualization, BI success, Business Intelligence concepts, data management, continuous improvement, and the organization of BI & Analytics.
Of course you can order our book here too.
Storing high data volumes is easy and cost-effective with the latest big data technologies, but the end-game of analytics is often only an afterthought. Business analytics and data integration platform Pentaho has released version 5.1 that offers enhanced capabilities for scaling up big data operations.
Specialised IT skills to prepare data and perform analytics are not required as Pentaho 5.1 helps to the gap from data to analytics. It offers code-free analytics directly on MongoDB, which accelerates and simplifies the data preparation process and it offers full support for Hadoop project YARN.
A decade ago the vast majority of data warehouse systems were hand crafted, but the market for ETL software has steadily grown and the majority of practitioners now use ETL tools in place of hand-coded systems. Does it make sense to hand-code (SQL) a data warehouse today, or is an ETL tool a better choice?
What are the 7 biggest benefits of using an ETL tool?
We now generally recommend using an ETL tool, but a custom-built approach can still make sense, especially when it is model-driven. This publication summarizes the seven biggest benefits of ETL tools and offers guidance on making the right choice for your situation.
WebFOCUS InfoDiscovery has been launched by Information Builders. This is a web-based, self-service visual analytics tool meant to help analysts, power users and business users to visually analyze complex data, create insights and enhance decision-making across the organization.
With InfoDiscovery analysts can retrieve and decipher data from spreadsheets, departmental databases, enterprise data warehouses and third-party data sources in a self-service environment. The solution also allows power users to combine charts, graphs, maps and other data sources to create sophisticated visualizations and publish dashboards made accessible with InfoApps. Information Builders says users now have the ability to visually analyze more data than ever before with the assurance that the data is secure, auditable, shared and governed.
Birst, the BI-platform in the cloud, has announced a partnership with CloudBase Services. Together they will deliver Birst’s enterprise-ready cloud business intelligence (BI) solution for business managers. Birst’s end-to-end platform combined with the consulting services provided by CloudBase Services should enable business users to make data-based decisions.
Debbie Taylor, President and CEO, CloudBase Services, says: “CloudBase Services’ clients are often challenged to fully make sense of multi-channel data sources. By partnering with Birst, CloudBase Services’ is empowering customers to make smarter business decisions. Birst lets business managers find the root cause of complex problems and operate more profitably.”
Users of the popular open source collection of machine learning algorithms R and Weka can broaden their view of customer behavior with the new Pentaho Data Science Pack. This toolkit will speed up the data flow process with analytics and allows data scientists to spend more time on advanced and predictive analytics. The Data Science Pack will be available from Pentaho this summer.
Ken Krooner, President at ESRG says: “There was a gap in the market until now and people like myself were piecing together solutions to help with the data preparation, cleansing and orchestration of analytic data sets. The Pentaho Data Science Pack fills that gap to operationalize the data integration process for advanced and predictive analytics. Using Weka with PDI, we use all equipment data sources to enable early prediction of potential machinery failure.”
Oracle has optimized its in-memory capabilities across applications, middleware, databases and systems. This increases the transactional performance and it enables organizations to discover business insights in real-time, where this could take hours before.
Hundreds of end-users have tested the in-memory for over 9 months. Yahoo was one of the companies that beta-tested and states that it can even be used on their largest data warehouses. Oracle claims that deploying the In-Memory Database with any existing Oracle Database-compatible application is easy. No alterations to these applications are required. It is fully integrated with Oracle Database’s scale-up, scale-out, storage tiering, availability and security technologies.
The Passionned Group has extended its Business Intelligence Tools Survey with a new element. The just-released 2014 edition now contains a Sentiment Analysis, in addition to the usual extensive analysis of functionalities of the tools and features of the Business Intelligence suppliers.
The reason for this additional research is that many BI-suppliers try to let you believe that Analytics would be the new BI. In many publications and presentations they promote to refer the term BI to the archive and to then only speak about Analytics.
Provider of embedded business intelligence and reporting software Jaspersoft Corporation has been bought by TIBCO Software.
TIBCO is a global supplier of infrastructure and business intelligence software. The combination of the two companies’ offerings allow TIBCO to cover the whole of the analytics market while providing value to the combined set of customers and prospects. The continuation of Jaspersoft’s open source mission is not yet clear.
SAS, supplier of business analytics software and services and a vendor in the business intelligence market, has updated its Master Data Management and Federation Server software. The software is designed to help organizations better access data, manage data and use data from any source. The ability to organize and manage the data pouring in from an ever-growing list of sources is critical, according to SAS.
In the third quarter of 2014 a free desktop version of QlikView.Next will be released by Personal Edition download. In addition this desktop client includes a free cloud service for small group sharing of analysis.
Users of these Personal Edition of QlikView business analytics could build customized dashboards by just dragging and dropping. These features, so-called Natural Analytics, won’t be available in the user interface of the server and enterprise functionality of the complete platform until the second half of 2014.
A suite of end-to-end SAP Big Data services has been made available to identify and use data-driven business opportunities. The SAP Big Data services, through to the ongoing management of a Big Data environment, range from short-term to ongoing strategic advice, advanced analytics, data science consulting and implementation support. This covers the entire end-to-end process and life cycle.
Stefan Schoepfel, global vice president Big Data and Analytics Services, SAP says: “Big Data solutions create opportunities for businesses to expand sales of existing products, increase the rate of speed they bring products to market, innovate new business models to better service their customers and help reduce operating costs.”
Predictive analytics is a powerful instrument for many organizations. It helps them to create competitive advantage and make their business processes more effective. Insurance companies and credit card issuers for example use it to detect fraud, cops use it to catch criminals, sometimes even before they commit a crime, and car dealers apply analytics to predict the chance that someone responds to a campaign. From our experience we strongly believe that there is a lot of added value in predictive analytics. But, prediction is very difficult – especially if it’s about the future (Niels Bohr) – and making automated predictions is even more difficult. There are many pitfalls on the road to success. We think this are the five most important pitfalls:
In a time of financial crisis and budgetary restraints, is it effective to invest money in Business Intelligence projects that historically have had dubious results? Yes, absolutely, but only if you get it right.
At the end of the day surviving in an economic crisis and business intelligence are about exactly the same thing – doing what you do better than the guy next door. It does not matter if you work for a 10-person company or the biggest multinational in the world, organizations that get it right stay in business and those that do not, fail.
Is the name Business Intelligence so out of date and tarnished that we are better off calling it something else – perhaps “Performance Management” or “Business Analytics” or “Data Discovery”?
This is something that we have discussed at length with both the vendors and the customers, interestingly enough the two groups have completely different views. The customers, and here we mean the people in the business who actually fund this type of initiative by buying the software, have a much better understanding of Business Intelligence than Performance Management, analytics or any of the other 50 or so terms that are being used.
Each day 2.5 quintillion bytes of data are generated by a broad variety of sources. IBM believes data is the newest resource for competitive advantages and analytics are its key.
On March 27 the company announced having plans to commit over 100 million dollars to build upon its portfolio on this resource. IBM expanded it’s big data capabilities for customer experience and plans to add another 1.000 employees worldwide to its payroll. Ten new Experience Labs around the world will be opened, amongst others in Groningen, New York and Bangalore, to present clients with the opportunity to work side-by-side with IBM’s experts.
Massive volumes of structured and unstructured data from business applications, machines, social media and more accumulate in organizations each day. The challenge is to exploit the value of this information within the context of end-to-end processes such as procure to pay, order to cash, idea to offering and plan to inventory.
SAP HANA’s intelligent business operations bundle addresses the struggle that many organizations have to gain one single view of their processes and make use of Big Data in a way that empowers line-of-business users to improve their operations.
Informatica’s Cloud Spring 2014 is now available. Ash Kulkarni, senior vice president and general manager of Data Integration at Informatica, calls it a game-changer.
Cloud-based enterprise-caliber business intelligence platform Birst has teamed up with solution provider Audaxium to accommodate their NetSuite users. The integration of Birst with NetSuite applications enables organizations to get more out of their ERP data with fast access to information and analysis through automatic creation and delivery of reports.
As customers were increasingly demanding the ability to analyze their ERP data across different business applications and data sources to make better business decisions based on instantly created detailed charts and reports, the partnership of Audaxium and Birst seemed logical.
BOARD International was founded in 1994 in Lugano, Switzerland, and delivers tool kits for Corporate Performance Management and Business Intelligence. Today there are 8 offices worldwide.
BOARD has released version 8.1 in January 2014. This version also allows for mobile use, on iOS as well as Windows 8. The main focus for this release was supporting large global deployments of BOARD and thus a solid back-end.
A new version of the Australian BI platform Yellowfin has been released in December. Yellowfin 7 according to its vendor strikes a fine balance between administrative requirements of the business and data exploration.
Several modules have been completely redesigned and new features have been added to the BI platform. This release focuses on enriched administration, improved content creation, multi-chart formatting, collaborative decision making and mobile report authoring.
Dynamics CRM Online customers can easily load their data into excel, with the new Power Query Microsoft provided last December.
It already enabled users of Office and Office 365 to query and analyze data, but this update lets the Power BI suite of tools reach the fastest growing portion of the Dynamics CRM customer base. After pouring the ‘power of the cloud’ into excel using the OData feed, visualisations can be made using Power View. This offers lots of new possibilities for users to connect with data.
Jaspersoft is an open source BI-platform, initially created by a Java-developer whom was asked to enhance an application by embedding reporting capabilities. He considered most of the well-known products in the marketplace too expensive or causing a lot of problems when integrating with Java, so he decided to create a reporting tool by himself.
Named after the crystal Jasper – and with Ja referring to Java – the product is now a commercial product, covering all BI aspects end-to-end. Still Jaspersoft is open source. Jaspersoft is maintained, enhanced and innovated by a community with more than 225 members. It has been downloaded more than 14 million times and used in 100 countries by over 14 thousand organizations.
Contrary to many studies that evaluate ETL tools based only on their current market share and functionality, our study (the ETL Tools & Data Integration Survey) also attempts to evaluate ETL tools based on their expected future performance. With the market changing as quickly as it is today, market share is about the past, and says little about the future. Many products, including WordPerfect, Lotus 123, and Harvard Graphics once had a huge market share, but proved to have very little future potential. The growth potential as defined within the confines of this ETL study looks at how frequently a vendor releases new, valuable features; an important indicator for innovation and the strategic importance of the product to the vendor.
The maximum number of points awarded in our ETL tools & Data Integration Survey to an ETL tool for its ease-of-use was eleven. A maximum of two points was awarded for each of the four attributes, indicated by one or more plus (+) signs, and points were allocated depending on the number of days of training suggested by the vendors (less training required – more points for Ease of Use).If the verdict was negative indicated by one or more minus (-) signs, points were deducted. If, for example, ease of use, WYSIWYG, and logical order earned a plus sign and screen design a minus sign, the tool was awarded two points (one point for each plus sign, minus one point for the minus sign). Since it can take considerable time to improve the usability of a product we have compared the product (where possible) over a three year period. The maximum number of points was 11.
The ETL tools were evaluated in our ETL Tools & Data Integration Survey based on the following characteristics:
Clarity and re-usability
Real-time ETL/EAI/Web services
Architecture and infrastructure
Growth potential and Market Stability
The ease-of-use criterion was the only aspect that involved a measure of subjectivity. We looked at the products from the point of view that they should be usable by a broad range of professionals and not just a few specialized IT professionals. This is clearly not everybody’s view of the world; some of the ETL suppliers we talked to were appalled, suggesting that allowing business users the chance to build data warehouses would guarantee failure. Others agreed that their major market was the business user and not the IT professional. We have made every effort to judge the products objectively in terms of ease-of-use, but admit to a bias in the direction of broad use.
Since 2003, we have been closely monitoring the market for ETL and data integration tools. In the past, the focus was on the market leaders who were often seen as visionaries and leaders.Many organizations used to assume that they had automatically made the right choice if they purchased a tool from one of the market leaders. Since the late nineties, however, the market has changed substantially. Practically all the Business Intelligence (BI) vendors have purchased or developed their own ETL tools. Since a centralized data warehouse is one of the cornerstones of a successful BI solution, this has turned out to be a wise choice. Market estimates show that 70-80% of the costs of a successful BI system relate to the creation of reliable ETL processes and data integration.
Aurora Fashions, famous for brands such as Oasis, Coast and Warehouse have chosen BOARD as their strategic BI solution.
Main reasons are, that BOARD is able to provide real time enhanced visibility across the business and delivers self-service analytics, reports, dashboards and performance management to the business. Aurora Fashions was searching for an enterprise-wide, agile and integrated solution that is easy to use.
Australia-based Yellowfin launched version 7 of their BI-platform. Most of the improvements in Yellowfin 7 are the creation of content, multi-chart formatting and enhanced administration. Furthermore the platform is extended with new collaborative decision making and functionality for authoring mobile reports.
The content creation module of Yellowfin BI tool enables users to conduct real-time ad-hoc analysis on business data, generating immediate business insight. Yellowfin 7’s new interface for report creation and ad-hoc analysis provides a canvas for content creation, where the seleted data is visualized immediately.
Bizzscore helps organizations to get a grip on performance management by improving the use of information and reporting results.
In any organization, decisions made based on trusted management information are very important. Performance management empowers executives to helm the organization in the right direction. Bizzscore is a complete solution for the support of performance management, leveraging the balanced scorecard model and INK, EFQM or any other management-or quality model.
Over more than a decade SAS Institute, a US-based vendor of BI solutions, is a top player in this field. SAS’ roots are at North-Carolina State University and since 1976 the company is managed by its inspiring chairman Dr James (Jim) Goodnight. Today SAS is still a privately held company with an annual turnover of more than 3 billion dollars.
A couple of years ago SAS started profiling itself as a vendor of Analytics solutions. Ever since its foundation the company had a strong preference for Data Mining and today it claims the Analytics market as its own. Although the main part of SAS’s turnover emanates from license sales of the SAS BI-platform – which includes ETL, data warehousing, data quality, reporting and analysis – the full focus is on Analytics.
BI-vendor Information Builders (IBI) extended their portfolio by providing a BI appstore for business users called InfoApps. With these apps, all stakeholders at any level of an organization can discover new information needed or discover potential new business possibilities.
The apps, built with IBI’s WebFOCUS platform, are easy to deploy without assistance from IT and easy to use on any desktop, laptop and mobile device. Without any training, users can design their own dashboards combining the data from different sources they pick themselves.
SAP transited the SAP Visual Intelligence software into a new data visualization desktop product, SAP Lumira. The product empowers business users to visualize large or small data volumes, so it promotes self-service and speeds up decision making.
A second version of the visual BI-software, SAP Lumira Cloud, is an HTML5-built product, so it reduces the time needed to develop and deliver new types of analysis, without any help from IT. It is easy to connect with data that is available on-premise and with SAP HANA.
With version 8.1 Tableau delivers a number of new features and capabilities, such as extended analytics, improved interaction with the internet, and apps for mobile devices.
The Tableau platform is a 64-bit and IPv6 ready suite with SAML authentication, with a strong focus on data visualization through dashboards.
The analytical functions are extended by integration with R, the well-known open source statistical library. The upgraded data connectors simplify access to data. Source: www.tableausoftware.com
In Q4 of 2013 Inetsoft released version 11.5 of its BI-platform Style Intelligence. Remarkable new feature is a data source connector for Hadoop, which enables users to add data from the Apache Hive data warehouse to their dashboards and analysis in the same easy way they connect any other data source.
Style Intelligence offers a completely restyled user interface and lots of new capabilities and features in the field of dashboard and visualisation design. Inetsoft also enhanced the audit reporting features and added more self-service capabilities.
Pentaho Community Edition 5.0 is the latest version of Pentaho’s open source data integration and business analytics platform.
Business Analytics platform: enables business users to access, discover en combine data of all types and sizes with advanced business analytics.
Data Integration: Pentaho’s tool also known as ‘Kettle’ – delivers ETL capabilities; as stand-alone application it can be used to visually program the process of extraction and transformation.