Are You a Business Intelligence Avoider?

From six different studies we can conclude that approximately only 5% of employees use BI tools to perform Analytics effectively.

I think this is in part related to the fact that except for folks with backgrounds in science, engineering, economics or finance; many people across the company do not feel too comfortable around numbers, math or logic functions.  I've observed this during years of training corporate employees on the strategic use of Business Intelligence and Analytics.

This is how the numbers work: close to 50% of employees have access to BI tools, about 20% of them actually use BI, and about half of them (5%) are in the analytical / power user categories.

Despite recent inroads of the more user friendly BI visualization tools of the last few years, the BI Scorecard’s annual survey of users, administrators, and directors reports flat utilization rate since 2006.

If you were to plot a histogram showing only those 20% actual BI users in an organization, it would probably approach a normal distribution (bell shape curve) consisting of the following categories, where close to two thirds of the users would fall in buckets 3 and 4.

1) Non Users: Run canned reports once a quarter or less frequently. These are people who are either math averse; do not like computers, or are executives that have their assistants run and print static reports for them.

2) Infrequent Report Users: Run canned reports about once a month.

3) Frequent Report Users: Run canned reports on a weekly or daily basis.

4) Analytic Users: Modify static reports and OLAP cube views by grouping, sorting, formatting as well as changing some dimensions and measures. These folks save their new customized reports for future use.

5) Power BI Users: Create new ad-hoc reports from scratch applying multiple dimensions, measures and using grouping, sorting and filters. These users travel interactive dashboards and OLAP cubes from corner to corner using drill down, drill through, and most of the available custom features in search of the root causes of both: problems and opportunities.  Power users create and share reports, dashboards and visualizations with folks in the same department.

6) Expert Analysts: Search find and provide new data bases, blend disparate data, design and build customized cubes, pivot tables and dashboards, perform statistical, financial or marketing analyses and usually export results to MS Excel to complete the last analytical mile.  Expert analysts create and share reports, dashboards, visualizations and analyses with management across the organization.

Buckets 4, 5 and 6 represent the 5% of employees that use BI tools to perform analytics effectively.   When it comes to Advanced Analytics (statistics, predictive modeling, data mining, etc.), data scientists or statisticians are probably close to 10% of that number or just about 0.5%.

To become a true analytic competitor, the company has to change the culture so everybody, not just the experts, thinks and acts based on facts and understands the drivers that support strategy and sustainable profitability. 

While this isn't easy, it’s possible.  I've seen it quite a few times.  It requires long term commitment from top management, adequate training, data structured to be business-intuitive and an interactive visual analytic software tool configured in an extremely user-friendly manner so all types of users can perform analytics with virtually no help from IT, analysts or even spreadsheets. 

In my experience, the organization improves its financial results through this implementation as people gradually advance to the next level, leaving buckets 1, 2 and 3 practically empty. 

Which buckets is your organization using most?

When Data Destroys Value

A survey by Gartner Research found that poor data quality costs companies an average $8 million per year. In a different study published by The Data Warehousing Institute more than 50% of companies experience customer dissatisfaction and cost problems due to poor data.

According to Gartner, about 80% of Business Intelligence implementations fail; while an Accenture survey of managers in Fortune 500 Companies found that 59% cannot find valuable information they need to do their jobs; 42% accidentally use the wrong information about once a week and 53% believe the information they receive is not valuable to them.

It’s pretty obvious that the main reason for the failure to meet Business Intelligence expectations is probably data quality; reinforcing the proverbial GIGO principle.

This is only getting worse considering that Aberdeen Group estimates the amount of corporate data companies need to digest is growing at 56% annual rate, coming from an average 15 unique data sources.

It’s clear that today’s Business Intelligence & Analytics software capability is light-years ahead of the semantic quality and structure of the data. After years and millions of dollars spent in BI deployment, folks in areas like Marketing, Sales or Strategic Management feel like they are drowning in an ocean of data and yet thirsty for the Strategic Knowledge they need to grow the business.

It’s common to see state of the art Business Intelligence applications that cannot deliver strategic analysis or direction until multiple analysts download the data into spreadsheets, manipulate, clean, fix and structure the data manually, for hours or days at a time.

Many folks recognize the transaction data has plenty of errors, at least at the customer, product line, brand, market and segment level, but they never get fixed and, even worse, they populate the BI systems.

This is a classic case of cultural miscommunication. Marketing and Sales people think this is an IT problem and expect the data warehouse analysts to fix it, while IT thinks it’s a business problem and expects Sales or Marketing to take action. The result is that data seldom gets corrected, people give-up, and running raw or bad data through sophisticated BI systems and dashboards becomes the norm.

After a while, Sales, Marketing and Management won’t trust the data, will not see the value of BI, will quit using the system and continue making decisions based on intuition.

When Analysts or Power users need a quick answer about market share, profitability or growth of a particular segment, product line, customer or competitor it takes days to go through the analyst’s manual process. This consists of running the right queries, exporting them to Microsoft Excel, manually cleansing data errors, adding look-ups from external data sources, and finally creating pivot tables to find the right answers.

Even worse, they have to go through the entire process over and over again every time they need a progress update, either the following week or at month-end, quarter-end or year-end for each one of the business units and markets they serve.  This not only is an inefficient process but fixing data based on analysts’ personal assumptions leads to undesirable multiple silos and different versions of the truth.

As a result, it’s common for people to spend a large portion of team meetings arguing whose data is correct instead of focusing on the critical issues. The numbers generated by Finance do not agree with the analysis performed by Marketing or the explanations provided by Sales. They need a single version of the truth, but different folks run different queries, made different data cleansing assumptions and customized their spreadsheets based on different metrics.

A Simple Solution that Works:

While IT may own the physical aspects of the data (storage, security, format, connectivity, etc.), the Business (e.g. Marketing & Sales) must own its strategic meaning and be committed to auditing and maintaining its high quality.

Proficient analysts who spend most of their time cleansing the data constantly after downloading it to spreadsheets every time they need a report, need to change and become Data Stewards that cleanse and structure the transaction data using external market intelligence BEFORE it enters the Data Warehouse or BI systems. They need to maintain, update, format and upload their data corrections on a weekly or monthly basis.

This small process change requires a committed partnership between IT and the Business Units but truly makes a difference because it breaks the Business Intelligence vs. Data Quality vicious cycle.

When the data makes sense to BI users they start to recognize their business models in their dashboards, develop trust in the data and become better users. They’ll soon find additional data issues and take the initiative to get them promptly corrected. In a few weeks there will be no more serious data quality issues as this process becomes a virtuous cycle where the BI utilization rate grows giving the company an analytical competitive edge.