Business Intelligence (BI) is about turning data into insight and using this insight to make better business decisions.  Today the opportunity to make better decisions is growing as the amount of data being stored increases. Between 2009 & 2020, the “digital universe” is projected to  grow by a factor of 44 – increasing from c. 0.8 Zettabyte (ZB) to greater than 35 ZB.  This, however,  creates a significant challenge – how to distill  useful insight out of a growing sea of increasingly disparate data?

This is a particular challenge for Small and Medium sized Enterprises (SMEs). Despite their size, SMEs are nearly always collecting significant amounts of data – sales, orders, customer relationship management (CRM) and website data. The key is how to extract the insight despite the challenges that these organisations face. These challenges  include

• A lack of specialist  analytical staff with advanced Business Intelligence & analytical skills.
• Restricted budgets – often there is no budget for Business Intelligence training or expensive tools.
• A rapidly evolving business environment that demands the rapid generation of business insight.

This blog is about how to use business intelligence & analysis to make better business decisions despite these challenges.

Business Intelligence can be considered to consisting of three core components : Data Integration, Reporting, & Modeling.

Data Integration

Data Integration is about getting multiple disparate data sources into a central file (Data warehouse) where it can  be analysed or used for reporting. A simple example might involve merging sales and customer contact data so the marketing team can report on sales or contact specific customers with a product offer. When data also needs to be cleansed, aggregated, filtered or have any advanced logic applied this data integration  task becomes difficult using generic tools like Microsoft Excel or Access. To make it worse, Excel & other traditional database tools provide little or no auditing of data quality.

Without  specialist data integration tools,  employees tend to burn an inordinate amount of time  merging data files  before they can generate say a monthly report. To make it worse each month they would normally need to repeat the process. The end result is they may spend  90% of their time integrating data & generating the report and only 10% interpreting the data. Wouldn’t it be smarter to turn this on its head? – spend 10% of the time integrating the data & producing the reports & 90% interpreting results and making recommendations? The problem is many companies aren’t aware of how to integrate data sources more efficiently.

Reporting

Once the data is integrated & in  one location (by definition a data warehouse), reporting becomes infinitely easier. Tools like Crystal reports, & Microsoft Reporting Services are often thought of as the best reporting tools, but in reality a lot of great reporting can be done is Excel, particularly Excel 2010 with its new data visualization & slicing tools (PowerPivots).  The added advantage of Excel is  most organisations already own it and very little training is usually required for users to be able to slice and dice data in reports. It continually surprising me, how many expensive reporting projects start with Reporting services or Crystal Reports and end up falling back to using Excel.

Its often more about what sits under the reporting rather than the actual reporting interface. For example an underlying OLAP cube provides reports  with the functionality to efficiently drill down and roll up data (for example from sales regions, to countries, to cities, to companies). In most reporting projects, the biggest challenge is actually integrating the underlying data.

Example of an Excel 2010 Powerpivot report using “slicers” that allow easy drilling down to specific subsets of the data.

Modeling

Relatively few businesses, and especially  SMEs,  spend much time doing modelling. This party because most analytical time is soaked up with reporting, but also because of lack of knowledge of tools and methodology. Modelling covers a range of techniques such as

• Scenario analysis where expected, best and worst case revenue scenarios are calculated.
• Taking this further approach further is Monte Carlo – where you calculating a probability that an even will occur.
• Building propensity/ predictive models, such as decision trees to predict customers likely to churn.
• Classification models to identify a common source of customer dissatisfaction.

How does Integration, Reporting & Modeling fit together?

In most businesses a disproportionate amount time is spent on data integration. Missing or incomplete records need to be filtered out, product codes in one spreadsheet need to be matched to product names in another & several months results need to appended to each other. The steps go on. To make it worse the process generally needs to be repeated in preparation for the next round of monthly reporting.

The problem is this essential,  but low ROI,  data integration work leaves little time for the higher value tasks. In the ideal world more time would be spend on using insight to help formulate & test hypotheses, running scenario analysis modelling and making recommendation on how to improve the business. The right  tools and training can help with enabling a business to focus on work that gives them the best Return on Investment.

Video demonstration of how the various elements of a business intelligence solution fit together.

Who is this blog aimed at and what is its purpose?

This blog  aims to help analysts get up to speed with Business Intelligence tools and analytics techniques. Its about using smart technology and techniques to improve the allocation of analyst time.