Data Warehousing & BI

Cliff Mills National Computing Centre

© 2005

PROVIDING PERSONAL AND PROFESSIONAL DEVLOMENT FOR IT LEADERS

© NCC

Managing Information • A data warehouse is a repository for an organisations electronically created data and is designed and optimised to facilitate reporting and analysis. • Business intelligence tools are designed to report, analyse and present data. The tools generally read data, though not necessarily, that has been stored in a data warehouse.

© NCC

‘But business intelligence is not just about hardware and software. It is about a company wide recognition that a company’s data are an important strategic asset that can yield valuable management information to improve decision making and implement change.’

Information

Insight

Action

© NCC

Having a strategy is important Consistent Strategy?

• Having a strategy is the cornerstone of a successful business intelligence project

21%

• Top level management need to see the potential for BI? • Why is BI being implemented in the organisation? • Communicate understanding throughout the organisation

11% 68%

• Selecting the right technology • Do we have the right data in the right format • On-going development for the BI system

Yes – across the whole organisation Different strategies in different part of the organisation No consistent strategy

© NCC

How would you describe the current status of data warehousing/business intelligence within your organisation? • Many organisations have a BI/DW solution of one sort or another • The result of tactical purchasing of products by individual departments • Result of implementing products with in-built business intelligence solutions • Result of merger activity – separate companies having their own products

11%

26%

5%

21% 21% 5%

11%

Widely implemented across the organisation

• Mature organisations have moved from departmental through operational BI to enterprise

Implemented in some parts of the organisation Currently in development/implementation Initial feasibility study/pilot study underway Formal analysis and specification in progress Identified as a future area of activity No plans to implement these systems

© NCC

Key business reasons for adopting data warehousing & business intelligence? Improving quality of decision making

4.5

Providing insight into trends

3.8

Improving performance measures

3.8

Improving data accuracy & integrity

3.8

Increased insight into customer behaviour

3.5

Providing access to real-time data

3.2 Scale 1 to 5, 1 is equivalent to ‘not important’ and 5 is ‘very important’

2.9

Improving marketing and sales information 0

1

2

3

4

5

• “Providing the evidence base for corporate policy making & improved service delivery” • “Defining key performance indicators and measures” • “Rationalisation and aggregation of information available from diverse applications that underpin the value of pan company CRM” • “Ability to provide correct business decisions in an accurate and timely fashion” • “Elimination of data duplication. Business needs one single source of accurate information in a timely manner”

Do your senior management see data warehousing/business intelligence as a key enabler in decision making? 11%

• Managers need more insightful information that they can trust • Understanding what is happening in your business

5%

21%

11%

• Key decisions not made on numbers in your own spreadsheet & gut feeling • Managing and competing on ‘information’ and ‘intelligence’ is a key driver of performance for the 21st century business

26% 26% Very important Important Medium importance Little importance Not important Don't know

© NCC

Overall how well do you think the objectives for data warehousing/business intelligence have been achieved? • Would indicate that many organisations still have a long way to go to become ‘intelligence’ driven

0%

16%

32%

• How do you define success? • Many organisations may need to redefine their BI objectives & strategy • Appoint champion to review information needs and delivery • Use external help if necessary

26% 5% 21% Very well

Well

Medium

Poor

Not very well

Don't know

© NCC

How do you see your use of data warehousing/business intelligence developing in the future? Implementing new tools to enhance data analysis

63

Ensuring existing assets are fully exploited

47

Expanding scope of system to new functional areas

47

Expanding scope of system to new users

37

Controlling costs of the system

32

Improving performance of the system

21

Other

5

% 0

10

20

30

40

50

60

70

© NCC

How important are the following issues in running your data warehousing/BI system? System does not have all the data we need

3.9

Can’t ask the questions we really need to

3.6

Process of asking new questions is slow

3.3

Reporting execution times are excessive

3.1

Need real time reporting

3.1

Questions can only be asked by specialists

2.8 2.1

System often taken off line for updates Scale 1 to 5, 1 is equivalent to ‘not important’ and 5 is ‘very important’

0

1

2

3

4

5

© NCC

How would you rate the quality of your data currently and before implementing a data warehouse? Before implementation

Currently

7

Excellent

5 27

Good

5 40

Satisfactory

37 13

Bad

26 7

Very bad

11 7

Don't know

16

% 0

10

20

30

40

50

© NCC

Do you feel you have achieved a 'single view' of your customers?

11%

5%

11%

Yes – for all customers Yes – for some customers

21%

Still working towards this No

52%

Not relevant

© NCC

How are BI tools being used?

14%

In research capacity only

7% 21%

By specialist analysts on an ad hoc basis

By specialist analysts on a regular basis

29% Routinely deployed as part of operational applications by non-specialists

29%

Other

© NCC

- good with information •

Large organisation – 3000 + food stores – using DW/BI for 15 years



300+ users of BI software across Finance, Trading, Marketing, Operations and Supply Chain



Data Warehouse now holds over 20 Billion rows of EPOS based data



Hold over 2 years history on granular data. This is over 2 billion customer baskets containing over 12 billion items



In addition to what the baskets contain we also know how they were paid for, which promotions they triggered, the promotional discount the customer received and any other reductions



Every day 9000 files are collected from the store estate. Over 25 million rows of data are loaded each night such that Monday’s sales data is available by 7.00am on Tuesday morning



Understand in detail the buying patterns of customers and members

© NCC

- good with information • Loss prevention – minimise leakage & wastage • EPOS transaction data analysed for unusual patterns of behaviour • Created bespoke dashboards & reports - 90 + users across country • Allows loss prevention team to be more pro-active • First year produced £2 million of savings – avoided loss

“Business intelligence is an integral part of running our operations day-to-day and without it we would be running our business blind.” Mark Hale – Director of IS - Co-operative – Food Retail

© NCC

Summary • Realisation that company data is a strategic asset • Effective strategy is essential • Need senior management to drive on-going development • Future looking as important as real time • Reports need to be flexible, customisable and interactive

© NCC