Dr.

Abdul Basit Siddiqui Assistant Professor FUIEMS

(Lecture Slides Week # 1)

Approach of the Course

Develop an understanding of the underlying RDBMS

concepts. Apply these concepts to VLDB / DSS environments and understand where and why they break down? Expose the differences between RDBMS and Data Warehouse in the context of VLDB. Provide the basics of DSS tools such as OLAP, Data Mining and demonstrate their applications. Demonstrate the application of DSS concepts and limitations of the OLTP concepts through lab exercises.

FUIEMS

Data Warehoue & Mining - Spring 2012

Summary of the Course

Introduction & Background De-Normalization Online Analytical Processing (OLAP) Dimensional Modeling Extract-Transform-Load (ETL) Data Quality Management (DQM)
Parallelism, Join and Indexing Techniques Data Mining Concepts Data Cleansing Association Rule Mining Clustering Classification

FUIEMS

Data Warehoue & Mining - Spring 2012

Books Books Reference

W. H. Inmon, Building the Data Warehouse,

John Wiley & Sons Inc., NY R. Kimball, The Data Warehouse Toolkit, John Wiley & Sons Inc., NY Paulraj Ponniah, Data Warehousing Fundamentals, John Wiley & Sons Inc., NY

FUIEMS

Data Warehoue & Mining - Spring 2012

Why this Course?

The World is changing / (in fact changed) Either change or Be left behind. Missing the opportunities or going in the wrong

direction has prevented us from growing. What is the right direction?

harnessing the data, in the knowledge driven economy. Doing what cant be or difficult to automate.

FUIEMS

Data Warehoue & Mining - Spring 2012

Historical Overview
1960: Master Files and Reports 1965: Lots of Master Files 1970: Direct Memory Access and DBMS 1975: Online High Performance Transaction Processing 1980: PCs and 4GL Technology (MIS/DSS) 1985: Extract Programs, Extract Processing

1990: The Legacy Systems Web

FUIEMS Data Warehoue & Mining - Spring 2012 7

The Need of the Time

drowning in data AND/BUT starving for information. Knowledge is power BUT Intelligence is absolute/super

power.

FUIEMS

Data Warehoue & Mining - Spring 2012

The Need of the Time

POWER ($/)

Intelligence Knowledge Information Data

FUIEMS Data Warehoue & Mining - Spring 2012 9

Scenario 1

ABC Pvt Ltd is a company with branches at Karachi, Quetta, Peshawar and Lahore. The Sales Manager wants quarterly sales report. Each branch has a separate operational system.

FUIEMS

Data Warehoue & Mining - Spring 2012

10

Scenario 1 : ABC Pvt Ltd.

Karachi

Quetta Sales per item type per branch for first quarter. Peshawar Sales Manager

Lahore

FUIEMS

Data Warehoue & Mining - Spring 2012

11

Solution 1:ABC Pvt Ltd.

Extract sales information from each database.

Store the information in a common repository at a single site.

FUIEMS

Data Warehoue & Mining - Spring 2012

12

Solution 1:ABC Pvt Ltd.

Karachi

Report Quetta Data Warehouse Peshawar Query & Analysis tools Sales Manager

Lahore

FUIEMS

Data Warehoue & Mining - Spring 2012

13

Scenario 2
One Stop Shopping Super Market has huge operational database. Whenever Executives wants some report, the OLTP system becomes slow and data entry operators have to wait for some time.

FUIEMS

Data Warehoue & Mining - Spring 2012

14

Scenario 2 : One Stop Shopping

Data Entry Operator

Report Wait Operational Database Management

Data Entry Operator

FUIEMS

Data Warehoue & Mining - Spring 2012

15

Solution 2
Extract

data needed for analysis from operational database. Store it in warehouse. Refresh warehouse at regular interval so that it contains up to date information for analysis. Warehouse will contain data with historical perspective.

FUIEMS

Data Warehoue & Mining - Spring 2012

16

Solution 2
Data Entry Operator Report Transaction Extract data Data Warehouse

Operational database

Manager

Data Entry Operator

FUIEMS

Data Warehoue & Mining - Spring 2012

17

Scenario 3
Cakes & Cookies is a small, new company. President of the company wants his company should grow. He needs information so that he can make correct decisions.

FUIEMS

Data Warehoue & Mining - Spring 2012

18

Solution 3
Improve the quality of data before loading it into the

warehouse. Perform data cleaning and transformation before loading the data. Use query analysis tools to support adhoc queries.

FUIEMS

Data Warehoue & Mining - Spring 2012

19

Solution 3
Expansion

sales Data Warehouse Query and Analysis tool time Improvement

President

FUIEMS

Data Warehoue & Mining - Spring 2012

20

Case Study
AFCO Foods & Beverages is a new company which produces dairy, bread and meat products with production unit located at Gujranwala. There products are sold in all the region of Pakistan. They have sales units at provincial Head Quarters. The President of the company wants sales information.

FUIEMS

Data Warehoue & Mining - Spring 2012

21

Sales Information
Report: The number of units sold. 113

Report: The number of units sold over time

January 14 February 41 March 33 April 25

FUIEMS

Data Warehoue & Mining - Spring 2012

22

Sales Information
Report : The number of items sold for each product with time
Jan Feb Mar Apr

Wheat Bread
Cheese Swiss Rolls 6 8 16 25

6
6 21

17
8

Product

FUIEMS

Data Warehoue & Mining - Spring 2012

23

Sales Information
Report: The number of items sold in each City for each product with time
Jan Karachi Wheat Bread Feb Mar 3 Apr 10

Cheese
Swiss Rolls Lahore Wheat Bread Cheese Swiss Rolls
FUIEMS

3
4

16
16

6
6 3 7
Product Time

3 4 9 15

Data Warehoue & Mining - Spring 2012

24

Sales Information
Report: The number of items sold and income in each region for each product with time.
Jan Rs Karachi Wheat Bread Cheese Swiss Rolls Lahore Wheat Bread Cheese Swiss Rolls 7.95 7.32 3 4 16.47 9 27.45 15 7.95 7.32 3 4 U Feb Rs U Mar Rs 7.44 42.40 16 15.90 29.98 16 10.98 7.44 U 3 6 6 3 17.36 7 21.20 8 Apr Rs U 24.80 10

FUIEMS

Data Warehoue & Mining - Spring 2012

25

Data Warehousing includes

Build Data Warehouse

Online Analysis/Analytical Processing (OLAP).

Presentation.
Cleaning ,Selection & Integration RDBMS Presentation

Flat File

Warehouse & OLAP server

FUIEMS Data Warehoue & Mining - Spring 2012

Client

26