Odi Intro Hol

Introduction to Oracle Data Integrator: Hands-on Lab
Introduction to Oracle Data

Integrator: Hands-on Lab
Last Updated: 31-Aug-10
Page 1 of 47
Hands on Lab -Introduction to Oracle Data Integrator EE

Abstract: This lab will highlight to Developers, DBAs and Architects some of the best practices for
implementing E-LT techniques to improve performance and reduce data integration costs using
Oracle Data Integrator Enterprise Edition (ODI-EE). In this lab, participants will walk through the steps
that are needed to load and transform various sources into a relational table target. This will also
include how to deploy and manage interfaces using ODI-EE.
Scenario: Our company is in the process of building a BI system for reporting on our customer and
sales data. The physical schema for our target star schema has been finalized and we have been
assigned to build the interfaces and packages in ODI to load this star schema.
Our supporting data is in various heterogeneous systems, located across the enterprise. Sales,
customer and country data will be imported from our Sales system located on SQL Server. Our
product management system contains our product and costing data and this information will be
provided as comma delimited extracts. Our marketing system is an internet based system; we will be
provided an XML extract of our promotional data from this system.
The following lessons will walk us through various steps that are needed to create the interfaces and
packages need to load the reporting star schema.
HANDS ON LAB -INTRODUCTION TO ORACLE DATA INTEGRATOR EE ................................................. 2
OVERVIEW .............................................................................................................................................. 3
Time to Complete ............................................................................................................................ 3
Prerequisites.................................................................................................................................... 3
ARCHITECTURE OVERVIEW.......................................................................................................................... 3
Sources: ........................................................................................................................................... 3
Target: ............................................................................................................................................. 3
PART 1: REVIEW TOPOLOGY SETUP .............................................................................................................. 4
PART 2: EXTRACTING, LOADING AND TRANSFORMING PRODUCT DATA TO ORACLE DIMENSION TABLES .................... 8
PART 3: EXTRACTING, LOADING AND TRANSFORMING SALES DATA TO ORACLE DIMENSION TABLES ....................... 26
OPTIONAL PART 4: EXTRACTING, LOADING AND TRANSFORMING PROMOTION DATA TO ORACLE DIMENSION TABLES 35
OPTIONAL PART 5: CREATING A NEW ODI PACKAGE TO EXECUTE THE FULL DIMENSION AND SALES TABLE BUILD. ...... 44
SUMMARY ............................................................................................................................................. 47
Page 2 of 47
Overview
Time to Complete
Create interfaces and packages 50 Minutes
Prerequisites
Before you begin this tutorial, you should
Have a general understanding of RDBMS concepts.

Have a general understanding of ETL concepts.
Architecture Overview
Sources:
Source
Sales
Customers
Countries
Promotions
Products
Costs
Type
SQL Server
SQL Server
SQL Server
XML
CSV
CSV
Target:
Source
Sales
Customers
Type
Oracle 10g
Oracle 10g
Page 3 of 47

Countries
Promotions
Products
Costs
Oracle 10g
Oracle 10g
Oracle 10g
Oracle 10g
Part 1: Review Topology Setup

1. The connectivity information has already been setup for this hand on lab.
This information is setup within the Topology Manager of ODI. The next
steps will walk you through how to review this information.
2. Double-click the icon Reset_ODI.bat from the desktop. This will reset the files
for all Hands-on Labs to the initial state.
Warning: If you have custom ODI projects in the repository on this machine,
they will be deleted.
3. Open the Oracle Data Integrator Studio using the Start Menu > Oracle >
Oracle Data Integrator > ODI Studio menu.
4. Go to the Topology Navigator and press Connect to Repository
5. Select HOL Basic from the drop-down list of logins and press OK.
6. Within the Physical Architecture accordion, expand the Technologies folder
Page 4 of 47
Page 5 of 47

7. For this HOL the connectivity information has been setup. Connectivity
information is setup for File, Oracle, SQL Server and XML sources and targets.
Page 6 of 47

8. Double-click on the Oracle_DW data server to review settings
9. Expand the data server and double-click on ORACLE_DW.OOW_HOL_DW

physical schema to review the settings.
10. Review any additional items as needed.
Page 7 of 47
Part 2: Extracting, Loading and Transforming Product Data to

Oracle Dimension Tables
1. The first interface to be created will load the products data (PRODUCTS.CSV)
into the Oracle star schema. The products data will be joined with the
current cost data (CURRENT_COSTS.csv) file so the product cost (UNIT_COST)
can be stored in the PRODUCTS dimension table (PROD_COST)
High level architecture
Page 8 of 47

2. Open the Oracle Data Integrator Studio using the Start Menu > Oracle >
Oracle Data Integrator > ODI Studio menu.
3. Choose the Designer tab; If not already logged in, click on Connect To
Repository
1. Select HOL Basic from the drop-down list of logins and press OK.
4. Open the Designer navigator menu and make sure that the Default Execution
Context is set to the Development context (not Production).
Page 9 of 47

5. Open the Models accordion in the Designer navigator by clicking on the label.
6. Expand the ORACLE DW TRG folder
7. The target Oracle star schema has been physically created within the RDBMS.
But the logical data stores to represent the physical model have not been
created within ODI. The logical data stores can be created by reverse
engineering the Oracle ORACLE_DW_SALES.
Info: A model is a set of metadata definitions regarding a source such as a database
schema or a set of files. A model can contain multiple data stores, which follow the
relational concept of columns and rows and can be database tables, structured files,
or XML elements within an XML document .
Page 10 of 47

8. To reverse double-click the existing model ORACLE_DW_SALES. The model
editor is opened on the right side.
9. Click the Selective Reverse-Engineering tab
10. Check the Selective Reverse-Engineering checkbox and the Objects to

Reverse Engineer checkbox. The list of tables to reverse engineer is shown.
11. Click the Reverse Engineer (
) button in the toolbar.
The reversed tables will show up in the model tree and are now logical data
stores within ODI that can be used as a source or target within an ODI
Interface.
Page 11 of 47

Note: The other data stores that are needed for the hands on lab have
already been created.
12. The first interface to be created will load the products data (PRODUCTS.CSV)
into the Oracle star schema. The products data will be joined with the
current cost data (CURRENT_COSTS.csv) file so the product cost (UNIT_COST)
can be stored in the PRODUCTS dimension table (PROD_COST)
To create a new interface, open the Project accordion within the Designer
navigator:
13. Expand the HOL BASIC > First Folder folder
Page 12 of 47
14. Right click on Interfaces and click New Interface
Info: An interface is a data flow to move and transform data from multiple sources
into one target. It contains declarative and graphical rules about data joining and
transformation.
15. In the Interface editor name the interface INT_LD_PRODUCTS
Note: The staging area does not need to be changed since we will use the
target as the staging area.
Page 13 of 47

16. Click the Mapping tab below the editor
17. For this interface we will load the PRODUCTS.csv file and
CURRENT_COSTS.csv from model FlatFileSrc to PRODUCTS table within the
model ORACLE_DW_SALES.
To view the models open the Models accordion
18. Open the FLAT FILE SRC folder and FlatFileSRC model and drag the
CURRENT_COSTS and PRODUCTS source csv files into the source panel of the
designer
Page 14 of 47

19. Insert target data store by dragging the PRODUCTS table from
ORACLE_DW_SALES model to the Target Datastore panel
20. Click Yes on the confirmation for auto mapping. ODI will perform some
mapping for you.
21. Join the csv files by click the appropriate column in each table and dragging
to one another. The tables are joined by PROD_ID and PROD_ID
22. Set the Join to an Left Outer Join so no rows are lost when loading to the
PRODUCTS table. Also change where the join will be executed by selecting
Staging Area. Since we are using a data type that does not accept joins we
must do this join in the staging area. Both tables will be loaded to the target
and then joined.
Page 15 of 47
23. The IKM Incremental update needs a primary key set to be able to
differentiate between updated and new records. To set the primary key, click
on the mapping for PROD_ID and click the Key check box in the mapping
below.
24. We will now map the remaining target columns PROD_INSERT_DATE,

PROD_UPDATE_DATE and PROD_COST that were not mapped by automapping earlier.
ODI allows for a special feature that will incrementally update the target
based on the source and target data. We only want to insert/update the
PROD_INSERT_DATE column where there is a new record. And we want to
update the PROD_UPDATE_DATE on every insert/update.
Click on the PROD_INSERT_DATE column within the target to bring up the
transformation mapping.
Page 16 of 47
25. Within the Implementation area enter SYSDATE, you can do this directly in
the text box or open the expression editor. Also make sure to choose Target
for the mapping, and uncheck the Update checkbox. Un-checking the update
turn off updates for this column when this row is updated based on the
primary key.
Page 17 of 47

26. To set the transformation for the PROD_UPDATE_DATE click the
PROD_UPDATE_DATE mapping
27. Within the mapping area enter SYSDATE, you can do this directly in the text
box or open the expression editor. Also make sure to choose Target for the
mapping, Insert and Update are checked by default. Make sure both are
checked to ensure the column is updated for both inserts and updates.
28. The PROD_COST must also be mapped to the UNIT_COST of the

CURRENT_COST table. To map this , drag the UNIT_COST field over to the
Page 18 of 47

PROD_COST target column.
29. The transformations have now been setup. To set the IKM click on the Flow
tab of the editor.
Note: The KMs that will be used have already been imported into the project.
Info: A knowledge module (KM) is a template that represents best practices to
perform an action in an interface, such as loading from/to a certain technology (Load
knowledge module or LKM), integrating data into the target (Integration Knowledge
Module or IKM), checking data constraints (Check Knowledge Module or CKM), and
others. Knowledge modules can be customized by the user.
ODI will select default KMs. For the small PRODUCT.CSV file the LKM File to
SQL should be used. For the larger CURRENT_COSTS.CSV file the LKM File to
Oracle (SQLLDR) should be used.
To set the LKM for each source, click the title of the source; this will open the
KM properties.
Page 19 of 47

30. Click the title of the PRODUCTS source and select the LKM File to SQL
31. Click the title of the CURRENT_COSTS source and select the LKM File to
Oracle(SQLLDR)
No coding is needed by the user. ODI will create all needed scripts.
32. In the Options for LKM File to Oracle (SQLLDR), set
DELETE_TEMPORARY_OBJECTS to false.
Page 20 of 47
33. Click the title Staging Area to bring up the IKM properties
Check that the IKM Oracle Incremental Update is being used.

NOTE: We will not be using FLOW_CONTROL for this lab, turn this option to
false. Set DELETE_ALL to true to overwrite any prior results.
Page 21 of 47
34. The interface has now been setup to join the Products and Costs file and load
into the Products table. To execute the interface click the Execute ( )
button in the toolbar.
35. When prompted to save click Yes
36. The execution context is Development, and the agent is Local.

Click OK
37. Click OK on the confirmation window
38. To review execution go to the Operator navigator.

Page 22 of 47
39. Expand Today and the appropriate execution

Double click any task to review the execution.
40. Click details or execution tab for more information such as inserts or updates.
Page 23 of 47
41. 72 new rows should have been inserted into the Products Table.
42. Go back to the INT_LD_PRODUCTS interface editor. You can view the new
data in the PRODUCTS table from the designer by right clicking the target and
clicking Data...
43. Review All Rows
Page 24 of 47
44. The data has now been loaded successfully for the PRODUCTS table with
new costing information from the CURRENT_COSTS.CSV table.
Page 25 of 47
Part 3: Extracting, Loading and Transforming Sales

Data to Oracle Dimension Tables
1. For this interface we will be de-normalizing the SALES table by including not
only the dimension IDs but also the dimension base members for faster
reporting. This will allow our BI application to query one table for detailed
reports.
Our source table SALES is on SQL Server, it will be joined to the CUSTOMER
table which is also on SQL Server to lookup the Customer Name. We will also
join SALES to the PRODUCTS and to the PROMOTION table which have
already been loaded to our Oracle target.
High Level Architecture
2. Start the ODI Studio if not already started and open the HOL_BASIC login
3. To create a new interface, open the Designer navigator and Project accordion
on the HOL_BASIC / First Folder:
Page 26 of 47

4. Right click and select New Interface
5. Name the interface INT_LD_SALES
6. Click the Mapping tab and drag and drop the sources SALES and CUSTOMERS
from SQL SERVER SRC / SQL_SERVER_SALES model onto the sources panel:
7. A join will automatically be created for the SALES and CUSTOMER Tables.
Click the join to view the join properties.
Page 27 of 47
Make sure the join is set to Source. This will allow the integration to take
advantage of the exiting indexes on the source. Doing this type of join in
conventional ETL tools would require custom SQL expressions.
8. Drag the PRODUCTS and PROMOTIONS tables from ORACLE DW TRG /
ORACLE_DW_SALES into the source panel.
9. Drag the table SALES from the ORACLE_DW_SALES model to the Target
Datastore area.
Page 28 of 47
10. Click Yes to allow for auto mapping

11. Join PRODUCT by dragging from PROD_ID to the SALES table PROD_ID
column, make sure the join is on changed to Staging Area.
12. Join the PROMOTIONS data by joining PROMO_ID to the PROMO_ID of the
SALES table, make sure the join is on the Staging Area.
Page 29 of 47
13. Map PRODUCT to PROD_NAME from the joined PRODUCT table.
14. Map the target field CUSTOMER by concatenating the first and last name of
the customer using a mapping on the Staging Area of
CUSTOMERS.CUST_FIRST_NAME || ' ' || CUSTOMERS.CUST_LAST_NAME
Page 30 of 47
15. Map the target field PROMOTION to PROMO_NAME from the joined
PROMOTIONS table.
16. The design and mappings are now complete. Click on the Flow tab.
Page 31 of 47
The power of ELT and ODI is apparent by how easy it is to set certain
transformation on the source, target or staging area. ODI will generate the
flow automatically, we have only defined what we want to do, not how it will
be done.
17. The KMs will be selected by default for the source and target.
Source LKM: LKM SQL to Oracle
Target IKM: IKM Oracle Incremental Update
18. Select the title of the Staging Area. Set the FLOW_CONTROL Option to false.
Set DELETE_ALL to true to overwrite any prior results.
Page 32 of 47

19. The interface has now been setup to extract, de-normalize and load the
SALES table from SQL Server to Oracle, using lookup tables on SQL Server and
on Oracle. To execute the interface click the Execute ( ) button in the
toolbar.

Click OK
Page 33 of 47
24. Double click any step for details, click the details, or execution tab for more
information. The execution of the interface might take up to a few minutes to
complete, during which time the completed steps will be shown.
Page 34 of 47
Optional Part 4: Extracting, Loading and Transforming

Promotion Data to Oracle Dimension Tables
1. This interface will load the Promotion data from an XML file to the Oracle star
schema PROMOTIONS table.
High Level Architecture
2. Start the ODI Studio if not already started and open the HOL_BASIC login
Page 35 of 47

3. For this interface the source PROMOTIONS.xml file will be used. ODI has a
built in JDBC driver for XML, that will be used to extract the data.
Info: ODI works with XML data by using a specialized JDBC driver that will map an
XML schema into multiple tables. Each non-leaf element is represented by a table,
hierarchy is reflected through PK-FK relationships. Using XML sources in interfaces is
the same as using a relational DB source.
4. To create a new interface, open the Designer navigator and Project accordion
5. Right click and select New Interface
Page 36 of 47
6. Name the interface INT_LD_PROMOTIONS
7. Click on Mapping tab

Open to the Models accordion on the left navigator and drag PROMODATA
into the source panel of diagram tab.
Page 37 of 47
8. Drag the PROMOTIONS data store from the ORACLE_DW_SALES model to

the Target Datastore panel
9. Click YES to auto mapping confirmation

10. Again for the logic of the incremental update a key must be set on the target
table. To set the PROMO_ID as the key select the mapping and check the
Key check box below.
11. The source data might contain records with a NULL begin data. We need to
set a filter to ignore these records.
Page 38 of 47

From the PROMODATA source field drag the PROMO_BEGIN_DATE field
onto an empty area of the diagram background.
12. Select the filter and edit the Implementation property of the filter. Complete
the expression to read: PROMODATA.PROMO_BEGIN_DATE is not null
13. The date columns in the XML file are setup as varchar fields. We can
implicitly convert these within the staging area by select the Staging Area for
the mapping of the two date fields PROMO_BEGIN_DATE and
PROMO_END_DATE.
Page 39 of 47
14. Click the Flow tab to review the Flow and KM settings
Note: By default the LKM SQL to Oracle KM should have been selected for
the XML Source.
If not, select this LKM. Even though this is an XML file, the JDBC driver allows
ODI to interact with the file as though it is a relational table.
Page 40 of 47

15. Select the Staging Area to review the IKM setting. By default the IKM Oracle
Incremental Update has been set.
25. Note: Make sure to set the IKM option for FLOW_CONTROL to NO. Set
DELETE_ALL to true to overwrite any prior results.
45. The interface has now been setup to load the Promotions XML file to the
PROMOTIONS table. To execute the interface click the Execute ( ) button
in the toolbar.

Click OK
Page 41 of 47
16. Review the execution of the steps for more information double click a step,
and view the details or execution tab
Page 42 of 47
17. You have now successfully loaded the Promotions XML file to the
PROMOTIONS Table on the Oracle DW Sale star schema.
Page 43 of 47
Optional Part 5: Creating a new ODI Package to execute the

full dimension and Sales table build.
1. Now that the interfaces have been created we can create a package within
ODI that will execute all of the interfaces in order.
2. To create a new package, open the Designer navigator and Project accordion
3. Right click Packages and select New Package.
Info: A package is a task flow to orchestrate execution of multiple interfaces and

define additional logic, such as conditional execution and actions such as sending
emails, calling web services, uploads/downloads, file manipulation, event handling,
and others.
4. Name the package PKG_LD_ALL
5. Click the Diagram tab

Drag and Drop the interfaces from the left onto the diagram panel, starting
with the interface INT_LD_PRODUCTS
Page 44 of 47
Notice the green arrow on this interface which means it is the first step.
6. Drag the interfaces INT_LD_PROMOTIONS and INT_LD_SALES onto the panel
7. Click the OK arrow toolbar button to select the order of precedence.
8. Drag and drop from the INT_LD_PRODUCTS to the INT_LD_PROMOTIONS to

set the link. Then drag and drop from INT_LD_PROMOTIONS to
INT_LD_SALES.
9. The package is now setup and can be executed. To execute the interface click
the Execute ( ) button in the toolbar.
Page 45 of 47

Click OK
13. To review execution go to the Operator navigator and open the latest session
execution.
Page 46 of 47
Summary
You have now successfully completed the Hands on Lab, and have successfully
loaded the Product and Promotions dimension table as well as de-normalizing and
loading the sales fact table. Once the interfaces were created to load and transform
the data, a package was created to execute all the interfaces in the correct order.
Page 47 of 47

Odi Intro Hol

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Odi Intro Hol

Caricato da

Copyright:

Formati disponibili

Introduction to Oracle Data Integrator: Hands-on Lab

Introduction to Oracle Data

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Hands on Lab -Introduction to Oracle Data Integrator EE

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Have a general understanding of RDBMS concepts.

Introduction to Oracle Data Integrator: Hands-on Lab

Part 1: Review Topology Setup

6. Within the Physical Architecture accordion, expand the Technologies folder

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

9. Expand the data server and double-click on ORACLE_DW.OOW_HOL_DW

10. Review any additional items as needed.

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Part 2: Extracting, Loading and Transforming Product Data to

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

6. Expand the ORACLE DW TRG folder

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

9. Click the Selective Reverse-Engineering tab

10. Check the Selective Reverse-Engineering checkbox and the Objects to

11. Click the Reverse Engineer (

) button in the toolbar.

Introduction to Oracle Data Integrator: Hands-on Lab

13. Expand the HOL BASIC > First Folder folder

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

14. Right click on Interfaces and click New Interface

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

24. We will now map the remaining target columns PROD_INSERT_DATE,

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

28. The PROD_COST must also be mapped to the UNIT_COST of the

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

Check that the IKM Oracle Incremental Update is being used.

Last Updated: 31-Aug-10

Introduction to Oracle Data Integrator: Hands-on Lab

36. The execution context is Development, and the agent is Local.

37. Click OK on the confirmation window

38. To review execution go to the Operator navigator.

Introduction to Oracle Data Integrator: Hands-on Lab

39. Expand Today and the appropriate execution