ODI Experts

The blog for Oracle Data Integrator

Home
Author
POST
Contact
BLOG
RSS

Changed Data Capture ( CDC )

Changed Data Capture as the term implies is used to capture the data that is inserted, updated and deleted at the source side and replicating the same at the target. ODI have a Journalizing
KM to do the required implementation and it is basically into two types simple and consistent.

Simple is used for CDC implementation on a single table and Consistent on Multiple tables or Model.

In this post i will be covering Simple CDC concept . Look for my future post for Consistent based CDC implementation.

Simple CDC – Oracle Table

I am going to implement the Simple CDC concept on this particular table – SRC_AGE_GROUP.

Step 1 – Adding the Table to CDC

Step 2 – Starting the journalizing

Select the user as Subscriber

Execute it

The journalizing process is established.

In the above execution ODI creates a Subscriber table in the work schema, J$ table, view and a trigger to capture any data changes when any Insert, Update and Deletion takes places.
After successful execution you will be able to see a small green clock icon.

Let’s do a simple mapping and test for CDC.

Drag the Journalized Table as source and the required target. On the Source Data store, check mark the option “ JOURNALIZED DATA ONLY “ and ODI will automatically add a
filter with a required date condition and subscriber information, use the proper LKM and IKM as per your technology. For this Example I have used IKM Oracle Incremental Update.

I have run it once to check whether its working fine or not. Now let me show you an example.

Original Source

Modified source

I have removed the last column and change the third row age_max 39 to 35.

There are two ways of seeing Journalized Data.

On the Journalized Interface source and Right click on filter and click on data.

Here is the sql from where the journalized data is being retrived
select * from ODI_TEMP.JV$DSRC_AGE_GROUP SRC_AGE_GROUP where (JRN_SUBSCRIBER = ‘SUNOPSIS’ /* AND JRN_DATE < sysdate */)

The Other way is to right click on Journalized data store in Model and Changed Data capture – > Journal Data and you will see the above data again too.

Let’s capture the Journalized data to my Target table.

Target Table Before running the Interface

As you can see, one Update and one Deletes.

Target Table after running the Interface

MSSQL – SIMPLE

Repeat the above steps for Journalizing SQL Server table as we did for Oracle namely

Add to CDC

Start Journal

And if everything is start the journalizing will start.

Source Data before Running the Journalized Interface

delete from dimtime_backup where TimeKey=6;

update dimtime_backup set DayNumberOfWeek=4 where TimeKey=4;
Journalized Data

Although there is update and Delete on the source but there is just delete in the ODI reason being I have update the same value DayNumberOfWeek=4 again for update so ODI have
correctly did only the Delete as the target already had the same value which is DayNumberOfWeek=4

How to say whether you have which JKM you have , right click on your model , Edit – >Journalizing Tab and you see what JKM is being used and whether Simple or Consistent

Rating: 8.7/10 (6 votes cast)

Posted: March 14th, 2010 | Author: Kshitiz Devendra and Cezar Santos | Filed under: How to, Logic, ODI, Tips and Tricks | Tags: CDC, Data Capture, JKM, Journalizing, ODI,
Simple, Simple CDC | Comments: 8 Comments »

8 Comments on “Changed Data Capture ( CDC )”

1. 1 Alberto said at 4:09 pm on September 27th, 2010:

Hello there,

Thanks for the tutorial.

The following question: How should be configured a replication real time?, How can I achive an automatic replication without manually executing
the Interfaz?

Thanks in advance.

2. 2 Cezar Santos said at 10:45 am on September 28th, 2010:

Hi Alberto…

there is few tools into package that are used for “wait” new CDC records…

Take a look in OdiWaitForLogData or OdiWaitForLogData for instance….

Does it help you?

3. 3 Navan said at 12:13 am on October 6th, 2010:

Hi Cezar
Thanks for explaining so nicely.
I tried the steps that you mentioned and was successful in doing so.
Now, as per this tutorial, we are deleting the records manually from the table which is journalized. What if we need to automate the deletion.
Eg: I extract members from Essbase database into a database table. I apply journalizing on this DB table. Now whenever a new member gets
added or any member gets updated, the journalized data shows records with JRN_FLAG as ‘I’. What if i delete any member from Essbase? When
 these members are extracted, journalized data should show records with JRN_FLAG as ‘D’. Is it possible?

4. 4 Alberto said at 11:45 am on October 8th, 2010:

That was helpfull.

I’ve achived a replication real-time using OdiWaitForLogData.

Thank you very much.

5. 5 Ritika said at 1:29 am on November 16th, 2010:

Hi,
Will the interface run automatically as soon the changes are made in the source table?
Thanks,
Ritika

6. 6 Kshitiz Devendra said at 7:27 am on November 16th, 2010:

ritika,

CDC make use of the source trigger to capture the changed record and it wont trigger the interface ,but when you trigger the interface or if you
call it in the package and scheduler it accordingly it can capture the changed records and sync the target.

7. 7 Ritika said at 11:10 pm on November 16th, 2010:

Hi Dev,
Thanks for the reply
Does ODIExperts site have steps to step up consistent CDC?
I was trying to find out but was unsuccessful!

8. 8 Kshitiz Devendra said at 6:13 am on November 17th, 2010:

Presently None but we will write one sooner or later.

Leave a Reply

SEARCH

TRANSLATE
Translate to:

AUTHOR

Cezar Santos
Kshitiz Devendra

Invited Experts

Craig Stewart

CATEGORIES