Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
05/11/2008
Hi-Tech ISU
Manish Gupta
manish.g@tcs.com
Requirement
Assumptions
There will be both inserts and updates in the Table A. These changes must be
reflected in Table B.
Solution
• There will be one initial load which will move all the historical records from Table
A to Table B.
• During subsequent incremental loads, new records will be copied and updated
records will be modified from Table A to Table B.
TCS Public
Strategies
There are in all 4 strategies which can be used for data migration during incremental
loads –
2. Compare all records in both database and data warehouse and write the
Deltas
The approach is to compare each field in the source (Table A) with the fields in the
data warehouse (Table B), identify the changes and insert/update the records in the
data warehouse (Table B).
Pros: None
This approach may be necessary if there is no column in source table (Table A) to
identify delta records but complete refresh of data is a better option.
Cons: This is not a practical approach for large volume databases.
TCS Public
4. Real Time Load
The approach is to load an intermediate table with delta records only as soon as
there are new/updated records in source table (Table A). The intermediate table can
be loaded using some trigger on source table (Table A) or some other methodology.
Pros: This is the best and easiest approach as we need to process only intermediate
table.
Cons: Tables similar to source (Table A) needs to be created at source.
TCS Public