Data Team

BOA and RTL Integration

CONFIDENTIAL – For planning purposes only. Subject to any applicable information and consultation requirements
BAO/RTL Integration Requirements

Four Fundamental Points of Integration/tasks

1. Technical integration between Information Server and DSP (SQL

Server)
2. Extraction of TaO Configuration client check tables via
DataStage/Oracle and posting tables to DSP backend database (SQL
Server)
3. Acquisition of “CLT” data from DSP backend database (SQL Server)
into RTL Staging (Oracle) database via DataStage
4. Legacy data extraction via DataStage using selection criteria and
writing results to DSP backend database (SQL Server)
TaO Configuration Extraction
TaO Configuration Tables

• Currently configuration extraction is being done using DataStage SAP

Packs
• There are approximately 600 configuration tables being extracted there
may be additional tables, but the volume will be small
• The configuration tables are extracted from jobs in the
TaO_Gap_Report DataStage project. They are pulled daily and
landed in the Pre-load database prefixed by JLT_
• This can be done using one DataStage job, one DataStage sequence
and one control file that contains all of the check tables to be extracted
 SAP Configuration Table Integration Architecture

BOA RTL
1. Extract Check Tables
3. Tables in the SQL from TaO
Server data base will be 2. Extraction is landed in
truncated, then loaded BOA _Landing_Area

SAP System
BOA DB
from which
BOA
Landing extraction is
happening will
vary. This is
SQL Server
dependent on
the status of the
configuration for
each DG

* Applies to all Oracle Staging

environments – SBC, SBC,
PBC and PBD
 SAP Configuration Table Integration Architecture

• Jobs that extract configuration are located in the DataStage project

TaO_Gap_Reports/Jobs/01 RTL_GAP_REPORTS/002
Extract_Check_Tables
• The master sequence is sq_TaO_JTL_Extracts_All. We need to
schedule this extract to be done daily.
• Parameter sets need to be updated with values – see next Slide
• The jobs that extract the SAP configuration tables into the
BOA_Landing_Area then into SQL Server should be moved to this
project under TaO DATA MIGRAITON OBJECTS
• Create audit table of Configuration table extraction
 SAP Configuration Table Parameter Sets

For the TaO_GAP_REPORTS in Z2T2DEFMVA023 (Dev/Test)

– SAPPE_Target – include values for source SAP System (i.e. B04)
– SAP_RLT_Database – include values for all BOA_Landing_Area
schemas SBC and SBD

For the TaO_GAP_REPORTS in Z2T2DEFMVA144 (Prod)

– SAPPE_Target – include values for source SAP System (i.e. B04)
– SAP_RLT_Database – include values for all BOA_Landing_Area
schemas PBC and PBD
 SAP Configuration Extraction Dev/Test

Configuration
BOA DB Configuration Tables BOA Tables
Landing

Audit Table
DSP Silver Oracle Development TaO Configuration
Database (SBC/SBD)

For Each Extract

Tables are Truncated
For Each Extract
Tables are Truncated

Development/Test
 SAP Configuration Extraction Production

Configuration
BOA DB Configuration Tables BOA Tables
Landing

Audit Table
DSP Gold Oracle Development TaO Configuration
Database (PBD)

For Each Extract For Each Extract

Tables are Truncated Tables are Truncated

Production
 DSP Development, Test and Production

DSP RTL TaO

SBC B04

Silver
Configuration source
depends on where the
SBD T04 latest configuration is
located.

Gold

PBD R04
Configuration Extraction Effort Estimation

• 3 new data migration artifacts need to be created

– Use existing DataStage extraction from TaO SAP system
– DataStage Sequence to iterate through a check table list and
execute above job
– Creation of a list of check tables to be migrated to DSP for above
DataStage Sequence
– Audit table:
• Configuration Table Varchar(50)
• Data Extracted Date
• Status Char(1) Y=Successful, N=Failed
• TaO_System Varchar(10)
• Update existing row if it exists, insert for new record
“CLT” Acquisition
“CLT” acquisition from SQL Server to Oracle

• Current process is when a CLT is uploaded it is posted to two Oracle

schemas - CLT CSV (for data migration) and CLT DQ (for Data
Quality)
– CLT CSV the data is replaced with newly uploaded data
– CLT DQ data is versioned for historical data quality reporting
• Objective is to make the downstream processes unchanged to
minimize impact
 SAP Configuration Table Integration Architecture
Red Text = New Objects Required

BOA RTL
2. Generic DataStage
1. Read extraction Job that loads
table that contains changed data into
views to be extracted CLT DG (Insert)
3. Replace tables that
were updated CLT CSV

DSP Server Information Server

Staging
CLT CLT DQ
BOA DB
Landing

SQL Server Oracle

* Applies to all Oracle Staging

environments – SBC, SBC,
PBC and PBD
“CLT” acquisition from SQL Server to Oracle Estimation

SQL Server tables to Oracle

• Create additional schema in Oracle to land extracted data from DSP SQL Server. Data
from this schema will be written to the current schemas – CLT CSV and CLT DQ – as is
the current process. The benefit is that no further downstream “CLT” processing needs
to be modified.
• 3 new data migration artifacts need to be created (for posting data to the new schema)
– Generic DataStage Job to read extracted configuration from above schema and
target to SQL Server
– DataStage Sequence to iterate through a check table list and execute above job
– Creation of a list of check tables to be migrated to DSP for above DataStage
Sequence
• Extraction of tables (Sales Conditions and Characteristics) that need pivoting
• Housekeeping job to remove older rows in CSV DQ (see next slides)
• Write audit details to RTL Audit table (see next slides)
Remove “Old Records” from DQ_CSV_DQ

• Housekeeping job to clean up DQ_CSV_DQ

– Delete records that have more than 2 versions
– For each object (DQ_CSV_DQ)
• Select the minimum and maximum version for each object. If
there are three versions remove the minimum version
• Job should be run at then end of each daily extraction
 BOA Extraction Control Table

BOA_View Num_Rows Date Blocked

Varchar(50) Integer Date Char(1)
BF_022_PRODUCT_COST_EST_LEGAL 121 18/08/2014 N
BF_601_BOM_COSTING 134 18/08/2014 N
BF_601_BOM_COSTING_HEADER 12 18/08/2014 N
BF_601_BOM_COSTING_ITEM
FI_011_PRODUCT_COST_COLLECTOR
100 18/08/2014
16 18/08/2014
Y
N Day 2
FI_017_AP_OPENITEM 56 18/08/2014 N
FI_017_OPEN_ITEM 78 18/08/2014 N
FI_023_PROC_ALTRNATIV_STO_TRF 67 17/08/2014 N
FI_025_MIXING_RATIO
FI_025A_MIXING_RATIO
48 17/08/2014
23 17/08/2014
N
N Day 1
FI_025A_MIXING_RATIO_ST 18 17/08/2014 N
FI_025B_MIXING_RATIO_SUBCNTRCT 556 17/08/2014 N

Day 1: DataStage job reads table selecting all rows with maximum date
and processes BOA views that have a blocked flag or “N”
Day 2: DataStage jobs ready table selecting all rows with a maximum
date and processes BOA view that have a blocked flag of “N”

All views that have been processed are recorded in the Audit tables

CONFIDENTIAL – For planning purposes only. Subject to any applicable information and consultation requirements
Pre-Population
“CLT” Pre-Population from Legacy Systems to DSP

• Selection criteria used to acquire data from legacy systems and write results to DSP
SQL Server
• Selection criteria already implemented via SQL Views that DataStage can use for data
extraction
• Tables in SQL Server need to be created as a landing area for CLT pre-population
• Process needs to be created in DSP to read landing area data and load into appropriate
DSP Templates
• Technical fields in DSP landing area need to be agreed
• All GDC800’s and GDC200s are prepopulated
 SAP Configuration Table Integration Architecture
Red Text = New Objects Required

BOA RTL

2. BOA process 1. DataStage Job and

inserts or updates Job Sequence
DSP Server execute extraction Information Server
new or existing GDC
tables in the BOA DB. from legacy data in
staging using already
existing views for pre-
population selection
criteria
Pre-Pop Staging
BOA DB Landing
Area

SQL Server SQL Server Oracle

* Applies to all Oracle Staging

environments – SBC, SBC,
PBC and PBD
“CLT” Pre-Population from Legacy Systems to DSP Architecture

Architecture to be included
“CLT” Pre-Population from Legacy Systems to DSP Estimation

• Creation of SQL Server Database for landing area

• Data modeling of SQL Server landing area
• Pre-populated data posting to SQL Server
– Generic DataStage job to read SQL View using selection criteria and write to SQL
Server
– DataStage sequence to iterate by market and execute above DataStage job
– Control file for each Market
– There will be these three objects for each GDC

Level of Effort (per GDC)

5 Days for the first set of artifacts
3 Days for subsequent artifacts

Example: there are about six GDC26x. Using above estimate the
level effort would be 20 days to have all GDC26x pre-populationl
Other Technical Considerations

• View naming conventions

• Column/Table name lengths (Oracle can only be 30 characters)
• Information Server connectivity to SQL Server via Data Direct Wire
Protocol Drivers