How To Archive in BW (NW2004)

SAP BW Archiving and Data
Aging Strategies
Ron Silberstein Mike Eacrett

Platinum Consultants
SAP Netweaver RIG US – Business Intelligence
SAP Labs, LLC
Agenda
Information Lifecycle Management
Data Aging Strategies in a

Heterogeneous Application Environment
SAP ERP Components
SAP Business Information
Warehouse
Summary
© SAP AG 2003, ASUG BITI Dallas 2003 Ron Silberstein / Mike Eacrett/ 2
Agenda

SAP ERP Components
Warehouse
Summary
Increasing Volume of Databases
Meta Group
“70% of all enterprise data currently resides in

databases”
Gartner Group
“Multi-Terabyte databases are a reality today – and will

grow to hundreds of TB by 2006”
“Database applications – like SAP – are a key driver of

storage growth”
“…growing at 64% rate”
Distribution of Activity vs. Age of Data
Frequency
of access
reads
updates
Age of data
Distribution of the memory costs
“ Hard disk costs do not even represent a quarter of the memory costs “
(Giga Information Group)
Misc
(Purchasing,
training)
10%
Personel Hard disk

45% 23%
Storage-
Environment Mangement
(Electricity, Space)
(Soft- & Hardware)
3% 19%
“ Administrative expense for 1 Terabyte of memory are appropriate for
five to seven times more higher than the memory costs themselves “
(Dataquest/Gartner)
Data Lifecycle vs. Storage Media Types/Costs
Cost Performance
Access Frequency RAM
DASD
Virtual
high Tape
Access Centric
Tape
Capacity Centric
Tape
low
time
days months years
Online- Near-line Offline
Storage Storage Archive
How to combine all this in a Data Aging Scenario?
Online Offline/
Query
Near-line File System
Read
Database
Data Objects
Storage
Management
Application System
Data
Archive Files
Move
Save
Benefits of a sound Data Archiving Strategy
Value proposition
for deploying Data Archiving
1 Availability
2
Data availability: faster data
loads, rollups, change runs, etc.
System availability: Faster and Resource consumption
simpler software and release Reduction of the hardware costs
mgmt and upgrades. Reduced for hard disks, main memory and
backup & recovery times. CPU, and the costs of the
3
system administration
Performance
Faster Query response times.
Faster Data load times.
Agenda

SAP ERP Components
Warehouse
Summary
BW Architecture: Layers & accessibility
Operational Data Store Data Warehouse Multidimensional Model

Operational Reporting Non volatile Multidimensional analysis
Near Real-Time / Volatile Granular Aggregated view
Granular Historical foundation Integrated
Built with ODS Objects Integrated
Built with InfoCubes
Built with ODS
Objects
Transaction Data Processing into the BW ADM Layer
SAP BW InfoCube
Material dimension
Customer dimension
4712
InfoCubes A
4713
Amount
Customer Material Time Company Information
ODS ODS
Currency
Object
Objects A 4712 200211 100
A 4713 200211 150
BW Architected Data Mart Layer
Time dimension
other InfoCubes
other InfoCubes 200211
monthly weekly daily
SAP ERP daily Process

Data
Master Amount
Master data
data
Customer Time Doc No Pos Material Local
Currency
Documents
Documents A 20021107-10am 1 10 4711 100 - New booking
Change A 20021107-3pm 1 10 4712 200 - Correction booking
Change Docs
Docs
A 20021107-4pm 2 10 4713 300 - New booking
Transaction Data Processing with BW EDW Layer
SAP BW Customer dimension InfoCube Material dimension
A 4712
4713
InfoCubes Amount
Customer Material Time Company
Currency Information
A 4712 200211 100
BW Architected Data Mart Layer A 4713 200211 150
Time dimension
other InfoCubes
other InfoCubes 200211
monthly weekly daily

BW Enterprise Information
Amount Amount
Data Warehouse Base
Customer Time DocNo Pos Material Local Company
Layer Currency Currency
A 20021107-10am 1 10 4711 100 50
ODS A 20021107-3pm 1 10 4712 200 100
A 20021107-4pm 2 10 4713 300 150
ObjectODS
Objects
daily Process
SAP ERP
Amount Data
Customer Time DocNo Pos Material Local
Master
Master data
data Currency
A 20021107-10am 1 10 4711 100 - New booking
Documents
Documents A 20021107-3pm 1 10 4712 200 - Correction booking
A 20021107-4pm 2 10 4713 300 - New booking
Change
Change Docs
Docs
Impact of Information Base EDW Layer
Pros Cons
Designed for the future Increasing Data Volume
Foundation of data for Data redundancy
future development Increased data management
Easier delta load control and coordination
Reduce redundancy in very Increased DW administration
granular data Upfront design considerations
Greater control of data
distribution
Fine tuning of data availability
in InfoProviders
Archiving of base data possible
Bill Inmon’s: Enterprise Data Warehousing
SAP BW DSS Applications

Departmental Data Marts Powered by SAP NetWeaver
Acctg Finance
ERP
Marketing Sales
ERP
ERP
CRM
ETL
Staging Area
Changed
eComm.
Data
EDW Bus. Int.
Exploration
Global warehouse/
ODS data mining
ERP
Corporate Oper.
Applications
Mart cross media
local Granularity
Storage mgr
ODS Manager
Session Near line

Analysis
Storage
Dialogue
Manager
Cookie Archives
Cognition Web Logs
Preformatted
Internet dialogues Source:Bill Inmon
Agenda

SAP ERP Components
Warehouse
Summary
Data Archiving and Storage within ERP Environment
mySAP System External Storage System
ArchiveLink Storage System

Database
Data Objects File System HSM-

System
Application data
Alternative
Storage
SAP Third Parties (optional)
Archive Development Kit – ADK
Development tool for data archiving solutions

From SAP
Integration of archiving functionality into SAP applications
From customers
Archiving solutions for customer tables and enhancements
Application
data
Database mySAP
Applications
ADK Code page adjustments, alphanumeric format, structural

changes, compression, file handling, job control, …
Archive Files
The Archiving object
Definition of the data associated to logical work units
All programs that are necessary for archiving,

e.g. Testing , Reading, Writing and Delete programs
Customizing of data selection Archiving Object

and processing
Customizing Data
Program
Data reduction in the ERP field!
Is the data Yes Can Data be Yes Consolidate

still needed? consolidated? Data!
No No Yes
Access to Can the data be archived?

Productive
data switched
off!
No Yes
Data remains Archive

in the
Data!
database!
Accessing Archived Data
Database Search DB + Archive

Business processes Display work units
DRB Application
SAP System Archive files
AS
100%
ADK
Index / Archive info structure
Indexing Technical display for all Archive object

Search on archived data Display of Object lists
Archive Information System
Flexible creation of Archive data indices

Freely configurable info. structures are stored in transparent DB
tables
Manually built with selected archive files
Automatically built during the delete phase of a archiving run
Search of archived data
Technical display for all archiving objects
Display of object lists

Based on info. structures
Ad-hoc display of selected archive files
Creation of display variants
Generic tools for customer developments
Definition of Data Objects and Influence on Indexing
ODS Object Archive File Archive index
Key Data Object key

YEAR YEAR
DOCNO CUSTOMER
Data Object 1
YEAR DOCNO CUSTOMER AMOUNT YEAR CUSTOMER FILE OFFSET
YEAR DOCNO CUSTOMER AMOUNT 1995 10000450 4710 100.00 1995 4710 ... ...
... 1995 10000452 4710 200.00 1995 4711 ... ...
1995 10000450 4710 100.00 1995 10000455 4710 400.00 1995 4712 ... ...
1995 10000451 4711 250.00
1995 10000452 4710 200.00 Data Object 2
1995 10000453 4712 150.00 YEAR DOCNO CUSTOMER AMOUNT
1995 10000454 4711 300.00 1995 10000451 4711 250.00
1995 10000455 4710 400.00 1995 10000454 4711 300.00
...
Data Object 3
YEAR DOCNO CUSTOMER AMOUNT
1995 10000451 4712 150.00
...
Document Relationship Browser (DRB)
User overview
• Archive Explorer
• Transaction ALO1 Document
Relationship
• User role SAP_DRB
• Application transactions
Browser
Display
Database
Display of company process

data from the archive files and Application Infra- Archive
data structure files
the underlying DB structures
Agenda

SAP ERP Components
Warehouse
Summary
Typical Data Warehouse Problems
End-User Challenges
Making timely, informed business decisions
- Users cannot wait for historical data to be restored
- Transparent access to data for regular reporting and ad-hoc
analysis
IT Management Challenges
Meeting end-user data demand while managing cost
- High costs of adding/managing online disk storage
- High costs of backup and recovery – especially when data is
infrequently accessed
- Data protection and availability
Motivation for a Data Aging Strategy : Benefits
Costs
offline vs. online storage costs
System usage overhead – CPU, Memory, etc
Control of system growth
System Availability vs. costs
Data availability – faster rollups, change runs, etc
System availability – less downtime for backups, upgrades,
etc
Performance vs. costs
Faster load times
Faster query times
See also "Scalability with SAP® Business Information Warehouse“
at http://www.sap.com/solutions/bi/brochures/
Legal Requirements
SAP BW Customer motivation to archive
Customer requirements (derived from ASUG):

Data objects relevant for archiving
InfoCubes
ODS Objects
PSA
Master data
Functionality
Both Archiving and Data Deletion (without
archiving)
Select data based on any criteria
Automatically scheduled on a periodic basis
Restoring of archived data
Data retention time

3 to 5 years in InfoCube and ODS Objects
Consistent archiving processes

How to avoid high Data Volumes in a DW Environment?
Can the
Is Data Yes Data be Yes Aggregate
still needed? aggregated? Data!
Maybe No Yes
Can the data be archived?
No Yes
We need to address Data Archive

the no longer needed remains in the
Data!
AND the maybe scenario! database!
Data Aging Strategy Implementation
Data aging is a strategy for managing data over time, balancing

data access requirements with TCO.
Each data aging Strategy is uniquely determined by the customer’s
data and the business value of accessing the data.
Which tools should I consider to use?
Online Database Near line BW data

Storage Storage archiving
Frequently read
/updated data
Infrequently
read data
Very rarely
read data
Where is Archiving and Near-line Storage applicable?
Access frequency
Archiving (BW 3.X)

Decreasing TCO For analysis, archived data must be
reloaded first again into the BW
reads
database
Archiving with SAP BW
Reduction in costs of data retention
on alternative media
Data age
Access frequency
Decreasing TCO
Near-line Storage (BW 4.X*)
Direct accesses to data to alternative
reads
storage media for Queries
Near Line Storage Performance and data retention costs
for access aged data can be minimized
Data age * Pilot project possible for BW 3.X

SAP BW Archiving Object Architecture
activates
BW Repository
generates
Archiving Object
Archive
DataManager
reads Administration
InfoCube Write (SARA)
deletes Schedules
Delete
ODS
Read Datamart
Object Extractor
ADK
File system, CMS, HSM
Technical Architecture: BW and ADK Integration
File system External Storage Systems

BW database
InfoCubes Archiving Object

• Optical
Data Objects
Storage system
Archive
Files
• HSM
System
ODS Archiving Object
object
Data Objects
• Storage-
system
Archive
Files
SAP Functionality - ADK 3rd party products

SAP BW data archiving technology : Write process
Time Slot archiving is a powerful tool for selection of time

dependent transactional archive data.
ODS Object
Current Date: Mar 2003
Sep 2002 Archive file
Oct 2002 Sep 2002

Nov 2002 Oct 2002
Dec 2002 Time slot archiving: Dec 2002
Archive complete Years 2002 & 2003
Jan 2003
Only complete fiscal years
Feb 2003 Exclude Nov 2002 & Mar 2002
Mar 2003 Protect archive areas
Pros: Complex time selection options

Cons: Limited to time selections only
New Data Dec 2002 Mar 2003

loads
SAP BW data archiving technology : Write process
Field Data selection archiving is a very powerful tool for free

selection of transactional archive data based on any criteria:
ODS Object
Current Date: Mar 2003
comp 10 Oct 2002 Archive file
comp 10 Nov 2002 comp 10 Oct 2002

comp 20 Nov 2002 comp 10 Nov 2002
comp 10 Dec 2002 Data selection archiving: comp 10 Dec 2002
comp 20 Jan 2003 Archive Years 2002 & 2003
Company 10 only
comp 20 Feb 2003
comp 20 Mar 2003
Pros: Flexible data selection

Cons:Time selections more complex
No protected archiving areas
New Data Comp 10 Dec 2002 Comp 20 Mar 2003

loads
SAP BW data archiving technology: Extraction/Reloading
BW Extractors and Export

DataSources are ‘Archive enabled’
InfoCube
ODS object InfoPackage is extended by an
option 'archive selection'
Selection options for available
Update Rules archive sessions and files
Only full extraction supported
InfoPackage Archive files are scanned with
selection criteria of the request
Export DataSource
Reload to original DataTarget is
possible but not recommended
Reload Recommendation:
InfoCube Archive Files
Extract to a copy of original
DataTarget instead
ODS object Use MultiProvider to combine
remaining data with reloaded data
Why would we consider Near-line storage?
Reasons for Direct Query Access to Archive Data
Business-driven reasons
Introducing new characteristics with a historical background
Strategic Analysis of data over long periods
Just because it’s our data! ☺
…
Legal reasons
Regulatory and industry specific requirements
Data is immediate accessible from a legal and technical perspective
Example: GDPdU demands a ten-year period for retention for tax
relevant data and…
the data’s immediate legibility and machine evaluation within the entire
period for retention (§§ 146 u. 147 AO)
Example: Food & Drug Administration (FDA)
…
Near-line Storage Solution for SAP BW (Key points)
Separating frequently used (kept in database) and infrequently used data

(stored in Near-line Storage)
Supporting both InfoCubes and ODS Objects
Transparent access to "non-archived" and "archived" data for queries
Hierarchical Storage Management (depending on the provider)

First Level: BW Database
Second Level + further Levels: Near-line Storage
Intelligent Data Access

Data Selection Analysis and Feedback
High level index in BW DB
Low level index in Near-line Storage
Openness
StorHouse® / FileTek, CBW® / PBS Software,
DiskXtender® for BW / Legato, …
Overview of SAP BW 3.X “Project” Solution
Query Query
Proprietary Interfaces
to select
MultiProvider 3rd Party Providers
No Intelligence
∪
Virtual InfoCube
Basis Cube or
ODS Object Copy Near-line Storage
Adapter
Near-line Storage Partner Solution
BW Database Optical
Robotic Jukeboxes
Tape
Libraries
High speed disk NAS or
Low-Cost Disk
SAP BW 3.X “Project” Solution – Sample screen shot
3rd Party
Options
SAP BW 4.x Open Solution (Planned)
Data Flow
Query Control Flow
InfoCube/ODS Object with Near-line Services
Union
High level index
Split and Dispatch
Archive/
BW DB Interface Restore Near-line Storage Adapter
BW Data Base Near-line Storage Partner Solution
Optical
Robotic Jukeboxes
Tape
Libraries
NAS or
High speed disk Low level index Low-Cost Disk
Data Manager
SAP BW Near-line Storage technology
A data view of BW and Near-line Storage solution integration

ODS Object
Relocate complete Near-line DB table
Sep 2002 Year 2002
Oct 2002 Sep 2002
Nov 2002 Oct 2002
Dec 2002 Nov 2002
Jan 2003 FTP Dec 2002
Feb 2003
ODS Object
Mar 2003
Multiprovider - Provides consistent view of data

Dec 2002
New Data Mar 2003

loads
BW Queries
Agenda

SAP ERP Components
Warehouse
Summary
What we plan to deliver in the future …
Operational Data Store

Non-SAP
Source
Persistent Inform-
Staging ation
Area Arch- Access
Data itected
SAP Warehouse Data
Source Marts
New!
Archiving & Near-Line Storage
What about your Data Volume growth?
700.00
Expected size
without Archiving
600.00
Allocated DB size
500.00
400.00
Allocated DB content
300.00
'Without' Initial
Archiving Archiving With regular archiving
200.00
100.00
DB growth: Reduction: DB growth: ~7 GB/month
~15 GB/month ~60GB
0.00
01.09.2002
01.11.2002
01.01.2003
01.04.2003
01.07.2003
01.09.2003
01.10.2002
01.12.2002
01.02.2003
01.03.2003
01.08.2003
01.06.2002
01.07.2002
01.08.2002
01.03.2002
01.04.2002
01.05.2002
01.05.2003
01.06.2003
The right time to start ....
1 The “healthy“ System

Data Archiving shouldn’t be the
final step to prevent a system
2
going into cardiac arrest!
Early Planning
Proactively maintaining and
sustaining performance in
the system
3 Interdisciplinary Process
Data Archiving makes a high level of
coordination necessary between IT (technical)
and Application (functional) groups
Further Information
Public Web:
www.sap.com Services Rapid ROI and Low TCO
SAP Customer Services Network: www.sap.com/services/
Consulting Contact
Roy Wood, VP SAP NetWeaver Consulting Practice (R.wood@sap.com)
SAP BW 3.x Near-line Pilots contacts

Americas – Tony Andreacchio (Anthony.andreacchio@SAP.com)
EMEA – Rainer Uhle (Rainer.uhle@sap.com)
Related SAP Education Training Opportunities
http://www.sap.com/usa/education/
BW360, SAP BW Performance and Administration
Questions?
Feedback
Please complete your session evaluation

and drop it in the box on your way out.

How To Archive in BW (NW2004)

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

How To Archive in BW (NW2004)

Caricato da

Copyright:

Formati disponibili

SAP BW Archiving and Data

Ron Silberstein Mike Eacrett

Information Lifecycle Management

Information Lifecycle Management

Data Aging Strategies in a

Information Lifecycle Management

Information Lifecycle Management

Data Aging Strategies in a

“70% of all enterprise data currently resides in

“Multi-Terabyte databases are a reality today – and will

“Database applications – like SAP – are a key driver of

“…growing at 64% rate”

Personel Hard disk

Information Lifecycle Management

Information Lifecycle Management

Data Aging Strategies in a

Operational Data Store Data Warehouse Multidimensional Model

monthly weekly daily

SAP ERP daily Process

monthly weekly daily

SAP BW DSS Applications

Session Near line

Information Lifecycle Management

Information Lifecycle Management

Data Aging Strategies in a

mySAP System External Storage System

ArchiveLink Storage System

Data Objects File System HSM-

SAP Third Parties (optional)

Development tool for data archiving solutions

ADK Code page adjustments, alphanumeric format, structural

Definition of the data associated to logical work units

All programs that are necessary for archiving,

Customizing of data selection Archiving Object

Is the data Yes Can Data be Yes Consolidate

Access to Can the data be archived?

Data remains Archive

Database Search DB + Archive

Index / Archive info structure

Indexing Technical display for all Archive object

Flexible creation of Archive data indices

Search of archived data

Technical display for all archiving objects

Display of object lists

Generic tools for customer developments

ODS Object Archive File Archive index

Key Data Object key

Display of company process

Information Lifecycle Management

Information Lifecycle Management

Data Aging Strategies in a

Customer requirements (derived from ASUG):

Data retention time

Consistent archiving processes

Can the data be archived?

We need to address Data Archive

Data aging is a strategy for managing data over time, balancing

Which tools should I consider to use?

Online Database Near line BW data

Archiving (BW 3.X)

Data age * Pilot project possible for BW 3.X

File system, CMS, HSM

File system External Storage Systems