Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
1 System Operations
5 System Management
6 High Availability
3 Performance Management
7 Printing
4 Database Operations
8 Change Management
SAP AG 1999
Database Operations
Purpose
Verify the backup/recovery procedure Verify the disaster recovery procedure Verify database performance management Verify database monitoring procedure Verify growing management procedures
Key Issues
Backup/recovery concept Disaster recovery concept File distribution Database performance Database monitoring Growing management
SAP AG 1999
Indicator Avg. DB Request Time in Dialog Task Avg. DB Request Time in Update Task Reads / User call SQL Statements with inefficient search strategy SQL Statements with high data transfer volume DB Size Last Month DB Growth Top growing tables
Value
Target
SAP AG 1999
Is backup strategy appropriate ? WHAT? Are emergency procedures defined ? Are administrative tasks for regular operation defined ?
Database administrator Operating system administrator WHO? System administrator Technical team lead/Technical consultant Network administrator
SAP AG 1999
Volume of data backed up? Backup duration? Backup cycle length? Costs? Administrator skills
SAP AG 1999
Recover database
SAP AG 1999
Monitoring Tasks
Free space
Database logs
SAP AG 1999
SAP AG 1999
Unit Content
Backup Strategy and Tape Management Scheduling, Performing, and Monitoring Backups Restore and Recovery Advanced Backup Techniques Storage Management
SAP AG 1999
Logical errors
(Such as a deleted table) DROP MARA
Data loss
A good database backup strategy prevents data loss and minimizes system downtime
Procedure and Escalation Plan
SAP AG 1999
Database backup
SAP AG 1999
Lost information
12:50 - 4:00
You must recover the complete database to a point in time before the error
SAP AG 1999
Lost information
Time
If one offline redo log file is lost, none of the files that follow it can be used
SAP AG 1999
External factors
(Such as fire or water damage)
SAP AG 1999
Verify the database Verify the backup Offline Verify the backup Online Offline redo log file backup (x2)
28 days
Additional
SAP AG 1999
Detail log
Detail log
Media
Media
SAP AG 1999
Backup Objects
Objects that need to be backed up
Database objects
Data files
BRBACKUP
Database backup Database backup
Online redo log files
BRRESTORE
Control file Profiles Offline redo log files
BRARCHIVE
Offline redo log Offline redo log file backup file backup
R/3 data
R/3 interfaces SAP executables
SAP AG 1999
Tape Pools
Frequency of backups
SAP AG 1999
Tape Initialization
Profile init<SID>.sap contains the tape names: ... volume_backup = (<SID>B01,<SID>B02... volume_archive = (<SID>A01,<SID>A02... ...
Write the label to the tape that also contains the tape name
.tape.hdr0
Initialize new tapes, non-SAP tapes, or locked tapes: brbackup -i force or brarchive -i force Rename non-locked tapes: brbackup -i -v <tape name> or brarchive -i -v <tape name>
SAP AG 1999
Tape name Database name Timestamp of last backup Number of backups performed
Tape checks:
Profile init<SID>.sap:
Error Warning
= 28 = 100
Tape Locking
Physical tape locking
C11B01
C11B02
expir_period = 30 expir_period = 30
.tape.hdr0
28 days
Days 1 2 28 29 30 31 32 33
Profile: init<SID>.sap
... volume_backup = (C11... volume_archive = (C11... ...
SAP AG 1999
Offline
28 days
Additional
BRBACKUP BRARCHIVE
Tables SDBAH C11B01 and SDBAD C11A01
Tape management active, BRBACKUP finds tape names + Tape label check active = Only accepts tapes with requested names and with expired lock period
SAP AG 1999
Offline
28 days
Additional
BRBACKUP BRARCHIVE
C11B01 C11A01
SAP AG 1999
Offline
28 days
Additional
brbackup -v C11M01,C11M02
BRBACKUP BRARCHIVE
C11B01 Search mechanism
SAP AG 1999
C11A01
Tape management is deactivated but tape name is checked + Tape label check active = Only accepts tapes - With the specified names - With expired lock period
Tape Layout
BRBACKUP:
.tape.hdr0 init<sid>.ora init<sid>.sap DB file 1 init<sid>.dba
DB file n
Control file
reorg.log struct.log
Detail log
Summary log
BRARCHIVE:
.tape.hdr0 init<sid>.ora Offline init<sid>.sap init<sid>.dba redo log 1
Summary log
SAP AG 1999
BRBACKUP
Data files
Log files
.... .... .... .... .... .... .... .... ....
init<SID>.sap
.... .... .... ....
sapr3.SDBAH sapr3.SDBAD
BRARCHIVE
SAP AG 1999
init<SID>.sap
.... .... .... .... ? ??? ? ?
SAP AG 1999
Correct
BRBACKUP/BRARCHIVE BRBACKUP: SAP follow-up tape
Incorrect
BRBACKUP/BRARCHIVE dd: Error
...
cpio/dd cpio/dd
...
cpio/dd
Data
... Data
Data
Data
... Data
Data
Hardware Compression
Tape station without hardware compression Tape station with hardware compression
BRBACKUP
400 MB
BRBACKUP
400 MB
400 MB
400 MB 400 MB 400 MB 400 MB
200 MB
200 200 200 200 200 200 200 200 MB MB MB MB MB MB MB MB init<SID>.sap compress = hardware .... .... .... compress_cmd = compress -b 12 -c $ > $ $ ....
init<SID>.sap
exec_parallel tape_size
= 0 = 1600M
18:00:00 Calendar
Full database offline + redo log backup Full database offline backup Full database online + redo log backup Full database online backup Redo log backup Partial database offline backup Partial database online backup Check optimizer statistics Adapt next extents
h - Backup database
SAPDBA
c - Backup device type d - Objects for backup e - Backup type g - Query only S - Start BRBACKUP
init<SID>.sap .... .... .... ....
Command prompt
-m all -c -u system
SAP AG 1999
Online
Offline
Retrieve file names of data and online redo log files from database Retrieve file names of data and online redo log files from database and retrieve names of control files from init<SID>.ora and retrieve names of control files from init<SID>.ora Shut down database Shut down database Back up tape header, init<SID>.sap, init<SID>.dba, and init<SID>.ora Back up tape header, init<SID>.sap, init<SID>.dba, and init<SID>.ora For all tablespaces to be backed up: For all tablespaces to be backed up: Begin tablespace backup mode Begin tablespace backup mode Back up tablespace data files Back up tablespace data files End tablespace backup mode End tablespace backup mode Back up saved control file Back up saved control file Log file switch Log file switch Back up data files Back up data files *Back up online redo log files *Back up online redo log files Back up control file Back up control file Start database Start database
Back up reorg.log, struct.log, detail log, and summary log Back up reorg.log, struct.log, detail log, and summary log
init<SID>.sap .... .... .... ....
SAP AG 1999
Data files
A B
..
File length
? =
Log files
.... .... .... .... .... .... .... .... ....
compress_dir
SAP AG 1999
Tape readable?
A B
..
Offline (binary)
? =
compress_dir
...
CCMS
SAPDBA
L - Show/Cleanup SAPDBA
x x x
Log files
.... .... .... .... .... .... .... .... ....
SAPDBA
e - BRBACKUP log files
Command prompt
SAP AG 1999
BRARCHIVE -cds
Mon
42 4 2 4 2 4 2
Tue
42
Wed
42 <SID>A01 43
ARCHIVED
43 43
42 43 44 4 5 4 6 <SID>A03 44 45 46
42
../saparch
../saparch
SAP AG 1999
18:00:00 Calendar
DB13
Full database offline + redo log backup Full database offline backup Full database online + redo log backup Full database online backup Redo log backup Partial database offline backup Partial database online backup Check optimizer statistics Adapt next extents
x x
SAPDBA
Copy, delete, and save archive logs tape
x s - Start BRARCHIVE
SAP AG 1999
DB13
SAPDBA
L - Show / Cleanup SAPDBA
x x x
Log files
.... .... .... .... .... .... .... .... ....
SAPDBA
f - BRARCHIVE log files
Command prompt
SAP AG 1999
SAPDBA
L - Show/Cleanup SAPDBA
x x
b - Cleanup log files / directories x SAPDBA a - SAPDBA log files and dump directories b - SAPDBA daily check log files c - BRBACKUP log files d - BRARCHIVE log files f - ORACLE traces and audits
= = = = =
11 5 30 30 1
Log files
.... .... .... .... .... .... .... .... ....
sapr3.SDBAH sapr3.SDBAD
Command prompt
delete b<timestamp>.and
SAP AG 1999
RZ20
-
alert_<SID>.log: Thread <n> cannot allocate new log, All online logs needed archiving
DB24
Resolving an archiver stuck
x x x
SAPDBA
f - Archive mode SAPDBA
Dummy
and
Run BRARCHIVE
Command prompt
x
cd saparch df-k.
../saparch
<SID>A01
SAP AG 1999
Database Errors
Most importantly, be prepared for disasters. Dont think you will never see a failure. Every DBA will experience a database failure. Its just a matter of when... Good luck. Rama Velpuri
SAP AG 1999
Do not make any rash decisions Analyze the problem in detail Create a problem-solving strategy Before restoring any files, check: What is causing the problem Whether there is enough disk space to save and restore files Whether a hardware extension is necessary The file system and mount points The availability of backups The availability of offline redo log files
SAP AG 1999
Scenario
10 10
10
3 38 38 8
Control files
10
11
12
...
36
37
3 8
10 1 0 10 38 38
38
Data files
Full backup
Legend: Log sequence number
SAP AG 1999
mount
open
38 38
38 38
38
38
Control files
10
11
...
37
3 8
39
...
38 10 10
Complete recovery
38 38
38
Data files
Partial restore
SAP AG 1999
Database Reset
open
10 10
10
Control files
10 9 10
11 11
... ...
37
38
10 10 10
Data files
Full restore
SAP AG 1999
mount
Open resetlogs
10 10
10 25
25
25 1
Control files
26 10 11
...
2 24
37
25 1
...
10 10 10 25 25
25
24 1 1
Data files
Full restore
SAP AG 1999
Detail logs
back<SID>.log
arch<SID>.log
Recovery script
Check Database
Find backups
Recover database
saparch
Database
SAP AG 1999
Detail logs
back<SID>.log
arch<SID>.log
Recovery script
Check Database
Find backups
Recover database
saparch
Database
SAP AG 1999
Problem
No data or offline redo log file backups available
Solutions
Perform a database reset Perform a point in time recovery Use the SAPDBA function Restore individual files
Logs
init<SID>.*
Restore these files from tape using command brrestore -n init_ora Copy one of its mirrors
SAP AG 1999
Detail logs
back<SID>.log
Overwrite all data files, control files, and online redo log files
Database mount
Database
SAP AG 1999
Overwrite all Data files and Offline control files redo log files
sapreorg
saparch
Database
SAP AG 1999
Detail logs
back<SID>.log
Input: time
arch<SID>.log
reorg<SID>.log
Recover until?
Status: allowed?
Not allowed if (for example): - No backup specified - No offline redo log files found - Recovery over tablespace reorg - Backup before open resetlogs
Database
SAP AG 1999
Recovery script
Overwrite all Data files and control files Offline redo log files (if necessary)
sapreorg
saparch
Database
SAP AG 1999
Duration of a backup Time window for a recovery High availability Training Administrative workload Acquisition costs
SAP AG 1999
Data files
...
...
Logs
...begin backup
...end backup
10 10
10 25
25
25
Control files
10
11
...
24
25
...
10 10 10
25
...
25
25
Data files
brbackup -t online_cons
init files
10
10
...
25
10
...
25
Logs
9
Control files
init<SID>.sap
exec_parallel tape_address archive_function tape_address_arch =0 = (dev1, dev2, dev3) = double_save_delete = (dev4, dev5)
BRBACKUP
BRARCHIVE
DDS DLT
DLT
SAP AG 1999
DLT
DDS
Mon
Tues
Sun
...
Data files
SAP AG 1999
Data tablespaces
brbackup -m all_data
SAP AG 1999
Data files
dir2
dir1
init<SID>.sap
exec_parallel tape_address backup_root_dir =0 = (dev1, dev2, dev3) = (lvol1, lvol2)
SAP AG 1999
$ORACLE_HOME
dbs sapdata1
. . .
NEW_DB_HOME
dbs btabd_1 sapdata1
. . .
btabd_1
sapdata<n> origlogA
brbackup -d disk_copy
origlogB
init<SID>.sap
new_db_home = /oracle/NEW
SAP AG 1999
Production server
Backup server
o irr
SAP AG 1999
Standby server
saparch
saparch
Recovery
brarchive -sd
-d disk -f -w
brarchive -ssd -f -m 60
brbackup -t offline_standby
SAP AG 1999
init<SID>.sap
backup_dev_type util_par_file = util_file_online = init<SID>.utl
Restore
init<SID>.utl
btabd_1
sapdata<n> saparch
SAP AG 1999
Datafile2
Segment 192K Segment 640K
... ...
Segment 192K
Extent 48K 1
8K 8K 8K 8K
Segment 224K
...
Extent 144K
8K 8K 2 8K 8K 8K 8K 8K 8K 8K 8K 8K 8K 8K 8K 8K 8K 8K 8K
Next Extent
Initial Extent
8K 8K
Disks
File system
...
Data block
Datafile1
Datafile2
SAP AG 1999
Objects (tables/indexes)
Tablespace
<tablespace>.data1
2
1 2 4 2 0 2 3 4 1 0 1 5 0 3
External fragmentation
4 3
5 1
Gaps
Used Extents
SAP AG 1999
18:00:00
1
Calendar
FRI
Full database offline + redo log backup Full database offline backup Full database online + redo log backup Full database online backup Redo log backup Partial database offline backup Partial database online backup
FRI
FRI
<Timestamp.chk> in /oracle/<SID>/sapcheck
sapdba -check
6 5 11
10
seconds
inaktiv aktiv
DB16
W E E E
10 10 10 10
67 19 28 13 7
DB17
Active Inaktiv 66 1
O... Actv. S.. Op... Val. U... Per... Unit E > E E W > E E E
Type Parameter
Date
U... C CorrMeasure P E SAPDBA: BACKUP ARCH... INIT<SID>.ORA SAPDBA: TABLESPACE A... DO A CREATE CONTROL...
Description ARCHIVE STUCK - FS SPAC... CONTROL FILES ARE MISSI... CONTROL FILE(S) ARE NOT... SEGMENT(S) #1 WOULD CAU... DATAFILE(S) #1 OFFLINE FILE TYPE DOES NOT MATC... FILE TYPE ( DATAFILE , RA...
80
SAPDBA: RESTORE/RECO... DATA FILES ARE MISSING EXTEND FILESYSTEM OR... # 1 FILE SYSTEM(S) # 2 ARE...
Error
if
80
Percentage
Time Unit
Configuration table
DBCHECKORA
SAPDBA: BACKUP ARCHIVE LOGS
Operation
Date
SAP AG 1999
Tablespace Extension
ADD A DATA FILE: File size depends on the estimated increase of the tablespace objects. Check for the number of data files in the database. RESIZE THE DATA FILE: Extend the size of the data file depending on the space available on the file system and size of critical object.
New file
<tablespace>.data1
Critical object
<tablespace>.data2
<tablespace>.data1
Critical object
OR
Extents
Original size
After Resize
SAP AG 1999
Logical storage parameters Data class APPL1 Size category 4 Transaction data. transparent tables Data records expected: 39.000 to 3.100.000
x
WED THU FRI SAT SUN
sapdba -next
2621440 1310720
sapdba -next
New value for NEXT: 163840 Next larger TGORA value: NEXT candidate: Next smaller TGORA value: Current NEXT: Technical settings: Category 3:
655360
Start time
Period (weeks):
18:00:00
1
THU
Calendar
FRI SAT
sapdba -next
SUN
327680
MON
TUE
Full database offline + redo log backup Full database offline backup
WED
Full database online + redo log backup Full database online backup Redo log backup Partial database offline backup Partial database online backup Check optimizer statistics
sapdba -next
2560
SAP AG 1999
Database System
Total number 27 Total size/kb 12.123.016 Total free/kb 3.050.320 Minimum free/kb 4.024 Max. autoextensible/kb AutoExtend off
Tables and indexes
25%
Total number Total size/kb More than 1 extent Missing in database Missing in R/3 DDIC Space-critical objects
Tables Indexes 13.064 15.305 5.915.664 2.979.432 1.082 1.591 0 1 0 0 0 0 Maintain table: TCOLL
Detailed analysis Detailed analysis Missing indexes Missing indexes Space critical objects Space critical objects Space statistics Space statistics
SAP AG 1999
Tables/Indexes Total Chg/day 6.619 0 6.751 0 4.404 0 3.325 0 15 0 2.344 0 3.222 0 47 0 54 0 0 0 4 0 114 0
Extents Total Chg/day 8.692 22 9.296 21 6.156 10 4.307 8 120 3 2.471 1 3.542 1 49 0 60 0 0 0 4 0 131 0
SAP AG 1999
Detail log: 9908221023.aly *********************************************************************** (12690 tables analyzed - sorted by empty space in descending order) TABLESPACE_NAME TABLE_NAME EMPTY(kb) NEVER_BEEN_USED(kb) USED(kb)
Pct_free.............. Transactions initial.. maximum.. Header minimum....byte Data maximum......byte Process freelists..... Freelist groups.......
SAP AG 1999
Reorganization: Basics
2 1 0
2 1 0
2 1 0
2 1 0
2) Recreate index sapreorg 2) Drop table Additional storage space required PSAPTEMP
Data_1 Data_1
0 2 1 0
0 2
1 0
Reorganization: Reasons
0 Oracle block
Disk_1
2 0 2 0 2 0 2 0 3 4 1 0 5 1 0 2 3 4 1 0 3 4 1 0 5 1 3 0 2 0 5 1 0 2 3 4 1 0 2 3 4 1 0 3 2 0 5 1 3 0 2 0 5 1 0 2 3 4 1 0
1 2 3 4 5 ...
Disk_2
3 2
Disk_3
3 0 5 1 3 0 2 2 2
Disk_4
3 0 3 4 1 0 0 2 3 5 1 3 4 1 0 5 1
Fragmented Indexes
Data file_1
2
Data file_1
Data file_2
0 2 1 0 2 1 0 2 1 0
Disk x
0 1 2 0 2 0 3 4 1 0 5 1 0 4 3
Disk y
0 1 2 0 2 3 4 1 0 5 1 6 1 8 4 3 0
Data file_new
2 5 1 0 0 0 11 5 0
4 1 0 10 7 4
Data file_2
9
Reorganization: Methods
Reorganization of indexes Reorganization of indexes Method Runtime Security Additional space Method Runtime Security Additional space Parallel Parallel Alter index ++ ++ Objects tablespace II Alter index ++ ++ Objects tablespace rebuild PSAPTEMP rebuild PSAPTEMP Index Index recreate recreate ++ ++ PSAPTEMP PSAPTEMP II
Reorganization of tables Reorganization of tables Method Runtime Security Additional space Method Runtime Security Additional space R3Chop Parallel Compress Restrictions R3Chop Parallel Compress Restrictions Create table ++ ++ Objects tablespace TT NA No tables with Create table ++ ++ Objects tablespace NA NA NA No tables with as select PSAPROLL LONG/RAW fields as select PSAPROLL LONG/RAW fields SAPDBA unload. ++ sapreorg NO P NO export <<max_file_size SAPDBA unload. ++ sapreorg NO P NO export max_file_size SQL*loader PSAPTEMP (usually 22Gbyte) SQL*loader PSAPTEMP (usually Gbyte) Oracle Oracle export/import export/import oo ++ sapreorg sapreorg PSAPTEMP PSAPTEMP YES YES P P YES YES
Legend: Legend: ++ = very good, + = good, o = average, NA = Not applicable ++ = very good, + = good, o = average, NA = Not applicable II = Parallel on index, T = Parallel on table. (Oracle PARALLEL clause) = Parallel on index, T = Parallel on table. (Oracle PARALLEL clause) P = Parallel on process level (SAPDBA creates several processes) P = Parallel on process level (SAPDBA creates several processes)
SAP AG 1999
Reorganization: Options
Freespace 0
1 2 0
4 3
1 2 0
4 3
sapreorg
4 GB
Database
Process 1 Process 2
sapreorg1
dump1
sapreorg2
dump2
1 GB
2 GB
sapreorg
SAP AG 1999
sapreorg
Subunit 1 Subunit 2 Subunit 3 Subunit 4 Subunit 5 Subunit 6 Subunit 7 Subunit 8 Subunit 9 Subunit 10
Oracle Transactions I/O Tuning and Disk Configuration Database File Properties I/O Tuning: Buffer Cache Tuning the Oracle Database Writer Tuning the Oracle Log Writer Tuning Archiving Tuning the Shared Pool Tuning Rollback Segments Tuning the Temporary Tablespace
SAP AG 1999
Oracle database
SQL net
Shadow process
SAP AG 1999
Oracle database
SQL net
Shadow process
C
Shared Pool Insert into mara (.. Update mara set ... Select * from ...
SAP AG 1999
Data files
2
SAP AG 1999
1. Read data from the data file 2. Read a block from the rollback segment
Data files
Rollback Segments
SAP AG 1999
1. Read data from the data file 2. Read a block from the rollback segment 3. Write REDO information to change the rollback block
Data files
Rollback segments
SAP AG 1999
1. Read data from the data file 2. Read a block from the rollback segment 3. Write REDO information to change the rollback block 4. Copy the old data block value to the rollback data block
1
4
Rollback Data Block
Data files
Rollback segments
SAP AG 1999
1. Read data from the data file 2. Read a block from the rollback segment 3. Write REDO information to change the rollback block 4. Copy the old data block value to the rollback data block 5. Write REDO information to change the data block
4
Rollback Data Block
Data files
Rollback Segments
SAP AG 1999
1. Read the data from the data file 2. Read a block from the rollback segment 3. Write REDO information to change the rollback block 4. Copy the old data block value to the rollback data block 5. Write REDO information to change the data block 6. Change the data block
4
Rollback Data Block
Data files
5 5
Rollback Segments
SAP AG 1999
Data files
5 5
Rollback segments
SAP AG 1999
LGWR DBWR
Shadow process
ARCH CKPT
SAP AG 1999
ON
SAP AG 1999
COMMIT Change
B C
6 4 1 2 3 5
COMMIT
SAP AG 1999
IT M C O M
LGWR
OrigLog
MirrorLog
SAP AG 1999
Acknowledge Transaction
LGWR
SAP AG 1999
Data files
Rollback segments
SAP AG 1999
Checkpoint CHKPT
Database Buffer Pool
Changed Data Block
Data files
Control files
Data files
5 5
SAP AG 1999
1. Roll forward: Apply redo until all records from last completed checkpoint have been applied
Data files
Rollback Segments
2. Rollback: Rollback segments are used to undo the effects of uncommitted transactions applied by the roll forward phase (this generates redo and archives)
Data files
Rollback Segments
SAP AG 1999
SAP AG 1999
Several I/O subsystems for SAP R/3 are possible. The most important ones are RAID Write back caching Read/Write cached I/O subsystem
SAP AG 1999
RAID Levels
Advantages Striping Low cost Good performance Reliability Good performance with striping Reliability Low cost Striping
RAID 1
Expensive
RAID 5
SAP AG 1999
Disk Striping
Datafile Data file
Disk
Data file
Datafile Disk 1
Disk 2
SAP AG 1999
RAID 5 System
Data block 1 Data block 2 Data block 3 Data block 4 Data block 5
P 4K
4K P
4K 4K P
4K 2K
4K 2K
P P 1 2 3 4 5
SAP AG 1999
Using RAID
RAID
Type of RAID
Control file
Avoid
Database file
Avoid
NONE
Shadowing
Recommended
OK
Recommended
OK
OK
OK
OK
Avoid
Avoid
SAP AG 1999
Type of caching
OK (1)
OK (1)
OK (1)
OK (1)
SAP AG 1999
Cache director
500 M
B/s
Cache
Disk director
SAP AG 1999
Each Oracle database file has specific properties The typical I/O load to a file depends on its usage within the database The properties of a database file must be considered when it is placed in a file system
SAP AG 1999
he es r t fil fo g d 5 o Lo ai ed R R No ne li On
SAP AG 1999
LGWR
MirrLogA (# 1,3,5,7,9)
Disk 3/4
ARCH
LGWR
OrigLog B MirrLogA
Disk 3/4
LGWR
MirrLogB (# 0,2,4,6,8)
ARCH
Disk 7/8
SAP AG 1999
LGWR
MirrLogA (# 1,3,5,7,9)
Disk 5 Disk 2
Disk 7
+
Disk 6
+
Disk 8
OrigLogB (# 0,2,4,6,8)
LGWR
MirrLogB (# 0,2,4,6,8)
Disk 3
MirrLogD (# 0,2,4,6,8) MirrLogF (# 0,2,4,6,8)
Disk 4
SAP AG 1999
5 id Ra No
SAP AG 1999
Rollback Segments
5 id Ra No
SAP AG 1999
Data Files
Data files
SAP AG 1999
Temporary Tablespace
PSAPTEMP
5 id Ra No
SAP AG 1999
Control Files
SAP AG 1999
Disk striping or RAID 5 RAID 1 + Striping RAID 1 + Striping RAID 1 + Striping RAID 0* or 1 + Striping
Temporary tablespace (PSAPTEMP) Rollback segments (PSAPROLL) REDO log group 1 and mirror log (OrigLogA, MirrorLogA) REDO log group 2 and mirror log (OrigLogB, MirrorLogB)
RAID 1 + Striping
SAP AG 1999
Oracle DB_Block_Buffer
. A B C
DB_BLOCK_BUFFERS
B C
Buffer Cache as defined in init<SID>.ora with DB_BLOCK_BUFFERS = 1250000 Buffer_POOL_Keep= (BUFFERS:560000,LRU_LATCHES:8) Buffer_POOL_Recycle= (BUFFERS:250000,LRU_LATCHES:24)
SAP AG 1999
SAP AG 1999
The database buffer quality percentage is not sufficient to make decisions about the size of the buffer Before increasing the DB Block Buffer Check the available memory resources on the database server Check the executed SQL statements for possible improvements (indexes, ABAP) Our experience is that the buffer hit quality in a well tuned SAP/Oracle system (including SQL statement tuning and index usage) Should be over 95% running with the Oracle cost-based optimizer Should be higher equal 98% running with the Oracle rulebased optimizer
SAP AG 1999
SAP AG 1999
LRU mechanism
LRU*-List
= Used Blocks
Unused buffers at the end of the list are replaced in the buffer cache Shadow process starts searching at the end of the LRU list
SAP AG 1999
DB_Block_LRU_Latches
(Available only with Oracle 7.3 onwards) Single LRU latch Multiple LRU latches
Buffer cache
SAP AG 1999
LRU latch
Buffer cache
LRU latches
Bloc k is p laced a
t the
end o f
Only a small number of buffers at the end of the list are replaced in the buffer cache
SAP AG 1999
SAP AG 1999
Latch name
Misses
SAP AG 1999
DBWR Events
SAP AG 1999
Buffer cache
LRU queue
Max scan septh Changed by scan depth increment/decrement steps Min scan depth
DBWR
SAP AG 1999
Buffer cache
No
2. Dirty buffer found on the LRU queue ? shadow process Scan Depth
Buffers Scanned 5 0001234 6 Sleep on event free buffer wait Signal DBWR
No
3. Use the buffer and move it to the head of LRU
Yes
Yes
2.1 Dirty queue length >= Max dirty queue length ?
No
LRU queue
Dirty queue
Yes
Signal DBWR
SAP AG 1999
DBWR Actions
Signal from shadow process
Go IDLE, Wait for signals 2.1 Free LRU latch
Yes
2. Dirty queue empty and known free buffers > 1/2 DBWR scan depth?
No
4.1 Scan LRU for dirty blocks
No
Yes
4.2 Stop if dirty queue has a write batch or max scan depth reached 5. Release LRU latch
Yes
8. New signals?
No
SAP AG 1999
Read/Write without Async I/O 1. I/O request Wait for completion 2. I/O request 20 ms
20 ms 20 ms
20 ms 20 ms
Wait for completion 3. I/O request Wait for completion 4. I/O request
20 ms
20 ms 20 ms 20 ms 40 ms
80 ms
SAP AG 1999
DBWR Processes
DB_WRITER_PROCESS 1
DB_WRITER_PROCESS 2
Data files
DB_WRITER_PROCESS n
Buffer cache
LRU latches
SAP AG 1999
Slave 1
DB_WRITER PROCESS
Slave 2
Slave n
Data files
Buffer cache
LRU latches
SAP AG 1999
Before tuning the DBWR analyze view V$SYSTEM_EVENT for I/O problems causing a slow DBWR For details refer to unit Monitoring File System Requests write complete waits write complete waits free buffer waits free buffer waits ----------------------Should not have any major impact Should not have any major impact
SAP AG 1999
Summed dirty queue length / write requests < 1 WHERE name like 'free buffer%' or name like 'dirty buffers%' Free buffers requested / dirty buffers inspected < 1 % Free buffers requested / free buffers inspected < 1 %
SAP AG 1999
Possible reasons:
I/O problems DB_BLOCK_BUFFERS too small Throughput I/O subsystem (expensive statements, DBWR problems, ...)
Possible solutions:
Tune I/O, tune expensive SQL statements, use async I/O if possible Increase DB_BLOCK_BUFFERS if possible Use multiple database writer processes
In any case dont change any parameterization without detailed problem analysis
SAP AG 1999
Checkpoint
Start Checkpoint 1 DBWR CHKPT Stop Checkpoint 1 Start Checkpoint 2 Stop Checkpoint 2 Start Checkpoint 3 Stop Checkpoint 3 Start Checkpoint 4
SAP AG 1999
Monitoring Checkpoints
Thread 1 advanced to log sequence 248 Current log# 2 seq# 248 mem# 0: /oracle/PRD/orig_logA/redologs02.log Thread 1 cannot allocate new log, sequence 249 Checkpoint not complete
-> Checkpoint not complete -> System is inaccessible
SAP AG 1999
Possible reasons:
I/O problems DB_BLOCK_BUFFERS too small SAP Recommendation if problems: Throughput I/O subsystem (expensive statements, DBWR problems, ...) SAP Recommendation if problems:
Possible solutions:
Tune I/O, tune expensive SQL statements, use async I/O if possible Increase DB_BLOCK_CHECKPOINT_BATCH Increase DB_BLOCK_BUFFERS if possible Increase DB_FILE_ SIMULTANEOUS_WRITES Use multiple database writer processes In case of Checkpoint not complete increase the number of Online Redo Log Groups or increase the size of the existing groups but only if the time between two Online Redo Log switches is lower the 1 minute
In any case dont change any parameterization without detailed problem analysis
SAP AG 1999
Time stamp starting checkpoint Logswitch Information Time stamp checkpoint completed
SAP AG 1999
SAP AG 1999
Name Background checkpoints completed Background checkpoints started Dirty buffers inspected Free buffer inspected Free buffer requested Summed dirty queue length Write requests
Value
Ratios
SAP AG 1999
When running multiple database writer (DBWR) slaves your system crashes and you receive the following error: ORA-07404: sfareq: Timeout occurred waiting for request to complete Because: The master database writer timed out waiting for a write or close to complete. One of the database writers may have stopped running. Action: Check all database writer trace files. Shut down the database and try to warm start.
SAP AG 1999
LGWR
OrigLog
MirrorLog
2048 Byte I/O request DBWR OS 2048 Byte 2048 Byte 2048 Byte Backup tool read 2 new and 2 old blocks
SAP AG 1999
LGWR Events
Each COMMIT When the REDO log buffer is 30% filled up After 3 seconds idle time When DBWR writes
SAP AG 1999
SAP AG 1999
Transaction ST04
Monitoring I/O problems of the LGWR Before tuning the LGWR analyze view V$SYSTEM_EVENT for I/O problems causing a slow LGWR For details refer to unit Monitoring File System Requests
log file sync log file sync Average <= 40 ms Average <= 40 ms
SAP AG 1999
SAP AG 1999
event log file sync log file parallel write log buffer space db file sequential reads
Position
---------
---------
SAP AG 1999
In any case dont change any parameterization without detailed problem analysis
SAP AG 1999
B C . A B C
Allocation latch
SAP AG 1999
B . C
B C
Allocation latch
SAP AG 1999
Allocation latch
SAP AG 1999
WHERE name like 'redo small copies%' or name like 'redo entries%'
Redo allocations: misses / gets < 0.001 Redo copy: misses / gets < 0.001
SAP AG 1999
REDO allocations: misses / gets and immediate misses / immediate gets < 0.001
Otherwise: Shadow processes wait too long for allocation latch Solution: Decrease Log_small_entry_max_size to handle more requests with REDO copy latches
REDO copy latches: misses / gets and immediate misses / immediate gets < 0.001
Otherwise: Shadow processes wait too long for REDO latch because latch is busy Solution: Increase LOG_SIMULTANEOUS_COPIES to twice the number of installed CPUs
SAP AG 1999
SAP AG 1999
SAP standard SAP standard recommendations recommendations Check note 124361 Check note 124361
SAP AG 1999
Archiver Fundamentals
ARCH
ARCH
LOG_ARCHIVE_BUFFERS
Read the log_archive_buffer Write the archive files with LOG_ARCHIVE_BUFFER_SIZE. Update control file, change REDO log status
ARCH
SAP AG 1999
Otherwise: Solution:
System becomes stuck Check the disks archives are placed on Increase LOG_ARCHIVE_BUFFER_SIZE Increase LOG_ARCHIVE_BUFFER_SIZE Use ARCH_IO_SLAVES
SAP AG 1999
SAP AG 1999
Shared pool available for all SQL statements Shared pool size as defined in the init<SID>.ora
Shared pool used for information about DD objects Self configured by the database
SAP AG 1999
Storing Statements
Statement: ABAP coding: SELECT from table WHERE field1 in itab Shared SQL Area: SELECT * FROM TABLE WHERE MANDT =:A1 AND FIELD1 IN (:A2, :A3, :A4, ..., :A600) Optimizer access path stored in Shared SQL Area: SELECT * FROM TABLE WHERE MANDT = 000 AND FIELD1 = 123 OR SELECT * FROM TABLE WHERE MANDT = 000 AND FIELD1 = 124 .... One OR clause ~20KB IN clause with 600 entries ~12MB
SAP AG 1999
Transaction ST04
?
reloads / pins <= 0,04 >= 2
SAP AG 1999
SAP AG 1999
SAP AG 1999
Reloads / pins <= 0.04 Otherwise: Too many parse calls, transactions become slow Solution: Check for statements with IN clauses with multiple values Change IN to FOR ALL ENTRIES Increase SHARED_POOL_SIZE Statements used less than 3 times < 20% Otherwise: Memory on database server is wasted Solution: Decrease SHARED_POOL_SIZE Dont increase the SHARED_POOL_SIZE larger then 400 MB without detailed analysis
SAP AG 1999
SHARED_POOL_RESERVED_MIN_ALLOC
SAP AG 1999
NAME
Recommended Settings for the init<SID>.ora: SHARED_POOL_RESERVED_SIZE = 10% of Shared Pool Size SHARED_POOL_RESERVED_MIN_ALLOC between 2048 and 3072
SAP AG 1999
SAP Recommendation: SAP Recommendation: Dont change the default value (FALSE) Dont change the default value (FALSE)
SAP AG 1999
ORA-04031, 00000, "out of shared memory when trying to allocate %s bytes (%s,%s)" Because: More shared memory is needed than was allocated in the shared pool. Action: Reduce your use of shared memory, or increase the amount of available shared memory by increasing the value of the init.ora parameter "shared_pool_size".
SAP AG 1999
Shadow process B
te Crea k ac Rollb
SAP AG 1999
PRS_1
PRS _2
PRS _3
PRS _4
PRS _5
PRS _6
PRS _7
PRS _8
PRS _9
PRS _n
Transaction is assigned to the rollback segment with the least number of active transactions
B C
SAP AG 1999
E3 T1
T1
T2
E2 T2
T1
...
SAP AG 1999
E4 T1
T1
T2
E1 T1
T2
...
SAP AG 1999
SAP AG 1999
Undo header + undo block : Buffer Busy Waits / Reads * 100 <= 0,1%
SAP AG 1999
Buffer Busy Waits / Reads * 100 <= 0.1 Otherwise: Check classes undo header and undo block. Maybe the high number of waits is caused by shadow processes waiting too long for rollback segments Reason: Write load may be too high on a few rollback segment data files caused by a small number of rollback segments. Solution: Increase the number of rollback segments Thresholds for the number of Rollback Segments SAP recommendation: At least workprocesses / 4 Oracle recommendation: Concurrent users / 2
SAP AG 1999
SAP AG 1999
Calculate Optimal Size: Number of RBS with HWM > Optimal size < 10%
SAP AG 1999
1. Number of RBS with HWM greater Optimal size < 10% Otherwise: To many space management operations on Rollback Segments Transaction take longer Solution: Create new Rollback Segments with appropriate storage parameters SAP recommendations for the setup of the Rollback Segments see note 165247 2. Create more than 1 tablespace for the rollback segments and create as many datafiles on different disks as possible Otherwise: Administration becomes more difficult Write load may be focussed on a few disks DBWR becomes slow because disk are to busy
SAP AG 1999
Transaction T1
COMMIT
Transaction T2 Update MARA ... Select from MARA = Select old values from rollback
Rollback Segment
Table MARA
SAP AG 1999
1. Expensive query
Database buffer pool Large number of Large number of buffers used on the buffers used on the database server database server ORA 1555
Application server
Snap dump
High processing High processing time on the time on the application server application server
Application server
SAP AG 1999
1.
A table is changed, changed, dirty blocks are written to disk, then the changes are committed A query runs against this table
B C . A B C
2.
. A
3.
. A B . A C B C
Further change activity writes to the RBS until it wraps and the lowest SCN in the RBS is higher than the starting SCN of the query When the query reads a dirty block, the required rollback information is already overwritten
4.
ORA 1555
Tune the SQL statement or the ABAP coding which causes the ORA-1555 Change the application design so that fewer COMMITs take place Increase the number of rollback segments Increase the size of the rollback segments
SAP AG 1999
CREATE INDEX SELECT ... ORDER BY SELECT DISTINCT ... SELECT ... GROUP BY SELECT ... UNION SELECT ... INTERSECT SELECT ... MINUS
SAP AG 1999
DBWR
PSAPTEMP
SAP AG 1999
Direct Sort
INITIAL
NEXT
NEXT
PSAPTEMP
SAP AG 1999
Tables SMON
Indexes
SAP AG 1999
SAP AG 1999
SAP AG 1999
ORA-01575: time-out waiting for space management resource Because: The system timed out before it could acquire the necessary resources to do space management. Action: Wait a few minutes, then retry the operation.
SAP AG 1999