Tuning I/O

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Objectives

After completing this lesson, you should be able to do the

following:
• Diagnose database I/O issues
• Describe the Stripe and Mirror Everything (SAME) concept
• Set filesystemio_options
• Choose appropriate I/O solutions
• Tune Automatic Storage Management (ASM)

19 - 2 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 I/O Architecture

Oracle Database 11g includes three standard storage

options:
• File system
– Network attached storage (NAS)
– Storage area network (SAN)
– Direct attached storage
• Automatic Storage Management (ASM)

19 - 3 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 File System Characteristics

Certain characteristics are better for database use:

• Write-through-cache ability
• Write acknowledgement
• Security
• Journaling
• High performance

19 - 4 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 I/O Modes

I/O can be written to disk in

several ways by using different
system calls:
• Standard I/O
Process
• Synchronous I/O Direct
• Asynchronous I/O Write

• Direct I/O

Flush

Process Buffer cache Disk file

19 - 5 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Direct I/O

Direct I/O is considered to be the high-performance

solution.
• Direct reads and writes do not use the OS buffer cache.
• Direct reads and writes can move larger buffers than file
system I/Os.
• Direct I/O can be synchronous or asynchronous.

Write

Read
Process Disk file

19 - 6 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Bandwidth Versus Size

I/O performance depends on bandwidth.

• Number of disks, not size
• Number of controllers

Disk controllers

Background
process

19 - 7 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Important I/O Metrics for Oracle Databases

Disk bandwidth Channel bandwidth

Metric = IOPS Metric = MBPS

and latency
Need large
Need high RPM and
I/O channel
fast seek time

OLTP DW/OLAP
(Small random I/O) (Large sequential I/O)

19 - 8 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 I/O Calibration and Enterprise Manager

19 - 10 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 I/O Calibration and the PL/SQL Interface

SQL> exec dbms_resource_manager.calibrate_io( -

num_disks=>1, -
max_latency=>10, -
max_iops=>:max_iops, -
max_mbps=>:max_mbps, -
actual_latency=>:actual_latency);

PL/SQL procedure successfully completed.

SQL>
2 FROM DBA_RSRC_IO_CALIBRATE;

MAX_IOPS MAX_MBPS MAX_PMBPS LATENCY

---------- ---------- ---------- ----------
137 12 6 64

19 - 11 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 I/O Statistics: Overview

V$IOSTAT_FUNCTION

AWR and EM .
V$IOSTAT_FILE

V$IOSTAT_CONSUMER_GROUP

19 - 13 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 I/O Statistics and Enterprise Manager

19 - 14 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 I/O Statistics and Enterprise Manager

19 - 15 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Stripe and Mirror Everything

• All data files to access all available bandwidth

• All database files to be on the same logical devices
• Highest performance configuration

19 - 16 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Using RAID

Redundant Array of Inexpensive Devices (RAID) levels:

• Level 0
– Striped for performance
– No redundancy
• Level 1
– Mirrored for safety
– Little performance benefit
• Level 5
– Block level redundancy (rebuild algorithm)
– Improved read performance
– Additional write cost

19 - 17 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 RAID Cost Versus Benefits

RAID cost is measured in performance and reliability.

• RAID 0:
– Fast
– The loss of any device damages the array.
• RAID 1:
– Safe and expensive
– Slight benefit in high-read environments
• RAID 5
– Fast
– Safe with loss of any one device
– Possible high write cost

19 - 18 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Should I Use RAID 1 or RAID 5?

RAID 1 (Mirroring) RAID 5 (Parity)

• Recommended by Oracle • DSS and moderate OLTP
• Most demanding applications

Advantages Advantages
• Best redundancy • Requires less capacity
• Best performance Disadvantages
• Low recovery overhead • Less redundancy
Disadvantages • Less performance
• Requires higher capacity • High recovery overhead

19 - 20 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Diagnostics

Indicators of I/O issues:

• Top waits are reads and writes plus:
– Buffer busy waits
– Write complete waits
– DB file parallel writes
– Enqueue waits
• The file I/O Statistics section shows high waits and AVG
Buffer Wait time higher than average on certain files.
Note: On a well-performing system, the top events are likely to
be CPU time, DB file scattered read, and DB file sequential
read.

19 - 21 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Database I/O Tuning

• Configuring storage for a database depends on many

variables:
– Which data to put on which disk; complicated by vendor-
configured logical units (LUNs)
– DB application workloads: OLTP, DSS, batch versus online
– Trade-offs between available options
– Ongoing tuning: changes in workloads
– Expanding or contracting your database

19 - 22 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following advantages is provided by the Stripe and

Mirror everything approach?
a. Administrators must continually monitor the data file
growth.
b. Data is protected by mirroring.
c. The redo log files should be placed on a different storage
for best performance.
d. All database files share all the available I/O bandwidth.

19 - 23 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 What Is Automatic Storage Management?

ASM:
• Is a portable and high- Application
performance cluster file system
Database
• Manages Oracle database files
• Distributes data across disks File
system
to balance load ASM
Volume
• Provides integrated mirroring manager
across disks
Operating system
• Solves many storage
management challenges
• Encapsulates the SAME methodology

19 - 24 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Tuning ASM

• Adjust rebalancing speed

• Set redundancy on a file basis
• Set DISK_ASYNCH_IO to TRUE

. . .. .. .. ..
...... .. .. .. .. ..
...... ...... .. .... ..
...... ...
.. ............................... ... ..
.. .. .. .. ..
..
. ..
....
.
................. .

19 - 25 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 How Many Disk Groups per Database

• Two disk groups are recommended:

– Leverage maximum of LUNs
Data DG FRA DG
– Backup for each other
ERP DB
– Lower performance may be CRM DB
used for FRA (or inner tracks)
HR DB
• Exceptions:
– Additional disk groups for different capacity or performance
characteristics
– Different ILM storage tiers

19 - 26 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Which RAID Configuration
for Best Availability?
A. ASM mirroring
B. Hardware RAID 1 (mirroring)
C. Hardware RAID 5 (Parity Protection)
D. Both ASM mirroring and hardware RAID

Answer: Depends on business requirement and budget

(cost, availability, performance, and utilization)

ASM leverages hardware RAID.

19 - 27 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Mirroring Guidelines

• Best choice for low-cost storage

• Enables Extended Clustering solutions
• No hardware mirroring

19 - 28 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Striping Granularity

COARSE FINE

1MB
AU

19 - 29 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 What Type of Striping Works Best?

A. ASM striping only (no RAID 0)

B. RAID 0 and ASM striping
C. Use LVM
D. No striping

Answer: A and B

ASM and RAID striping are complementary.

19 - 30 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Striping Only

Advantages Disadvantages
• Drives evenly distributed for Data & FRA • Not well balanced across
• Higher bandwidth all disks
• Allows small incremental growth (73 GB) • LUN size limited to disk size
• No drive contention

Oracle DB size: 1 TB
Data DG FRA DG
Storage configuration:
1 TB 16  73 GB 8  arrays with 2 TB 32  73 GB
LUNs 12  73 GB disks per array LUNs
RAID 1

19 - 31 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Hardware RAID Striped LUNs

Advantages Disadvantages
• Fastest region for Data DG • Large incremental growth
• Balanced data distribution • Data and FRA “contention”
• Fewer LUNs to manage while max spindles

Database size: 1 TB
Data DG FRA DG
Storage configuration:
1 TB 4  250 GB 8  arrays with 2 TB 4  500 GB
LUNs 12  73 GB disks per array LUNs
RAID 0+1

19 - 32 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Guidelines

• Use external RAID protection when possible.

• Create logical units (LUNs) using:
– Outside half of disk drives for highest performance
– Small disk, high RPM (for example, 73 GB/15k RPM)
• Use LUNs with the same performance characteristics.
• Use LUNs with the same capacity.
• Maximize the number of spindles in your disk group.

19 - 33 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Instance Initialization Parameters

• ASM instances have static memory needs.

• Use Automatic Memory Management.
• Using default SGA sizing parameters should be enough
for most configurations:

INSTANCE_TYPE = ASM
DB_UNIQUE_NAME = +ASM
ASM_POWER_LIMIT = 1
ASM_DISKSTRING = '/dev/rdsk/*s2', '/dev/rdsk/c1*'
ASM_DISKGROUPS = dgroupA, dgroupB
MEMORY_TARGET = 256M /* Default value */

19 - 34 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Dynamic Performance Views

V$ASM_TEMPLATE
V$ASM_CLIENT V$ASM_DISKGROUP

Disk group A Disk group B

V$ASM_FILE
V$ASM_ALIAS
Storage system

V$ASM_DISK
V$ASM_OPERATION

19 - 35 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Monitoring Long-Running Operations by Using
V$ASM_OPERATION

Column Description
GROUP_NUMBER Disk group
OPERATION Type of operation: REBAL
STATE State of operation: WAIT or RUN
POWER Power requested for this operation
ACTUAL Power allocated to this operation
SOFAR Number of allocation units moved so far
EST_WORK Estimated number of remaining allocation units
EST_RATE Estimated number of allocation units moved per minute

EST_MINUTES Estimated amount of time (in minutes) for operation

termination

19 - 37 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Instance Performance Diagnostics

SELECT event, total_waits t_wait,

total_timeouts t_timeout,
time_waited t_waittm,
average_wait a_waittm, wait_class
FROM V$SYSTEM_EVENT
WHERE wait_class <> 'Idle' and time_waited > 0
ORDER BY 4 DESC;

EVENT WAIT TOUT WAITT AVG CLASS

------------------------------ ------ ----- ----- ------- -------
ASM mount : wait for heartbeat 1 1 439 438.85 Admin…
kfk: async disk IO 578 0 377 .65 SystI/O
log write(odd) 7 3 296 42.33 Other
rdbms ipc reply 37 1 259 7.01 Other
log write(even) 8 2 197 24.58 Other
SQL*Net message to client 139249 0 103 0 Network
os thread startup 9 0 79 8.77 Conc…
buffer write wait 1 0 60 60.31 Other
DBFG waiting for reply 16 0 1 .04 Other

19 - 38 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Performance Page

19 - 39 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Database Instance Parameter Changes

• Add the following to SHARED_POOL_SIZE:

(DB_SPACE/100)*#_External_Redundancy +2 OR
(DB_SPACE/50)*#_Normal_Redundancy +4 OR
(DB_SPACE/33)*#_High_Redundancy +6

SELECT d+l+t DB_SPACE

FROM (SELECT SUM(bytes)/(1024*1024*1024) d
FROM V$DATAFILE),
(SELECT SUM(bytes)/(1024*1024*1024) l
FROM V$LOGFILE a, V$LOG b WHERE a.group#=b.group#),
(SELECT SUM(bytes)/(1024*1024*1024) t
FROM V$TEMPFILE WHERE status='ONLINE');

• Add at least 16 to PROCESSES.

• Add 600 KB to LARGE_POOL_SIZE.

19 - 40 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 ASM Scalability

ASM imposes the following limits:

• 63 disk groups
• 10,000 ASM disks
• 4 petabytes (PB) per ASM disk
• 40 exabytes (EB)of storage
• 1 million files per disk group
• Maximum file size:
– External redundancy: 140 PB
– Normal redundancy: 42 PB
– High redundancy: 15 PB

19 - 41 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Quiz

Automatic Storage Management (ASM) is a replacement for a

logical volume manager, which enables you to administer a
group of disks as a logical unit. ASM uses the SAME
methodology.
a. True
b. False

19 - 42 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

 Summary

In this lesson, you should have learned how to:

• Diagnose database I/O issues
• Describe the Stripe and Mirror Everything (SAME) concept
• Explain the benefits of asynchronous I/O
• Choose appropriate I/O solutions
• Tune I/O using Automatic Storage Management (ASM)

19 - 43 Copyright © 2010, Oracle and/or its affiliates. All rights reserved.