Best Practice For Connecting IBM Storage To Power Systems-2016

Best Practices for Performance, Design and Troubleshooting IBM
Storage Connected to IBM Power System

Chuck Laing
Senior Technical Staff Member (STSM)
IBM GTS SO/IS Delivery Organization
Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
Knowledge is POWER!
As Systems Administrators we dont always

KNOW what we dont know about storage
Ask for storage leveraging what you know
Avoid bottlenecks
Use tools available
Speed problem isolation
Make more informed architectural decisions
What we are NOT going to do today:

Try to turn you into storage administrators
Boil the ocean
IBM Systems Technical Events | ibm.com/training/events
Copyright IBM Corporation 2016. Technical University/Symposia materials

may not be reproduced in whole or in part without the prior written permission of
IBM.
Throughput and Performance Key Optimization Factors

Throughput
Spreading and balancing IO
across hardware resources
IO Performance Tuning
Using utilities, functions and
features to tweak (backend, frontend)
Controllers
Ports & zoning connections
PCIs Cards
Qdepths
HBA transfer rates
CPUs, RAM
Disks spindles
LVM striping vs spreading

Data Placement
Compression
Thin Provisioning
Easy Tier - SSD
FC adapters
Random versus sequential
Spreading versus Isolation

Application characteristics
Etc.
Configuring throughput optimally increases potential performance scalability


IBM.
Foundation What causes performance degradation and Outages?
The 3 most common root causes are:

Configuration changes
Hardware component failure
IO load shift or increase
You should consider designing the environment to

withstand load shift in the event of half the
environment failing such as:
Controller outages
Fabric outages
Server Outages
You should design configurations to known Best

Practices
Just because you can do something should you?

IBM.
Agenda -Troubleshooting as growth and change occur

Top 5 most common things that go wrong by configuration design:
From Field Experience - Deviation from Best Practices/Trending
Bottlenecks in the Physical Design

1.
2.
3.
4.
5.
Bottlenecks in the SAN Configurations Design

1.
2.
3.
4.
5.
Suboptimal zoning practices

Port Placement for Best IO spread and redundancy
Slow Draining Devices -Bad or dirty SFP connections
Fillwords
Port Speeds fixed vs. auto negotiate
Bottlenecks in the Storage Configurations Design

1.
2.
3.
4.
5.
Rack and Stack Design

Manageability designing for physical growth
Fibre cabling practices
Fabric Topology
Cooling
Data Layout - Storage Pools and Tiered pools - IO Spread vs IO isolation

IO Spread and Load Balancing across CPU Complexes
IO Spread and Load Balancing across DA Pairs
IO Spread and Load Balancing across Disk Enclosures
IO spread and Load Balancing across ports
Bottlenecks in Host Configurations Design

1.
2.
3.
4.
5.
Using the right Host Multipath Drivers to match Storage Microcode

Striping , Spreading, Data Layout,
HBA transfer rates
Adjusting Qdepth
Workload IO increases due to Data Growth

IBM.
Bottlenecks in the physical design

A deeper dive into the Physical Topology

IBM.
Bottlenecks in Physical setup Rack and Stack Best Practices

Rear View
Front View
A right way example

Clean and Neat

IBM.
Bottlenecks in Physical setup Rack and Stack

# 1 Incorrect ways to rack and stack
#2 Manageability for growth
#3 Fibre cabling practices
#4 Fabric topology

8

IBM.
Bottlenecks in Physical setup Rack and Stack

Determine if placement follows best practices taking advantage of
hot/cold aisle and perforated tiles. The hot exhaust of one device blowing
into the cold air intake of another could lead to overheating.

IBM.
Bottlenecks in physical setup - Cooling #5

Whats wrong here?
It was off in a corner with no surrounding
air flow.
Impact: Shortened component life span
from over heating.
Everytime the cabinet door was opened
for service, it blew:
A power supply
Fans always failing
Application outages

IBM.
10
Quiz #1
.
Unserviceable
fabric rack
1. Bend Radius
exceeded
2. Insufficient
strain relief
Cable weight
pull on other
cables
3. Cables loose
on floor
Susceptible to
pinching,
getting caught
in door, being
stepped
onetc.
11

IBM.
11
Quiz #2
What is the impact of this?
Cant service
Blocked air exhaust

IBM.
12
Quiz #3
Whats Wong!!

IBM.
13
Bottlenecks in the SAN Fabric - Best Practice
Suboptimal zoning practices

IBM.
14
#1 Correct Zoning to Multiple Storage devices
Create separate zones for each Storage device

V7000
2
4
1
1
CAUTION
4
1
3
DS8K-2
2
1
2
Disconnect all
supply power for
complete isolation
1
3
2
4
CAUTION
1
3
2
1
2
4
CAUTION
XIV
Zones
Fabric1 Core1
DS8K-1
Disconnect all
supply power for
complete isolation
V7K
Zones
XIV
2
4
Disconnect all
supply power for
complete isolation
2
4
1
3
2
4
1
3
CAUTION
1
3
Disconnect all
supply power for
complete isolation
DS8K-1
Zones
DS8K-2
Zones
V7K
Zones
XIV
Zones
Fabric 2 Core2
DS8K-1
Zones
DS8K-2
Zones

IBM.
15
#1 - Incorrect Zoning to Multiple Storage devices

Zone each device to only one core to avoid extra ISL IO traversing cutting down ISL hops
Spread Multiple Storage devices as evenly possible across Cores
Create separate zones for each Storage device
V7000
1
3
2
4
1
1
CAUTION
2
4
1
3
2
4
Disconnect all
supply power for
complete isolation
XIV
DS8K-1
DS8K-2
2
1
Disconnect all
supply power for
complete isolation
1
3
2
4
CAUTION
1
3
Disconnect all
supply power for
complete isolation
Disconnect all
supply power for
complete isolation
1
3
2
4
1
3
2
1
2
4
CAUTION
CAUTION
Storage
Zone-1
Fabric1 Core1
Storage
Zone-2
Fabric 2 Core2

IBM.
16
One Zone per SVC Node per Fabric for each back-end storage device
Making 1 zone per Node per Fabric with
the same ports from a single backend
storage unit, will reduce the overheads
associated with large numbers of logins
Production
SAN Fabric
A
STG
Zone-2
STG
Zone-3
STG
Zone-1
Physical
port
number
16G HBA
8G HBA
1 2
1 2
1 2
2 2
1 2 3 4
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 1
Slot 5
1 2
1 2
1 2
2 2
1 2 34
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
Production
SAN Fabric
B
STG
Zone-4
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 2
Slot 5
Logical
port with
wwpn #
embedded
I/O Group 0
DH8 1 Building Block 16 FC ports per node


IBM.
17
#1 - Storage Zone Type Good Core Edge Design 1ISL Hops

Back-end Storage
HBA1
Host X
HBA2
P P
1 2
B1
A1
P P
1 2
Host
Zone-1
STG
Zone-2
STG
Zone-1
Host
Zone-2
IO request 1 ISL Hop1

Fabric1
Physical
port
number
16G HBA
8G HBA
1 2
1 2
1 2
2 2
1 2 3 4
Slot 1
Core1
2 2
1 2
Fabric 1
2 2
3 4
5 6 7 8
Slot 2
Edge 1
Fabric2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 1
Slot 5
1 2
1 2
1 2
2 2
1 2 34
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
Edge1
Fabric 2
Core 1
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 2
Slot 5
Logical
port with
wwpn #
embedded
I/O Group 0


IBM.
18
The same back-end ports are used for each Node Zone
XIV Storage
Module1
HBA1
HBA2
P P
1 2
STG
SAN
Zone1
P P
3 4
Module2
HBA1
P P
1 2
STG
Zone3
HBA2
P P
3 4
Module3
HBA1
HBA2
P P
1 2
Fabric 1
P P
3 4
Module4
HBA1
P P
1 2
STG
SAN
Zone2
HBA2
P P
3 4
STG
Zone4
Module5
HBA1
P P
1 2
HBA2
P P
3 4
Module6
HBA1
P P
1 2
HBA2
P P
3 4
Fabric 2
Up to 16 zones for an 8 node Cluster

2 zones /fabric/ node
Up to 16 Ports max from any
back-end storage

IBM.
19
Host Zones 1 host initiator to 2 SVC node target ports

Should be (1:2) zoning
Host
A1
B1
1 host initiator port
Fabric2 Edge1
Fabric1 Edge1
2 SVC Node
Target ports
Physical
port
number
16G HBA
8G HBA
1 2
1 2
1 2
2 2
1 2 3 4
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 1
Slot 5
1 2
1 2
1 2
2 2
1 2 34
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 2
Slot 5
Logical
port with
wwpn #
embedded
I/O Group 0


IBM.
20
Host Zones Too few

No (1:1) zoning
Host
A1
B1
1 host initiator port
Fabric2 Edge1
Fabric1 Edge1
1 SVC Node
Target ports
Physical
port
number
16G HBA
8G HBA
1 2
1 2
1 2
2 2
1 2 3 4
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 1
Slot 5
1 2
1 2
1 2
2 2
1 2 34
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 2
Slot 5
Logical
port with
wwpn #
embedded
I/O Group 0


IBM.
21
Host Zones 4 paths per Vdisk

2 preferred 1 red & 1 blue
2 non preferred red & blue
Host
A1
B1
Fabric2 Edge1
Fabric1 Edge1
Non Preferred Paths
Preferred Paths
Physical
port
number
16G HBA
8G HBA
1 2
1 2
1 2
2 2
1 2 3 4
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 1
Slot 5
1 2
1 2
1 2
2 2
1 2 34
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 2
Slot 5
Logical
port with
wwpn #
embedded
I/O Group 0


IBM.
22
Host Zones too many paths and target ports

Host
A1
B1
Fabric2 Edge1
Fabric1 Edge1
Non Preferred Paths
Preferred Paths
Physical
port
number
16G HBA
8G HBA
1 2
1 2
1 2
2 2
1 2 3 4
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 1
Slot 5
1 2
1 2
1 2
2 2
1 2 34
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 2
Slot 5
Logical
port with
wwpn #
embedded
I/O Group 0


IBM.
23
Host Zones Too few

Spectrum Virtualize - DH8
Host
A1
B1
Fabric2 Edge1
Fabric1 Edge1
Non Preferred Paths
Preferred Paths
Physical
port
number
16G HBA
8G HBA
1 2
1 2
1 2
2 2
1 2 3 4
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 1
Slot 5
1 2
1 2
1 2
2 2
1 2 34
Slot 1
2 2
1 2
2 2
3 4
5 6 7 8
Slot 2
3 3 3 3
5 5 5 5
1 2 3 4
1 2 3 4
9 10 11 12 13 14 15 16
Slot 3
Node 2
Slot 5
Logical
port with
wwpn #
embedded
I/O Group 0


IBM.
24
Quiz #1 - Right or Wrong?
Fabric_A Zone Definitions

A1=10:00:00:00:c9:3f:75:37
A2=10:00:00:00:c9:3f:80:54
A3=10:00:00:00:c9:42:89:ac
A4=10:00:00:00:c9:42:80:16
Fabric_B Zone Definitions

B1=10:00:00:00:c9:3f:74:16
B2=10:00:00:00:c9:3f:f7:2f
B3=10:00:00:00:c9:42:b4:4b
B4=10:00:00:00:c9:42:b6:18
b03vio210
A1 A2 A3 A4 B1 B2 B3 B4
DIR2 SAN Fabric
DIR1 SAN Fabric
P1 P2 P3 P4
HBA 1
HBA 2
Node 1
P1 P2 P3 P4 P1 P2 P3 P4
HBA 1
HBA 2
Node 2
HBA 1
HBA 2
Node 3
SVC Host Definitions

P1 P2 P3 P4
HBA 1
Node 4
iogrp 0
iogrp 1
2048 LUNs max
2048 LUNs max
wwpn
5005076801
HBA 2
4 Node SVC MAX Vdisk 4096
id:8
name:b03vio210_allvg
10000000C942B618
10000000C942B44B
10000000C9428016
10000000C94289AC
10000000C93FF72F
10000000C93F7416
10000000C93F8054
10000000C93F7537
In this example we show

1 SVC Host definitions and
2 Fabric zones, per host per Fabric.
This does not allow proper failover from
Primary to Alternate nodes within the
SVC iogrp.
See the next page for proper zoning

IBM.
25
Right
b03vio210_1
A3
B2
A2

B1=10:00:00:00:c9:3f:74:16
A1=10:00:00:00:c9:3f:75:37

B2=10:00:00:00:c9:3f:f7:2f

A3=10:00:00:00:c9:42:89:ac
SVC Host Definitions
HBA 2
Node 1
P1 P2 P3 P4 P1 P2 P3 P4
HBA 1
HBA 2
Node 2
HBA 1
HBA 2
Node 3
P1 P2 P3 P4
HBA 1
iogrp 1
2048 LUNs max
2048 LUNs max
wwpn
HBA 2
Node 4
iogrp 0
5005076801
B4

A4=10:00:00:00:c9:42:80:16

B4=10:00:00:00:c9:42:b6:18
DIR2 SAN Fabric
DIR1 SAN Fabric
HBA 1
A4
B3

B3=10:00:00:00:c9:42:b4:4b

A2=10:00:00:00:c9:3f:80:54
P1 P2 P3 P4
b03vio210_4
b03vio210_3
b03vio210_2
B1
A1
id:1 name:b03vio210_1
10000000C93F7537
10000000C9428016
10000000C93F8054
10000000C93FF72F
10000000C94289AC
10000000C942B44B
In this example we show

4 seperate SVC Host definitions and
2 seperate Fabric zones, per host
per Fabric
This does allow proper failover from
Primary to Alternate nodes within the
SVC iogrp.
4 Node SVC MAX Vdisk 4096
10000000C93F7416
10000000C942B618

IBM.
26
Summary - 3 factors determine right/wrong paths
Example of correct/incorrect number of datapaths with Host to Spectrum Virtualize

Correct datapathing has 3 factors
Proper zoning
Proper Spectrum Virtualize Host definitions (Spectrum Virtualize logical config of the host def)
Proper redundancy for the Spectrum Virtualize preferred /non preferred pathing

IBM.
27
Dual VIOS to Multiple LPARs

Map LUNs within an LPAR in a round robin fashion to the active pseudo hosts. For example:
Vdisk1 to Pseudo1 Vdisk2 to Pseudo2 Vdisk3 to Pseudo1 Vdisk4 to Pseudo2
VIO Server1
P1
FCA1
P2
P5
P3
FCA2
VFCA1.1
P4
P7
VIO Server2
FCA3
SAN
Pseudo1
VP7.1aVFCA
FCA4
VFCA5.1
P8
VP5.1a
VP5.1i
VFCA6.1
VP2.2i VP2.2a
VFCA3.1
VP6.2a
VP3.1a
Spectrum
Virtualize
VFCA4.1
VP4.2i
Avoiding this
Active
Client
Logical
Partition
(LPAR1)
VP5.1aVFCA
Pseudo2
VP1.1i
VP1.1a
VFCA2.1
VP3.1i
VP1.1aVFCA
P6
VP4.2a
VFCA
VP3.1a
VP6.2a
VFCA7.1
VP7.1a VP7.1i
VFCA8.1
VP8.2a VP8.2i
VP2.2aVFCA
Pseudo1
VFCA
VP8.2a
VP6.2aVFCA
I
Active
Client
Logical
Partition
(LPAR2)
Pseudo 2
VFCA
VP4.2a
Fame1 Hypervisor

IBM.
28
VIO Server1
FCA1
FCA2
P1
P2
P3
SAN
P4
P5
VIO Server2
FCA3
P6
P7
FCA4
P8
VP1.1i VP1.1a
VP5.1a
VP2.2i VP2.2a
VP6.2a VP6.2a
SVC
VP3.1i VP3.1a
VP5.1i
VP1.1aVFCA
Pseudo1
VP7.1aVFCA
VP5.1aVFCA
Pseudo2
VP3.1aVFCA
VP2.2aVFCA
Pseudo1
VP8.2aVFCA
VFCA4.1VP7.1i
VP7.1a
VP8.2a VP8.2i
VP4.2i VP4.2a
I VFCA
VP6.2a
Pseudo 2
VP4.2aVFCA
inactive & active

vWWPN pairs
Active
Client
Logical
Partition
(LPAR1)
Active
Client
Logical
Partition
(LPAR2)
LPM
Fame1 Hypervisor
P1
P2
P3
P4
SAN
VFCA
VFCA
P3 P64
Fame2
Hypervisor
P1
P2
VFCA
VFCA
VP1.1i VFCA
P3
P4
SAN
P3 P64 I
LPM
Could go to
Frame2 or
Frame3
Pseudo1
VP7.1i VFCA
VP5.1i VFCA
Pseudo2
VP3.1i
Active
Client
Logical
Partition
(Pseudo
LPAR1b)
During LPM
VFCA
During LPM the number of paths double from 4 to 8

Starting with 8 paths per vdisk will render an unsupported 16
paths during this time - could lead to IO interruption
Fame3
IBM
Systems Technical Events | ibm.com/training/events
Hypervisor

IBM.
29
#2 - Port Placement for Best Practice IO spread and redundancy
Port_groups
Spread across blades

and port_groups
IBM.
30
#2 Incorrect Port Placement for IO spread and redundancy
All 24 ports on blade 2, are taken by the DS8K


IBM.
31
#3 Slow Draining Devices

Slow Drain Explained
A Slow Drain Device is a device that does not accept frames at the rate
generated by the source. In the presence of slow devices, Fibre Channel
networks are likely to lack frame buffers, resulting in switch port credit
starvation and potentially impacting Inter-Switch Links (ISLs) and/or port
channels.
Slow Drain typically occurs when either a storage or end device is not
receiving or sending frames fast enough to ensure that frames are flowing
through the fabric
This creates a situation in which back pressure is felt behind the devices creating the slowness
Symptoms are usually felt fabric wide
**Slow drain can impact devices not directly involved in the problem

IBM.
32
#3 - Slow Drain Causes

Edge devices
Server performance problems
Application or Operating System issues
Host bus adapter (HBA) problems
driver or hardware issue
Speed mismatches
one fast device and one slow device
Non-graceful virtual machine exit on a virtualized server
resulting in frames held in HBA buffers
Storage subsystem performance problems
including overload, runaway process, software bug
Physical Layer issues
Failing SFP, Bad Optical Cable, Bad Panel
Inter Switch Links (ISL)
Lack of B2B credits for the distance the ISL is traversing
The existence of slow drain edge devices cause ISLs to stop moving frames
Edge devices with faster speeds than ISLs even when port-channeled
Example: 4 credits per KM @ 8Gbps
Example: 16Gbps Storage device traversing 8Gbps ISLs
Applies to port channels as well

IBM.
33
#3 - Slow Drain Troubleshooting for Cisco

Use DCNM Host/Enclosure and pathtrace view to help understand
end-to-end connectivity
Use Port-Mon alerts to target specific ports
Use DCNM Slow Drain analysis to find slow draining ports
Start tracing the ports through the fabric
If F-Port is the problem recover if not move to next upstream port

Continue moving from each port to upstream port to find source
Manual steps to recover
Identify the port involved and remedy
Kill process
Reboot server
Fix optical cable or SFP
Bounce or block port to clear buffers
If speed mismatch move to upgrade slower links or makes faster links slower by forcing speed
changes on the port
Monitor that port with Port-mon to ensure reoccurrence does not

happen

IBM.
34
#4 Bad or Dirty SFP connection

IBM.
35
#5 Port Speeds Best Practice of Fixed versus Auto Negotiate?

#1 Top Reason - Improper configuration changes or even worse No configuration
changes
Severity
Why the error has
occurred
Minor
This error has occurred because, port speed on ISL / IFL are not set to fixed speed
ie: A good example should look like this:
128 1 16 658000 id 4G Online E-Port 10:00:00:05:1e:36:05:b0 "<SWITCH_NAME>" (downstream)(Trunk
master)
129 1 17 658100 id 4G Online E-Port (Trunk port, master is Slot 1 Port 16 )
A bad example could be any of these:
48 4 0 653000 id N4 Online E-Port 10:00:00:05:1e:36:05:b0 "<SWITCH_NAME>" (Trunk master)
49 4 1 653100 id N4 Online E-Port (Trunk port, master is Slot 4 Port 0 )
60 4 12 653c00 id N4 Online E-Port (Trunk port, master is Slot 4 Port 13 )
Note : When ever you use the command "portcfgspeed" command, port will go offline and will come online,
hence it is disruptive for that particular link. Implement the change in an appropriate time.
Potential Risks
Having ISL / IFL ports in "Auto Negotiate" mode switches will keep on check for the
connectivity. Which will lead to both the switches to exchange the capabilities which may
lead into principal switch polling
Actions to correct the Make sure that you set the port speeds of ISL / IFL on all switches to a fixed value.
error

IBM.
36
#5 - Brocade Best practice to Trunk

- on all Fibrechannel ISLs
changes
Severity
Major
Why the error has occurred
One or more ISL does not have Trunking enabled.
Potential Risks
1. Running with a ISL without Trunking, ie. a single Fibre

connection(s) between 2 SAN switch's there are some risks:
Single Point of Failure, causing Fabric Segmentation and loss of connectivity if the
only (or last) connection between 2 switch's are lost
Performance bottleneck, ISL Trunking is designed to significantly reduce traffic
congestion in storage networks.
2. If there are 2 ISLs between the switches there are multiple

scenarios why a trunk is not formed
Actions to correct the error
Either there are no Trunking license

The links are cabled in different ASICs in either end
The difference in length of the cables are to long
There are "noise" in one cable, could be a bad connector or patch panel, a too much
bend cable etc.
Add more connections between this switch and the neighbor switch running
with a single connection and/or purchase a Trunking License.

IBM.
37
Bottlenecks in the Storage Design

The Top 5 common things that are or go wrong with Storage Best Practices

IBM.
38
What causes THRASHING?
#1 -Data Layout/Placement
Placing Applications on the same LUNs/Pools result in IO contention
Most commonly when workloads peak at the same time or log files and data files share physical spindles
made of strips on the
outerstorage
edge of the DDMs
(1s) server
also could have
App A Raid-5 7+P monitoring tools to
For existing LUN1
applications,
use
and
performance
understand current application workload characteristics such as:
1
1
Read/Write
ratio
2
2
1
2
1
2
5
4
3
1
2
5
4
3
5
5
4
4
Random/sequential
3
3
Remote mirroring link utilization and throughput
5
ratio3 4
1
2
5
4
3
1
2
5
4
3
1
2
5
4
3
Average transfer size (blocksize)

Peak workload (I/Os per second for random access, and MB per second for sequential access)
made of strips in the middle of the DDMs (3s) also could have App B Raid-5 7+P
PeakLUN3
workload
periods (time of day, time of month)
Copy services requirements (Point-in-Time Copy, Remote Mirroring)
Host connection utilization and throughput (HBA Host connections)
Extent pool or 8 Ranks
Strip1
Strip2
Strip3
Strip4
Strip5

IBM.
39
39
Data Placement on Power Systems

#1 - Data Layout Summary
Does data layout affect IO performance more than any tunable IO parameter?
Good data layout avoids dealing with disk hot spots
An ongoing management issue and cost
Data layout must be planned in advance
Changes are generally painful
iostat might and filemon can show unbalanced IO

Best practice: evenly balance IOs across all physical disks unless TIERING
Random IO best practice:

Spread IOs evenly across all physical disks unless dedicated resources are needed to
isolate specific performance sensitive data
For disk subsystems
Create RAID arrays of equal size and type

Create VGs with one LUN from every array
Spread all LVs across all PVs in the VG

IBM.
40
40
#1 - Storage Pools
Easy Tier v3: Support for up to 3 Tiers
Support any combination of 1-3 tiers
Tier 0
Tier 1
Tier2
Flash/SSD
ENT
NL
Flash/SSD
ENT
NONE
Flash/SSD
NL
NONE
NONE
ENT
NL
Flash/SSD
NONE
NONE
NONE
ENT
NONE
NONE
NONE
NL

IBM.
41
#1 - Storage Pools - Example Only

Drive Selection in an Easy Tier Environment

IBM.
42
Distributed Storage: Standard Building Block

For MES to existing Storwize Virtual cluster or ESC designs
Intelligent ILM
recommended
w/auto-compression
Tunable based on historical or provided performance guidance

Virtualization
Layer
2145-DH8
SAN Volume Controller
TIER 0
FlashSystem 900
TIER 1
Using SVC
EasyTier
XIV GEN3 15 module w/SSD

Cache &
FlashSystem 900
TIER 2
XIV GEN3 15 module w/SSD
TIER 3
Using SVC
EasyTier
v5030 GEN2
1PB usable with 8 node cluster

Can go higher with historical performance stats
Optional: Flash only pools are not recommended

as you get bigger bang for buck with leveraging in
mixed pools w/ET. If required by client need then
you can use. Use subset of 10% in Flash only pool.
10% FlashSystem 900 (tier0)

50% XIV 2/3/4TB (tier1)
(Drive size based on capacity needs)
Optional: TIER2 pool with XIV only is optional. May
be good fit for larger environments leveraging 4 or
6TB drives
40% v5030
Easy Tier Pools
Up to six (6) 4TB NL-SAS RAID6 & one (1)
1.2TB 10k SAS arrays per pool (p 31)
IBM.
43
#1 Storage Pools Block Decision Tree

Non-mission Critical
data/appl/workload?
Client acceptable to off
premise Cloud use?
No
Yes
Spectrum Accelerate
on bare metal in
Softlayer See slides
in backup (40-46)
Enterprise On Premise below
Application Highly
Sensitive to IO
Response Time?
Yes
Virtualization Needed?
No
See slide 2 for SAN

strategy (DS8K, XIV)
Yes
No
Virtualization Needed?
Yes
SVC 4 or 8 node
Cluster Recommended
SVC 4 or 8 node
Cluster Recommended
No
Yes
TIER2 needed?
See TIER2b for v5k

Options or alternates
See slide 2/3 for SAN

strategy (XIV, v5k)
No
See Tier3 for v5k
Options or alternates
See pages extras for Guidance on Tier Performance Levels

If mission critical, SDS in Cloud is not approved

IBM.
44
#2 - IO Spread and Load Balancing across CPU Complexes

Even numbered extpools
Primary IO Data flow
ownership
Odd numbered extpools

Primary IO Data flow
ownership
Balance
Just like an onion -virtualization has many layers
45
Arrays
across
Enclosures
IBM.
45
45
#3 - IO Spread and Load Balancing across DA Pairs

Quiz Which has better throughput?
Unbalanced I/O to DA Cards
DA_0
DA_1
DA_2
DA_3
DA_4
DA_5
DA_6
DA_7
S1=Array_
0
S3=Array_
10
S5=Array_
S2=Array_
5
S4=Array_
14
S6=Array_
44
S7=Array_
52
S9=Array_
18
S11=Array_
48
S8=Array_
56
S10=Array_
22
S12=Array_
26
S13=Array_
34
30
S14=Array_
37
S15=Array
_1
S17=Array_
11
S19=Array_
S16=Array
_6
S18=Array_
15
S20=Array_
45
S21=Array_
22
S23=Array_
19
S25=Array_
49
S22=Array_
57
S24=Array_
23
S26=Array_
27
S27=Array_
35
S29=Array_
31
S28=Array_
38
S30=Array_
40
S31=Array
_2
S33=Array_
42
S32=Array
_7
S34=Array_
12
S35=Array_
46
S37=Array_
16
S36=Array_
50
S38=Array_
54
S39=Array_
20
S41=Array_
58
S40=Array_
24
S42=Array_
28
S43=Array_
36
S45=Array_
32
S44=Array_
39
S46=Array_
41
S47=Array
_3
S49=Array_
43
S48=Array
_8
S50=Array_
13
S51=Array_
47
S53=Array_
17
S52=Array_
51
S54=Array_
55
S55=Array
_4
S57=Array_
59
S56=Array
_9
S58=Array_
21
S59=Array_
29
25
S60=Array_
33
DA_0
DA_1
DA_2
DA_3
DA_4
DA_5
DA_6
DA_7
Balanced I/O to DA Cards

DA_0
DA_1
DA_2
DA_3
DA_4
DA_5
DA_6
DA_7
S1=Array_
0
S3=Array_
2S5=Array_
S2=Array_
1
S4=Array_
3S6=Array_
4
S7=Array_
6
S9=Array_
8
S11=Array_
5
S8=Array_
7
S10=Array_
9
S12=Array_
10
S13=Array_
12
11
S14=Array_
13
S15=Array_
14
S17=Array_
16
S19=Array_
S16=Array_
15
S18=Array_
17
S11=Array_
18
S21=Array_
20
S23=Array_
22
S25=Array_
19
S22=Array_
21
S24=Array_
23
S26=Array_
24
S27=Array_
26
S29=Array_
25
S28=Array_
27
S30=Array_
28
S31=Array_
30
S33=Array_
29
S32=Array_
31
S34=Array_
32
S35=Array_
34
S37=Array_
33
S36=Array_
35
S38=Array_
36
S39=Array_
38
S41=Array_
37
S40=Array_
39
S42=Array_
40
S43=Array_
42
S45=Array_
41
S44=Array_
43
S46=Array_
44
S47=Array_
46
S49=Array_
45
S48=Array_
47
S50=Array_
48
S51=Array_
50
S53=Array_
49
S52=Array_
51
S54=Array_
52
S55=Array_
54
S57=Array_
53
S56=Array_
55
S58=Array_
56
S59=Array_
58
57
S60=Array_
59
DA_0
DA_1
DA_2
DA_3
DA_4
DA_5
DA_6
DA_7

IBM.
46
#4 - IO Spread and Load Balancing across Disk Enclosures
Array 1
Array 2
Array 3

IBM.
47
#5 - IO spread and Load Balancing across ports

Examples of correct Host to SVC Volume Balancing
vdisk3
vdisk1
vdisk4
vdisk2
Preferred path for vdisk1 is

SVC N1P2 & N1P3
Preferred path for vdisk2 is

SVC N2P2 & N2P3
Non Preferred path for vdisk1

is SVC N2P2 &N2P3
Non Preferred path for vdisk2

is SVC N1P2 &N1P3
vdisk1
vdisk2
vdisk3
vdisk4

IBM.
48

Quiz - Which has better throughput?
SAN
Fabric
SAN
Fabric
SAN
Fabric
SAN
Fabric

IBM.
49

Quiz - XIV example Right or Wrong?

IBM.
50
Bottlenecks in the Host Design

The Top 5 common things that are or go wrong with Host Best Practices

IBM.
51
#1 - Using the right Host Multipath Drivers/ Storage Microcode

changes
Severity
Why the error has
occurred
Potential Risks
Actions to correct the

error
Warning
Microcode is below minimum targets
Lower levels of microcode susceptible to problems inherent to that level.

If hardware support is engaged for any reason, the first recommended
course of action will be to upgrade the microcode
Lower code levels will prolong PD/PR activities
Perform microcode update pursuant to Vendor specific support
recommendation of level
Test to verify server redundancy on each server attached to SAN prior to
any concurrent microcode upgrades
Coordinate microcode upgrades with storage Vendors, Account and
Storage team
SAs, CE's and Storage Admins should jointly monitor and coordinate
microcode upgrades

IBM.
52
#1 - Using the right Host Multipath Drivers/ Storage Microcode

-Things that change - Create a script can show current status
For AIX VIO - driver used - sddpcm
https://w3-connections.ibm.com/wikis/home?lang=enus#!/wiki/Global%20Server%20Management%20Distributed%20SL/page/Script%20to%20Capture%20SAN%20Path%20Info
Multipath installed = Yes, Set to: 4 paths/ hdisk, fcsi_settings:2145: fast_fail,
Multipath Policy =load_balance
For Linux/ESX/VMWare
https://w3-connections.ibm.com/files/app#/file/1ba027ec-5d20-4c60-a281-f18f16192f7a
Device mapper multipath HBA elements=4,
For Windows driver used = MPIO=SDDDSM
https://w3-connections.ibm.com/files/app#/file/3e52f54c-a445-4b17-aa5d-a5da43d4bedb
Multipath installed = Yes, HBA elements = 4, MPIO Policy = Optimized
For Solaris driver MPxIO
Https://w3-connections.ibm.com/files/app#/file/66ea3228-4b26-48bd-a8fd-55751a02fc42
Multipath installed = MPxIO, Path Subscription= 4, MPIO Policy = round-robin
Content input from : Bill Marshall, Jason Moras, Brad Worthen, Ramesh Palakodeti
53

IBM.
53
#1 - SDD Driver Testing for Proper HBA Failover

On an AIX vio server, check the AIX system to verify IO activity still continues
on the alternate ports by testing with SDD and /or SDDPCM commands
The Server Admin - Create a mount point for logical volume that can be manipulated to generate IO
traffic for the purpose of this test
The Server Admin - Verify and record selected (targets yet to be determined) datapaths for
preferred and alternate status (active and inactive) by using the SDD "pcmpath query device" or
"datapath query device" command on the AIX vio server
Note the path selection counts on the multiple paths. There should only be two paths under the
"Select" column, above zero (0). These are the two open paths on the preferred node. (if paths 0
and 2 show numbers under the "Select " column, other than zer0, then do the following:
1.
2.
3.
4.
5.
Take one path off-line by issuing the command (pcmpath set device 0 path 0 offline) or (datapath set device 0 path 0 offline) Path 0 should now be in a dead state .
Go to the mount point of lv and edit a file to create traffic. After creating the traffic, reissue the pcmpath or datapath query
command "pcmpath query device" or "datapath query device" and look at the path selection numbers. Notice only path selection
count for Path 2 increased for the other preferred path
Close Path 2 by issuing the command "pcmpath set device 0 path 2 offline" or "datapath set device 0 path 2 offline"
Return to the mount point and add or edit files to create IO.
Execute the "pcmpath query device" or "datapath query device command, to look at the path selection count. Disk access should
now be via the other paths. (This is now load balancing to the nonpreferred SVC node for this Vdisk)
Reestablish both preferred paths by executing the following commands: "pcmpath set device 0 path 0 online" and "pcmpath
set device 0 path 2 online" or "datapath set device 0 path 0 online" and "datapath set device 0 path 2 online

IBM.
54
#1 - Non SDD Driver Testing for Proper HBA Failover
Further "proper" testing should be done during a maintenance window
Testing the redundancy between the Fabric and the host

1.
2.
Open a change record to reflect the change (Make sure all necessary approvers are notified)
The Server Admin - Identify and verify which host HBA's are active for I/O activity by performing a test read and write to the
SAN disk from the host
3. The Server Admin - Stop I/O between the host and the Disk Storage
4. The SAN Admin - On the San Fabric, disable the Switch port on the "even" fabric zoned between the host and the storage
device.
5. The Server Admin - Perform another read/write test to the same LUN
6. The Server Admin - Identify and verify which host HBA is active for I/O activity
7. The SAN Admin - On the even SAN fabric enable the Switch port
8. The SAN Admin - On the San Fabric, disable the Switch port on the "odd" fabric zoned between the host and the storage
device.
9. The Server Admin - Perform another read/write test to the same LUN
10. The Server Admin - Identify and verify which host HBA is active for I/O activity
11. The SAN Admin - the odd SAN fabric enable the Switch port
If the I/O activity toggles between the two HBA's then the test is successful
When a new Host server or Storage device is added to the environment testing is strongly
recommended
Note: Ideally this type of test is best done during the initial implementation of new
equipment, before it is turned over to the customer or placed in production

IBM.
55

#2 - Random IO Data layout
1
What does random LV creation order, help

prevent?
1
datavg
4
5
# mklv lv1 e x hdisk1 hdisk2 hdisk5

# mklv lv2 e x hdisk3 hdisk1 . hdisk4
..
Use a random order for the hdisks for each LV
RAID array
LUN or logical disk
PV
Slide Provided by Dan Braden

IBM.
56

#2 - Data Layout - OS Spreading versus Striping
Is there is a difference? Whats the diff?
Do you know what your volumes made of!
File system spread

IBM.
57

#2 - Data Layout Summary
Does data layout affect IO performance more than any tunable IO parameter?
Good data layout avoids dealing with disk hot spots
An ongoing management issue and cost
Data layout must be planned in advance
Changes are generally painful
iostat might and filemon can show unbalanced IO

Best practice: evenly balance IOs across all physical disks unless TIERING
Random IO best practice:

Spread IOs evenly across all physical disks unless dedicated resources are needed to
isolate specific performance sensitive data
For disk subsystems
Create RAID arrays of equal size and type

Create VGs with one LUN from every array
Spread all LVs across all PVs in the VG

IBM.
58
58
#3 & 4- Tips Most Common OS IO Tuning Parameters

Device Queue Depth
Queue Depth can help or hurt performance per LUN
aware
Queueof
Depth
is central
to the
following
fundamental
performance
Be
Queue
Depth
when
planning
system
layout,formula:
adjust only if necessary
calculate
IO Rate
= Number
of is
Commands
Response
Time per
Command
To
- best
thing to do
go to each *device
Information
Center
URLs listed in link slide
For
example:
What are the default Queue Depths? ___
IO Rate = 32 Commands per Second / .01 Seconds (10 milliseconds) per Command = 3200 IOPs
Some real-world examples:
OS=Default Queue Depth= Expected IO Rate

adapters
HBA transfer rates

FC
AIX Standalone = 16 per LUN = 1600 IOPs per LUN

AIX VIOS
= 20 per LUN = 2000 IOPs per LUN
AIX VIOC
= 3 per LUN = 300 IOPs per LUN
Windows
= 32 per Disk = 3200 IOPS per LUN
LVM striping vs spreading

Data Placement
Random versus sequential
Spreading versus Isolation
Content provided by Mark Chitti
What are the most common/Important OS I/O tuning parameters?


IBM.
59
#5 - Workload IO increases due to Data Growth
Should you ever stripe with previrtualized volumes?

We recommend not striping or spreading in SVC, V7000 and XIV Storage Pools
Avoid LVM spreading with any striped storage pool
You can use file system striping with DS8000 storage pools
Across storage pools with a finer granularity stripe

Within DS8000 storage pools but on separate spindles when volumes are created sequentially
i.e (when the pools are created using the rotateext vs.rotatevol parameter
Striped Pools
No Host Stripe
Sequential Pools
Host Stripe
S
t
r
i
p
e
Host Stripe
Host Stripe - Raid-0 only

IBM.
60
#5 - Workload IO increases due to Data Growth

- careful tracking helps Data placement in Pools
On the host server using SDD- Mapping Virtual LUNS to Physical Disks
DEV#: 81 DEVICE NAME: hdisk81 TYPE: 2145
SERIAL: 60050768019002F4A8000000000005C7
ALGORITHM:
Load Balance
======================================================================
Path# Adapter/Path Name
State
Mode
Select
Errors
0
fscsi0/path0
FAILED NORMAL
89154
2
1*
fscsi0/path1
FAILED
NORMAL
63
0
2
fscsi1/path2
OPEN
NORMAL 34014
3
3*
fscsi1/path3
OPEN
NORMAL
77
0
LUN
to
Pool
to
Array
Ask the StoAdmin to find disk/device UID or Raid-group in Storage Pool
StorAdmin cross-references Storage Pool UID with Controllers Arrays in Pools

IBM.
61
Summary
Knowing - what's inside will help you make informed decisions?

You should make a list of the things you dont know
Talk to the Storage Administrator or those who do know
A better Admin understands

1.
2.
3.
4.
The backend physical makeup

The backend virtual makeup
What's in a Storage Pool for better data placement
Avoids the Pitfalls associated with IO Tuning
5. Know where to go to get right multipathing device drivers

6. Knows why documentation matters
7. Keep Topology Diagrams
8. keep Disk Mapping documentation
9. Be able to use Storage Inquiry Tools to find answers
10. Understand how to troubleshoot storage
performance bottlenecks

IBM.
62
62
Questions-

IBM.
63
Session Evaluations
YOUR OPINION MATTERS!
Submit four or more session

evaluations by 5:30pm Wednesday
to be eligible for drawings!
*Winners will be notified Thursday morning. Prizes must be picked up at
registration desk, during operating hours, by the conclusion of the event.

IBM.
64
EXTRAS
Use a troubleshoot method to divide the problem in half each time

IBM.
65
Consistent Troubleshooting Methodology

IBM.
66
#1 - Storage Pools Tiered Storage Classification
COST / PERFORMANCE / AVAILABILITY
TIER
TIER0
Flash
Systems
Preferred
Solid State
drives
alternate
TIER1(a)
TIER1(b)
Description
Ultra High
Performance.
Meet QoS for
High End
Technical Examples High level Guidance Local variations on technology exist
IBM (block) SVC Recommended for Open
900 FlashSystem RAID5

-- Excellent Turbo TIER1 when
coupled with XIV GEN3 (SSD cache)
-- V9000 not approved for use --
High Performance.
Drive up utilization of
high-ended storage
subsystems and still
maintain
performance QoS
objectives.
For low capacity

requirements smaller
less powerful
devices may meet
tier definition
Fibre Channel Only

for block
DS8886 w/SAS 600GB 15K disk drive

RAID5/RAID6 arrays ***
-300GB 15K to be used only if DISK Magic shows
need
DS8884 can be used for smaller mainframe solutions
50TB approx. demark
-- RAID6 should be seriously considered for
improved resiliency --- Mainframe only. See slide 5 & 6 for more
detailed guidance
XIV* (GEN3) model 214 with 2TB SAS drives

(11 Module or greater)
XIV* (GEN3) model 214 with 3TB/4TB SAS
drives (11 Module or greater)
-- SSDs (solid state drives) required --- XIV GEN2 removed from strategy --- XIV is a lower cost option. DS8
recommended for mainframe. Exception
based only for distributed envs. --
Performance range capability
DS8886 -> Greater than 250,000 IOPs/5500+

MBs
900 FlashSystem Greater than 500,000
IOPs mixed work load (70/30).
Greater than 14000 IOPs/TB (RAID 5)
DS8886 -> 200,000+ IOPs / 5000 MBs
Greater than 600 IOPs/TB in a full 1536 drives
configuration RAID 5
Greater than 1200 IOPs/TB in a full 1536 drives
configuration RAID 1
FAS8040 -> 1024 IOPS/TB
V9000 -> TBD

XIV 2TB
XIV (GEN3 15 mod) less than 130,000
IOPs/3,400 MBs
Around 1000 IOPs/TB
IOPs/3,200 MBs
Less than 1000 IOPs/TB
XIV 3TB
IOPs/3,400 MBs Less than 700 IOPs/TB
IOPs/3,200 MBs Less than 500 IOPs/TB

IBM.
67
RAID5 not recommended on drives greater than 900GB
Tier rating is based on performance AND reliability
#1 - Storage Pools Tiered Storage Classification
TIER
Description
Medium
Performance.
Meet QoS for
applications/data
that resides here.
TIER2
For low capacity

requirements
smaller less
powerful devices
may meet tier
definition
Fibre Channel
Only for block

IBM (block) SVC Recommended
for Open/DS8K Recommended
Direct Attached (NAS) strategy
For Mainframe
DS8886 w/SAS 600GB 15k disk
drive
RAID5/RAID10 arrays -Mainframe only. See slide 5 &
6 for more detailed guidance
v5030 w/SAS 600GB 15K using
RAID5 and SSD Easy Tier
See slide 29/30/31
-- V5020 acceptable for 3rd
location for Quorum disk for
ESC w/600GB SAS or for
small remote locations
-- V5010 not approved
-- V7000 remove from strategy
FAS8040
3TB SATA
1TB FlashCache/Node
Increments
- Min Controllers - 2
- Min Disks/Shelves 2xDS4246 1
p/node
- Max shelves per node before new
node 13
FAS2552 remote office use
1TB SATA
Min Controllers 2
Min Disk/Shelves 2
Max Shelves - full
DS8886 -> less than 80,000 IOPs or

less than 3,000 MBs
Less than 600 IOPs/TB in a full 1536
drives configuration RAID 5
v5030 Gen2 (block) -> less than

75,000 IOPs or less than 2,500 MBs
FAS8040 -> 256 IOPS/TB
SoNAS & V7000U not supported
Medium
performance/Clou
d and Commodity
Storage
TIER2b
Fibre Channel
Only for block
minus SDS. SDS
will be iSCSI
SDS Block Cloud

Spectrum Accelerate. See
best practices (and slide later in
deck). 50TB min. If smaller
needed use Endurance or
Performance offering from SL.
XIV* (GEN3) model 214 with
6TB SAS drives (11 Module or
greater unit). 800 GB SSDs
required for CACHE
File Interface
SDS File Offering
Quantastor 800GB SSD drives
running RAID5. See Quantastor best
practices.
Object Interface
SwiftStack
SoftLayer Object Workload and
storage both in SoftLayer
File + Object Interface
Swiftstack w/TwinStrata
SoftLayer NAS and Object
Workload and storage in SoftLayer
XIV -> less than 100,000 IOPs or less

than 2,500 MBs
Less than 400 IOPs/TB (XIV Gen3 4TB
11 mod)
QS SDS Block -> less than 50,000
IOPs or less than 1,000 MBs
SA SDS Block -

IBM.
68
#1- Storage Pools Tiered Storage Classification

TIER
Description
Low Performance.
Meet QoS for
applications/data
that resides here.
TIER3
Fibre Channel Only

for block
Low Performance.
Meet QoS for
applications/data
that resides here.
TIER3b
Fibre Channel Only
for block minus
SDS. SDS will be
iSCSI

IBM (block) SVC
Direct Attached (NAS) strategy
Recommended for Open
FAS8040
DS8886 with NL-SAS tech
DS8886 -> less than 25,000 IOPs or
3TB SATA
using RAID6 when customer
less than 1,000 MBs
NO FlashCache
already has DS8K with room
for this storage -- Mainframe Increments
only. See slide 5 & 6 for
-Min
Controllers
2
more detailed guidance
-Min Disks/Shelves 2xDS4246 1
p/node
v5030 model with 4 TB, or 6
v5030 (block) -> less than 30,000
-Max shelves per node before new
TB NL-SAS using RAID-6,
IOPs or less than 300 MBs
node 50
gen2 only. Drive size
selection based on I/O
FAS2520 remote office use
density and capacity
3TB
SATA
requirements.
Include 1.2TB drives and use Min Controllers 2
Min Disk/Shelves 2
SVC EasyTier for this TIER3
Max Shelves - full
pool
See slide 29/30/31
SoNAS & V7000U not supported
SDS Block Cloud

Spectrum Accelerate. See
best practices (and slide later
in deck). 50TB min. If smaller
needed use Endurance or
Performance offering from
SL.
SDS File Offering

Quantastor 4TB or 6TB drives
running RAID10. See Quantastor
best practices. Low end use only.
SDS & File Block: up to 35K and 280

MB/sec and when configured as
specified
SDS Object Offering

SwiftStack
SDS Object: Performance is high

configuration dependent and measured
in GETs and PUTs.
SoftLayer Shared Object

Workload and storage both in
SoftLayer
Object + Gateway
Swiftstack w/TwinStrata
SoftLayer Shared NAS and Object
Workload and storage in SoftLayer
TIER4
Archival, long term

retention, backup
Virtual Engines, Tape ATLs,

ProtecTIER
Less than 25,000 IOPs or less than

300MB/sec
drives configuration RAID 5 Quantastor
4 TB
drives configuration RAID 1 Quantastor
4 TB
N/A Tier based on features.

IBM.
69
Spectrum Virtualize (SVC) ESC Best Practice

Use Case example
Host Server
Host Server
Clustered Servers
where applicable
Node 1
Mission Critical vols
Node 2
Example 1 Stretched I/O Group
SVC Cluster 3
SVC Cluster 0
Site-1
Storage Pool 1
Site-2
SVC Global Mirror
Node 1
Node 2
ASYNC
Quorum
DR Site
Storage Pool 2
Storage Pool 3
B COPY
C COPY
Mission Critical Block for Applications

HA Storage: can sustain .3-1 ms latency hit)
DR, Dev, Test, QA .. etc.

IBM.
70
Spectrum Virtualize (SVC) ESC Site-1 & Site-2 current building block example
Example Use Case Scenario description:
Public VSAN
Public VSAN
3X /
node
Nodes are stretched across sites to make an SVC

Enhanced Stretch Cluster (ESC) for High Availability
(HA) application requirements
3X /
node
Private VSAN
Private VSAN
SVC DH8/4nodes /
3HBAs @ 8Gb
Cisco MDS
9710 Core
2X
2X
A Core Private Cisco VSAN is configured for Node to

Node Communication port connections
2X
2X
Node to
node
A Core Public enterprise class Cisco VSAN is configured

for Host/Storage port connections
Cisco MDS
9710 Core
2X
2X
12 Port Nodes are used with 4 ports each at 8Gb port

speeds
Node to
node
2X
2X
Edge Switches per port count requirements are used for

a true BP Core-Edge design to allow future growth
scalability
GM
4X/Flash, 6X/XIV, 4X/V7K
4X/Flash, 6X/XIV, 4X/V7K
GM
4x ISL / Edge
4x ISL / Edge
Storage building blocks of Flash, XIV and V7000 are used

for a 3 tier or mixed tier approach
9396 Edge x 3
9396 Edge x 3
Host Servers
1X / host
Separate Storage zones are created for each device

providing isolated zones from each device type
Host Servers are connected to the Edge Switches
1X / host
Edge Switches are ISLed to the Core Public VSANs

Servers
1X / SVC Node
1X / SVC Node
FCIP Router 9250i

FCIP Router 9250i
IBM Systems Technical
Events | ibm.com/training/events
FCIP Routers are used for GM to the DR site. SVC GM

ports are plugged directly into the FCIP Routers
IBM.
71
Spectrum Virtualize (SVC) ESC DR site current building block example

16X (2x/ SVC Node)
FCIP Router
9250i
GM
GM
Example Use Case Scenario

description:
FCIP Router
9250i
Cisco MDS
9710 Core
Cisco MDS
9710 Core
Public
VSAN
Private
VSAN
3x /
nod
e
3x /
nod
e
2
X
Public
VSAN
Private
VSAN
A Core Private Cisco VSAN is configured for Node

to Node Communication port connections
SVC DH8
2
X
2
X
12 Port Nodes are used with 4 ports each at 8Gb

port speeds
2
X
2
X
2
X
2
X
2
X
2
X
2
X
2
X
2
X
2
X
Edge Switches per port count requirements are

used for a true BP Core-Edge design to allow future
growth scalability
2
X
2
X
2
X
4x/Flash,
6x/XIV,
2x/V7K
Storage building blocks of Flash, XIV and V7000 are

used for a 3 tier or mixed tier approach
Separate Storage zones are created for each device
providing isolated zones from each device type
4x/Flash,
6x/XIV,
2x/V7K
8x ISL / Edge
8x ISL / Edge
Quorum
9396 Edge x 3
9396 Edge x 3
A Core Public enterprise class Cisco VSAN is

configured for Host/Storage port connections
Host Servers are connected to the Edge Switches

Edge Switches are ISLed to the Core Public VSANs
??? Servers
1X / host
1X / host
IBM Systems Technical Events

| ibm.com/training/events
Servers
FCIP Routers are used for GM to the DR site. SVC

GM ports are plugged directly into the FCIP Routers
IBM.
72
Spectrum Virtualize (SVC) ESC site-1 & site-2 synchronous with DR asynchronous
Putting it all together in the example template
2X @ 16Gb/s LW CWDM SFP
2X @ 16Gb/s LW CWDM SFP
Site1
Site2
FCIP GM
Priv Dark Fibre
Pub Dark Fibre
4X
4X
4X
4X
2X @ 16Gb/s CWDM LW SFP
2X @ 16Gb/s CWDM LW SFP
DR Site

IBM.
73
Continue growing your IBM skills
ibm.com/training
provides a comprehensive
portfolio of skills and career
accelerators that are designed
to meet all your training needs.
If you cant find the training that is right for you with our
Global Training Providers, we can help.
Contact IBM Training at dpmc@us.ibm.com
Global Skills Initiative

IBM.
74

Best Practice For Connecting IBM Storage To Power Systems-2016

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Best Practice For Connecting IBM Storage To Power Systems-2016

Caricato da

Copyright:

Formati disponibili

Best Practices for Performance, Design and Troubleshooting IBM

Storage Connected to IBM Power System

As Systems Administrators we dont always

Make more informed architectural decisions

What we are NOT going to do today:

Copyright IBM Corporation 2016. Technical University/Symposia materials

Throughput and Performance Key Optimization Factors

LVM striping vs spreading

Easy Tier - SSD

Random versus sequential

Spreading versus Isolation

Configuring throughput optimally increases potential performance scalability

Copyright IBM Corporation 2016. Technical University/Symposia materials

Foundation What causes performance degradation and Outages?

The 3 most common root causes are:

You should consider designing the environment to

You should design configurations to known Best

Copyright IBM Corporation 2016. Technical University/Symposia materials

Agenda -Troubleshooting as growth and change occur

Bottlenecks in the Physical Design

Bottlenecks in the SAN Configurations Design

Suboptimal zoning practices

Bottlenecks in the Storage Configurations Design

Rack and Stack Design

Data Layout - Storage Pools and Tiered pools - IO Spread vs IO isolation

Bottlenecks in Host Configurations Design

Using the right Host Multipath Drivers to match Storage Microcode

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

Bottlenecks in the physical design

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

Bottlenecks in Physical setup Rack and Stack Best Practices

A right way example

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

Bottlenecks in Physical setup Rack and Stack

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

Bottlenecks in Physical setup Rack and Stack

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

Bottlenecks in physical setup - Cooling #5

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

Copyright IBM Corporation 2016. Technical University/Symposia materials

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

Bottlenecks in the SAN Fabric - Best Practice

Suboptimal zoning practices

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

#1 Correct Zoning to Multiple Storage devices

Create separate zones for each Storage device

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

#1 - Incorrect Zoning to Multiple Storage devices

IBM Systems Technical Events | ibm.com/training/events

Copyright IBM Corporation 2016. Technical University/Symposia materials

DH8 1 Building Block 16 FC ports per node

Copyright IBM Corporation 2016. Technical University/Symposia materials