Sei sulla pagina 1di 104

Chuck Laing

Senior Technical Staff Member


IBM GTS Storage

IBM SVC Zoning Best Practices 101

Copyright IBM Corporation 2014


Technical University/Symposia materials may not be reproduced
in whole or in part without the prior written permission of IBM.

9.0

Please fill out an evaluation for


pSN561
ibmtechu.com

Some great prizes


to be won!

Copyright IBM Corporation 2014

Session objectives
By the end of this session you should know:
Zoning is one of the most important configuration for proper:

Failover
Redundancy
Resiliency
Performance

Understand best practice zoning for 4 port SVC models


Understand best practice zoning for 8 port SVC models CG8
Understand best practice zoning for 12 port SVC models DH8
Understand best practice zoning for Dual Core Fabric to SVC
Hopefully enjoyed learning Right and Wrong Zoning Scenarios by
playing along
Had Fun

Copyright IBM Corporation 2014

SVC Zoning Evolution

IBM SVC Zoning - Best Practices 101


A Closer Look

Copyright IBM Corporation 2014

Starting with CF8 Models


Physical to Logical evolution

Copyright IBM Corporation 2014

SVC CG8 with extra FC HBA


Physical to Logical evolution

Copyright IBM Corporation 2014

SVC CG8- Port Positioning


Physical to Logical evolution
Logical port #s reflecting the embedded WWPN
Blue ports go to one Fabric
Red Ports go to the other Fabric

Logical port with wwpn # embedded


Physical port number

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

SVC DH8 12 Ports


Physical to Logical evolution
DH8 - Physical to Logical Port numbering for existing clusters

Copyright IBM Corporation 2014

SVC DH8 12 Ports


Physical to Logical evolution
Understand the physical slot and port positioning
DH8 - Physical to Logical port numbering for new builds.

Copyright IBM Corporation 2014

Evolution and Types of Zones

IBM SVC Zoning - Best Practices 101


Backend/Storage to SVC Zoning
A Closer Look

Copyright IBM Corporation 2014

10

SVC CG8 8 Ports / DH8 12 Ports


Evolution and Types of Zones
Think of 4 types of zones per fabric (Use case - existing environment

FA = Fabric A; FB = Fabric B
SPx = Storage Port
NxPx = SVC Node x Port x
= include multiple remaining node ports the same way

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

Storage to SVC Zones 4 Port Node


Evolution and Types of Zones
Example Back-end Storage to SVC Zoning
DS8K Right I/O Enclosures

DS8K Left I/O Enclosures


Bay 2
C0 C1

Bay 0
C0 C1
L2

L4

L3

Bay 6

C0 C1

C0 C1

L4

L7

2 2 2 2 2 2 2 2
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

0 0 0 0 0 0 0 0
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

Bay 4
L8

L3

4 4 4 4 4 4 4 4
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

L7

6 66 6 6 6 6 6
3 33 3 0 0 0 0
0 12 3 0 1 2 3

Bay 1
C0 C1
R2

R4

1 1 1 1 1 1 1 1
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3

Bay 3

Bay 5

Bay 7

C0 C1

C0 C1

C0 C1

R3

R7

3 3 3 3 3 3 3 3
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3

R4

R8

5555 5555
0000 3333
0123 0123

R3

R7

7 7 7 7 7 7 7 7
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3

DIR2 SAN Fabric

DIR1 SAN Fabric

Supported DS8K to SVC Zoning


P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 1

HBA 2

Node 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

Node 4

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn
5005076801
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

HBA 2

4 Node SVC MAX Vdisk 4096

port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

port1=110529e
port2=120529e
port3=130529e
port4=140529e

Either ports 1&3 and 2&4 should be


zoned to a fabric or ports as shown on the
next page, both configs are supported
Newer SVC nodes may contain
one HBA card with 4 ports

port1=110528b
port2=120528b
port3=130528b
port4=140528b

Copyright IBM Corporation 2014

Storage to SVC Zones 8 Port Node


Evolution and Types of Zones
Example excluding heartbeat and MM/GM ports

SAN

Fabric 1

SAN

Copyright IBM Corporation 2014

Fabric 2

Storage to SVC Zones - 12 Port Node


Evolution and Types of Zones
Example excluding heartbeat and MM/GM ports

SAN

Copyright IBM Corporation 2014

Fabric 1

SAN

Fabric 2

Storage to SVC Zones 12 Port Node


Write Data Rate > 3GB/sec
Example Back-end Storage to SVC Zoning

SAN

Fabric 1

SAN

Copyright IBM Corporation 2014

Fabric 2

XIV Storage to SVC Zone Example


Storage Zone Type
Example XIV Storage to SVC Zoning Ports 1 and 3 only

Copyright IBM Corporation 2014

Flash Systems to SVC Zones


Storage Zone Type
Flash Systems

SAN

HBA1

HBA2

P P
1 2

P P
1 2

Fabric 1

SAN

Copyright IBM Corporation 2014

Fabric 2

V7000 Storage-SVC - 12 Ports


Storage Zone Type
2
4

1
1

CAUTION

2
1
3

2
1

Disconnect all
supply power for
complete isolation

Disconnect all
supply power for
complete isolation

1
3

2
4

1
3

2
1

2
4

CAUTION

CAUTION

SAN

2
4

Disconnect all
supply power for
complete isolation

2
4

1
3

2
4

1
3

CAUTION
1
3

Disconnect all
supply power for
complete isolation

Fabric 1

SAN

Fabric 2

Slide provided by Tron Pryor


Copyright IBM Corporation 2014

18

EMC Symmetrix Storage


to SVC zoning with 12 ports
IBM, EMC Clariion, HP, etc. use one WWNN per subsystem; each appears as a single
controller with multiple ports/WWPNs
Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of 1024
Rule of 17

D
i
r
1

D
i
r
2

D
i
r
3

D
i
r
4

D
i
r
5

D
i
r
6

D
i
r
7

D
i
r
8

F
E

F
E

F
E

F
E

B
E

B
E

F
E

F
E

SAN

S
l
o
t

S
l
o
t

S
l
o
t

S
l
o
t

S
l
o
t

S
l
o
t

S
l
o
t

S
l
o
t

1
0

1
1

1
2

1
3

1
4

1
5

1
6

1
7

D
i
r
9

D
i
r
1
0

D
i
r
1
1

D
i
r
1
2

D
i
r
1
3

D
i
r
1
4

D
i
r
1
5

D
i
r
1
6

F
E

F
E

B
E

B
E

F
E

Fabric 1

F
E

F
E

F
E

SAN

Slide provided by Tron Pryor


Copyright IBM Corporation 2014

Fabric 2

19

HDS USP Storage to SVC Zones


with 12 ports
HDS uses one WWNN per subsystem; each appears as a single controller with multiple
ports/WWPNs
Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of 1024
CL 1 - FRONT CHAs
CHPxx
Port-Label
MPID
HardID
Port-Label
00
00
1A / 5A
01
01
3A / 7A
02
02
1B / 5B
03
03
3B / 7B
04
04
1C / 5C
05
05
3C / 7C
06
06
1D / 5D
07
07
3D / 7D

SAN

Copyright IBM Corporation 2014

Fabric 1

P/K
LOC

2Q

1E

P/K
LOC

CL 2 - REAR CHAs
CHPxx Port-Label
MPID
HardID Port-Label
40
40
2A / 6A
41
41
4A / 8A
42
42
2B / 6B
43
43
4B / 8B
44
44
2C / 6C
45
45
4C / 8C
46
46
2D / 6D
47
47
4D / 8D

SAN

Slide provided by Tron Pryor

Fabric 2

Storage Zones Hitachi Example


Storage Zone Type
HDS uses one WWNN per subsystem; each appears as a single controller with multiple
ports/WWPNs
Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of 1024

Hitachi 9980 connected to SVC


Left Controller(1)
CHA P
1P 1Q
A B C D

Right Controller(2)
CHA P CHA P

CHA P
1R

F G H

J KL
L M

2V 2W

1S
N

A B C D

P Q R

F G H
F

01 02 03 04 05 06 07 08

Copyright IBM Corporation 2014

P Q R
P

ODD SAN Fabric

09 10 11 12 13 14 15 16

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

01 02 03 04 05 06 07 08

Even SAN Fabric

09 10 11 12 13 14 15 16

SVC

2X 2Y
J K L M

HBA 2

Node 1

HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

HBA 2

Node 4

Storage to SVC Zones DS4k/5K


Storage Zone Type
Server 2

Server 1

Best Practice

SVC Cabling
and Zoning

SAN Fabric A

SAN Fabric B

HOST ZONING

SVC ZONING

Create a SVC/Host zone


for each server that
receives storage from
the SVC cluster.
Example:

I/O G-0

Zone Server 1 port A (RED)


with all SVC node port 3's.

1
3

Zone Server 2 port A (RED)


with all SVC node port 1's.

Example:

Node 3
4

All odd (RED) SVC node


ports in one zone and all
even (BLUE) SVC node ports
in one zone.

2
Node 4

SVC
Cluster

Note: For a cluster to be


created and to operate
correctly all node ports must
be zoned together.

STORAGE
ZONING

MDisk Group 1 / DS5K_1


MDisk13 / Array13

MDisk12 / Array12

Copyright IBM Corporation 2014

MDisk11 / Array11

MDisk9 / Array9

MDisk10 / Array10

MDisk8 / Array8

MDisk7 / Array7

MDisk6 / Array6

MDisk5 / Array5

VDisk 2

MDisk4 / Array4

Channels 1
and 3

MDisk3 / Array3

VDisk 1
Cntrl A
Channels 2
and 4

MDisk2 / Array2

22

MDisk1 / Array1

To create more then 256


host objects in the cluster
you must zone a host to a
subset of the I/O groups,
you must assign the host
object at host creation time
to that same subset of I/O
groups and then you must
assign that hosts VDisks to
one of those I/O groups in
that same subset.

Node 2

Zone Server 2 port B (BLUE)


with all SVC node port 4's.
*** NOTE ***
SVC supports a maximum of
256 host objects per I/O
group thus a maximum of
1024 per cluster. The above
host zoning results in each
server being seen by every
I/O group and the default
host object creation
behavior results in each
host object counting as one
towards this 256 maximum.

I/O G-1
2

Node 1

Zone Server 1 port B (BLUE)


with all SVC node port 2's.

Create one zone in the


RED fabric with all the
SVC node ports cabled
to Fabric A and create
one zone in the BLUE
fabric with all the SVC
node ports cabled to
Fabric B.

Create a SVC/Storage
zone for each storage
subsystem virtualized
by the SVC cluster.

VDisk 3
Cntrl B
Channels 1
and 3

Example:

Channels 2
and 4
VDisk 4

Zone DS5K_1 controller A


and B daughter card
channel ports 2 and 4 with
all SVC node ports 2 and 4
in the BLUE fabric.

Zone DS5K_1 controller A


and B daughter card
channel ports 1 and 3 with
all SVC node ports 1 and 3
in the RED fabric.

Types of Zones
Host to SVC Zoning Evolution

IBM SVC Zoning - Best Practices 101


Host Zones a Closer Look

Copyright IBM Corporation 2014

23

Host Zones Evolution 4 Port Node

Copyright IBM Corporation 2014

Host Zones evolution 8 Port Node

Copyright IBM Corporation 2014

Host Zones Evolution 12 Port Node


SVC - DH8

Correct
Host
A1

Fabric1 Core1

B1

Fabric2 Core1

Copyright IBM Corporation 2014

Zoning Multi HBA hosts for


Resiliency
Right way to zone a multiport Host
Make four zones (Red, Blue, Orange and Green zones)
Spread IO across

multiple SVC Ports


Multi Port Host

P
1 2

SAN

HBA1
P P
1 2

Fabric 1

HBA2
P P
1 2

SAN

Copyright IBM Corporation 2014

Fabric 2

Zoning Multi HBA hosts for


Redundancy and Resiliency
Sys Admins provide PCI slot to Port WWPN identity to Storage Admins
Storage Admins define the SVC host definitions to match
Avoid single points of hardware failure at the Host HBA, Fabric and SVC
Make four zones, one for each sudo host per fabric(Red, Blue, Orange and Green zones)

SVC defined Sudo Host1


HBA1
P
1

SAN

HBA2
P P
1 2

Physical Host
HBA1
P P
1 2

Fabric 1

HBA2
P P
1 2

SAN

Copyright IBM Corporation 2014

SVC defined Sudo Host2


HBA1
P P
1 2

HBA2
P
1

Fabric 2

Zoning Multi HBA hosts for


Resiliency
Sys Admins provide PCI slot to Port WWPN identity to Storage Admins
Storage Admins define the SVC host definitions to match
Avoid single points of hardware failure at the Host HBA, Fabric and SVC
Make four zones, one for each sudo host per fabric(Red, Blue, Orange and Green zones)

SVC defined Sudo Host1


HBA1
P
1

SAN

HBA2
P P
1 2

Physical Host
HBA1
P P
1 2

Fabric 1

HBA2
P P
1 2

SAN

Copyright IBM Corporation 2014

SVC defined Sudo Host2


HBA1
P P
1 2

HBA2
P
1

Fabric 2

Types of Zones - Host to SVC


AIX VIOS to SVC Zoning Concepts

IBM SVC Zoning - Best Practices 101


A Closer Look

Copyright IBM Corporation 2014

30

VIOS Simplistic Dual Path Overview

Copyright IBM Corporation 2014

N_Port ID Virtualization

c0:50:76:07:0f:5f:00:30

c0:50:76:07:0f:5f:00:20

Copyright IBM Corporation 2014

Host Zones
Best Practices Concepts
Planning and design for AIX Live Partition Mobility (LPM)
Create two separate and isolated zones on each fabric for each LPAR
Do not put both the active and inactive LPAR wwpns in either the same zone or
same SVC host definition

Create Sudo host SVC host definitions containing only two virtual
wwpns, one from each fabric
Follow the rules shown in the next few slides

Map LUNs to the virtual host FC HBA port wwpns not he physical host
FCA Adapter wwpn
Exception to map only boot from SAN LUNs to the physical host FC HBA wwpn
Share the Vdisks to both LPARs (both the active and inactive) to enable LPM to
see the LUNs in both LPARs
Note: Following these rules will result in not having a degraded path error on the
SVC, and keep the 4 paths per Vdisk rule

Copyright IBM Corporation 2014

Dual VIOS to Single LPAR Example

VIO Server1

VIO Server2
4

H
B
A
1

P1

P3

P2

P4

VP1

VP3

H
B
A
2

VHBA2

VHBA1

VP4

VP2

SVC

Copyright IBM Corporation 2014

Dual VIOS to Multiple LPARs


Zoning for Resiliency
VIO Server2

VIO Server1

VP1.1
Sudo 1

P3

P1
HBA1

SAN

HBA2

P2

P4

VP1.1
VP1.2

VP3.1

VP2.2
P3 P8

Client
Logical
Partition
(LPAR1)

Sudo 2

VP2.1

VP3.2
VHBA2

VHBA1

VP2.1

VP4.1
VP3.1

SVC

VP4.1
VP4.2
P3 P8

VP1.2
Sudo1

VP4.2
VP3.2
Sudo2

VP2.2

Hypervisor
Copyright IBM Corporation 2014

Client
Logical
Partition
(LPAR2)

Dual VIOS to Multiple LPARs


One VIOS Failure
VIO Server2

VIO Server1

P3

P1
HBA1

SAN

P4

VP1.1
VP1.2

VP3.1
VP3.2

P3 P8

VP4.1
VP3.1
Sudo 2

Client
Logical
Partition
(LPAR1)

VP2.1

VHBA2

VHBA1

VP2.2

Sudo 1

HBA2

P2

VP2.1

VP1.1

SVC

VP4.1
VP4.2
P3 P8

VP1.2

xSudo 1

VP4.2
VP3.2
Sudo 2

VP2.2

Hypervisor
Copyright IBM Corporation 2014

Client
Logical
Partition
(LPAR2)

Dual VIOS to Multiple LPARs


One SAN Fabric Failure
VIO Server2

VIO Server1

P3

P1
HBA1

SAN

HBA2

xP2

P4
x

VP1.1
VP1.2

VP3.1
VP3.2

VP2.2
P3 P8

Sudo 1

VP4.1
VP3.1
Sudo 2

Client
Logical
Partition
(LPAR1)

VP2.1

VHBA2

VHBA1

VP2.1

VP1.1

SVC

VP4.1
VP4.2

P3 P8

VP1.2
Sudo 1

VP4.2
VP3.2
Sudo 2

VP2.2

Hypervisor
Copyright IBM Corporation 2014

Client
Logical
Partition
(LPAR2)

Types of Zones
Host ESX to SVC Zones

Copyright IBM Corporation 2014

Types of Zones
Host ESX to SVC Zones
2+2 =4 Paths per LUN

Copyright IBM Corporation 2014

Types of Zones
SVC Node to Node / Heartbeat Zones

IBM SVC Zoning - Best Practices 101


A Closer Look

Copyright IBM Corporation 2014

40

Node to Node / Heartbeat Zones


The Intra-cluster/Heartbeat/node to node communication zone
Make the local_fc_port_mask (Intra-cluster) = 000011000000 for 12 ports

Only make the local_fc_port_mask (Intra-cluster) = 110011000000


if Write Data Rate > 3GB/sec per Iogrp

In dual core fabrics, all dedicated heartbeat ports should reside on only one core

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

Types of Zones
Metro Mirror / Global Mirror Zones

IBM SVC Zoning - Best Practices 101


MM/GM - A Closer Look

Copyright IBM Corporation 2014

42

Mirror Zones Evolution


MM/GM and Heartbeat 4 Port Nodes

One SVC port from each node on each fabric should be zoned for GM traffic, taking care not to include both ports that a
server might use. This means ports that would also be used for replication would be either ports 1 & 2, or ports 3 & 4
For each node in a cluster, exactly two fibre channel ports should be zoned to exactly two fibre channel ports from each node
in the partner cluster.
If dual-redundant ISLs are available, then the two ports from each node should be split evenly between the two ISLs, i.e.
exactly one port from each node should be zoned across each ISL.
Local cluster zoning should continue to follow the standard requirement for all ports on all nodes in a cluster to be zoned to
one another.
This is discussed more verbosely on the Flash published for this issue on the IBM website:

http://www-01.ibm.com/support/docview.wss?uid=ssg1S1003634 and https://www-304.ibm.com/support/docview.wss?uid=ssg1S1003634

Copyright IBM Corporation 2014

Mirror Zones Evolution


SVC GG8 8 Port Node
Make the local_fc_port_mask (Node to Node/heartbeat) = 11000000
Make the partner_fc_port_mask (MM/GM)
= 00110000

WAN direct connection to the


FCiP connection to the SVC ports
GM/MM zone A

Copyright IBM Corporation 2014

WAN direct connection to the


FCiP connection to the SVC ports
GM/MM zone B

Mirror Zones Evolution


SVC DH8 12 Port Node
Make the local_fc_port_mask(Intra-cluster)=000011000000 for 12 ports
Make the local_fc_port_mask(Intra-cluster)=110011000000 - Write Data Rate > 3GB/sec per
IOgrp

Make the partner_fc_port_mask (MM/GM) = 000000110000

WAN direct connection to the


FCiP connection to the SVC ports
GM/MM zone A

WAN direct connection to the


FCiP connection to the SVC ports
GM/MM zone B

Copyright IBM Corporation 2014

Planning Design example for correct


Dual Core fabric zoning

IBM SVC Zoning - Best Practices 101


Dual Core Design - A Closer Look

Copyright IBM Corporation 2014

46

Dual Core Fabric


Storage Back-end Problem Scenario
Problem: A single back-end zone had ports p1 and p2 from SVC 7 and port a3 from XIV (example). SVC 7 will form paths
from p1 to a3 (purple) and from p3 to a3 (green). Purple paths traverse 2 cores and 1 edge switch interfering with host
edge-to-core traffic. This issue is pervasive. Also, eliminate (unnecessary) separate heartbeat zone.
Fix: Split zones so that only common-core switch (green paths) can occur. Reduce paths, contention, latency. Also
group XIVn array ports attaching common core switch in common zone.

Copyright IBM Corporation 2014

Slide provided by Kirby Dahman

Dual Core Fabric


Host Front-end Problem Scenario
Problem: Many host HBA ports are in a single zone with SVC ports cabled to different core switches. This
does not hurt the host directly, but as a side-effect it permits SVC inter-node cache updates (after each host
write) from preferred to non-preferred node in an IOgroup to cross from core-switch-to-edge switch-to-core
switch (see purple path).
Fix: Split zones so that only common-core switch (green paths) can occur for inter-node cache updates.

Copyright IBM Corporation 2014

Incorrect Dual Core fabric zoning


example for a 2 node SVC Cluster

Copyright IBM Corporation 2014

Slide provided by John Locke

Correct Dual Core fabric zoning


example for a 2 node SVC Cluster

Copyright IBM Corporation 2014

Slide provided by John Locke

SVC Cluster zone for a


Single vs. Dual Core per fabric
A Single core cluster zone Brocade
Example
CLI View of one Fabric /8 node SVC

zone:

lsan_allsvc_zone_a
50:05:07:68:01:10:54:28
50:05:07:68:01:40:54:28
50:05:07:68:01:10:54:62
50:05:07:68:01:40:54:62
50:05:07:68:01:10:51:07
50:05:07:68:01:40:51:07
50:05:07:68:01:10:54:73

50:05:07:68:01:40:54:73
50:05:07:68:01:10:4e:c3
50:05:07:68:01:40:4e:c3
50:05:07:68:01:10:45:64
50:05:07:68:01:40:45:64
50:05:07:68:01:10:4f:67
50:05:07:68:01:40:4f:67
50:05:07:68:01:10:4f:66

50:05:07:68:01:40:4f:66

A Dual Core cluster zone Brocade


Example
CLI View of one Fabric /8 node SVC Fabric 1
Core1

zone:

lsan_allsvc_zone_a
50:05:07:68:01:10:54:28
50:05:07:68:01:10:54:62
50:05:07:68:01:10:51:07
50:05:07:68:01:10:54:73
50:05:07:68:01:10:4e:c3
50:05:07:68:01:10:45:64
50:05:07:68:01:10:4f:67
50:05:07:68:01:10:4f:66

CLI View of one Fabric /8 node SVC Fabric


Core2

50:05:07:68:01:40:54:28
50:05:07:68:01:40:54:28
50:05:07:68:01:40:54:62

50:05:07:68:01:40:51:07
50:05:07:68:01:40:54:73
50:05:07:68:01:40:4e:c3
50:05:07:68:01:40:45:64
50:05:07:68:01:40:4f:67
50:05:07:68:01:40:4f:66

Slide provided by Chuck Laing

51

Incorrect vs. correct Dual core fabric


Host to SVC port zoning
Right or Wrong?
Host
A1

Fabric1
Core1

Fabric 1
Core2

B1

Fabric2
Core1

Fabric 2
Core2

52

Incorrect vs. correct Dual core fabric


Host to SVC port zoning
Incorrect

Correct

Host
A1

Fabric1
Core1

Fabric 1
Core2

Host
A1

B1

Fabric2
Core1

Fabric 2
Core2

Fabric1
Core1

Fabric 1
Core2

B1

Fabric2
Core1

Fabric 2
Core2

53

Correct Dual core fabric / Host to


SVC port zoning
Right or Wrong?
Red and Blue cores have ISLs between cores respectively?
Host1
A1

Host2
A1

B1

Fabric1 Core1

Fabric 1 Core2

Fabric2 Core1

Fabric 2 Core2

43

12

56

78

43

12

56

78

43

12

56

78

43

12

56

78

00

00

00

00

00

00

00

00

00

00

00

00

00

00

00

00

12 34 56 78
HBA 1 HBA 2

12 34 56 78
HBA 1 HBA 2

Node 1

Node 2

12 34 56 78
HBA 1 HBA 2

12 34 56 78
HBA 1 HBA 2

Node 1

Node 2

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

4 Node CG8 SVC


Copyright IBM Corporation 2014

B1

Correct Dual core fabric / Host to


SVC port zoning
Correct

Correct

Host1
A1

Host2
A1

B1

Fabric1 Core1

Fabric 1 Core2

Fabric2 Core1

Fabric 2 Core2

43

12

56

78

43

12

56

78

43

12

56

78

43

12

56

78

00

00

00

00

00

00

00

00

00

00

00

00

00

00

00

00

12 34 56 78
HBA 1 HBA 2

12 34 56 78
HBA 1 HBA 2

Node 1

Node 2

12 34 56 78
HBA 1 HBA 2

12 34 56 78
HBA 1 HBA 2

Node 1

Node 2

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

4 Node CG8 SVC


Copyright IBM Corporation 2014

B1

Dual core fabric / Multiple Host - ESX


Cluster to SVC port zoning
Host_1b

Host_1a
A1

Fabric1 Core1

B1

A1

B1

Fabric 1 Core2

Host_2a
Host_
A1
A1

B1

Fabric2 Core1

Host_2b
A1

B1

Fabric 2 Core2

Slide provided by Chuck Laing


56

Dual core fabric / Multiple Host - ESX


Cluster to SVC port zoning
Host_1b

Host_1a
A1

Fabric1 Core1

B1

A1

B1

Fabric 1 Core2

Host_2a
Host_
A1

B1

Fabric2 Core1

Host_2b
A1

B1

Fabric 2 Core2

Slide provided by Chuck Laing


57

Dual core fabric / Multiple Host - ESX


Cluster to SVC port zoning
Host_1b

Host_1a
A1

Fabric1 Core1

B1

A1

B1

Fabric 1 Core2

Host_2a
Host_
A1

B1

Fabric2 Core1

Host_2b
A1

B1

Fabric 2 Core2

Slide provided by Chuck Laing


58

Dual core fabric / Multiple Host - ESX


Cluster to SVC port zoning
Host_1b

Host_1a
A1

Fabric1 Core1

B1

A1

B1

Fabric 1 Core2

Host_2a
Host_
A1

B1

Fabric2 Core1

Host_2b
A1

B1

Fabric 2 Core2

Slide provided by Chuck Laing


59

Dual core fabric / Multiple Host - ESX


Cluster to SVC port zoning
Zone a single HA/ESX cluster to the same core to avoid ISL IO traversing
Spread Multiple Host clusters evenly across Cores
Create separate zones for each Host node in the ESX cluster or AIX HA cluster

Copyright IBM Corporation 2014

Correct Dual Core Zoning to Multiple


Storage devices
Zone each device to only one core to avoid extra ISL IO traversing cutting
down ISL hops
Spread Multiple Storage devices as evenly possible across Cores

Create separate zones for each Storage device


DS8K_2

DS8K_1

Fabric1 Core1

Fabric 1 Core2

XIV

Fabric2 Core1

Copyright IBM Corporation 2014

V7000

Fabric 2 Core2

Correct Dual Core Zoning to Multiple


Storage devices
Zone each device to only one core to avoid extra ISL IO traversing cutting
down ISL hops
Spread Multiple Storage devices as evenly possible across Cores

Create separate zones for each Storage device


DS8K_2

DS8K_1

Fabric1 Core1

Fabric 1 Core2

XIV

Fabric2 Core1

Copyright IBM Corporation 2014

V7000

Fabric 2 Core2

Correct Dual Core Zoning to Multiple


Storage devices
Zone each device to only one core to avoid extra ISL IO traversing cutting
down ISL hops
Spread Multiple Storage devices as evenly possible across Cores

Create separate zones for each Storage device


DS8K_2

DS8K_1

Fabric1 Core1

Fabric 1 Core2

XIV

Fabric2 Core1

Copyright IBM Corporation 2014

V7000

Fabric 2 Core2

SVC DH8- Mirror Zones


Make the local_fc_port_mask (Node to Node/heartbeat)=000010001000
Make the partner_fc_port_mask (MM/GM
= 000001000100

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

Right or Wrong way Scenarios

IBM SVC Zoning - Best Practices 101


Lets Have Some Fun, with Right or Wrong

Copyright IBM Corporation 2014

65

Zoning Multi HBA hosts for


Resiliency
Right or Wrong?
Multiport Host
P
1 2

SAN

HBA1

HBA2

P P
1 2

P P
1 2

Fabric 1

SAN

Copyright IBM Corporation 2014

Fabric 2

Zoning Multi HBA hosts for


Resiliency
Is this better?
Multiport Host
P
1 2

SAN

HBA1

HBA2

P P
1 2

P P
1 2

Fabric 1

SAN

Copyright IBM Corporation 2014

Fabric 2

Zoning Multi HBA hosts for


Resiliency
Right way to zone a multiport Host
Make four zones (Red, Blue, Orange and Green zones)
Spread IO across

multiple SVC Ports


Multi Port Host

P
1 2

SAN

HBA1
P P
1 2

Fabric 1

HBA2
P P
1 2

SAN

Copyright IBM Corporation 2014

Fabric 2

Right or Wrong Game/Storage - SVC


Which example is Right?
1

2
DS8K Right I/O Enclosures

DS8K Left I/O Enclosures


Bay 2
C0 C1

Bay 0
C0 C1
L2

L4

L3

Bay 6

C0 C1

L4

L7

2 2 2 2 2 2 2 2
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

0 0 0 0 0 0 0 0
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

Bay 4

C0 C1
L8

L3

4 4 4 4 4 4 4 4
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

6 66 6 6 6 6 6
3 33 3 0 0 0 0
0 12 3 0 1 2 3

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

HBA 2

Node 4

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn 4 Node SVC MAX Vdisk 4096

5005076801
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

port1=110529e
port2=120529e
port3=130529e
port4=140529e

R2

R4

1 1 1 1 1 1 1 1
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3

Bay 5

Bay 7

C0 C1

C0 C1

R3

R4

R7

R8

5555 5 555
0000 3 333
0123 0 123

3 3 33 3 3 3 3
0 0 00 3 3 3 3
0 1 23 0 1 2 3

R3

port1=110528b
port2=120528b
port3=130528b
port4=140528b

A2

DS8K Right I/O Enclosures

DS8K Left I/O Enclosures

Bay 3

C0 C1

R7

7 7 7 7 7 77 7
0 0 0 0 3 33 3
0 1 2 3 0 12 3

Bay 2
C0 C1

Bay 0
C0 C1
L2

L4

L3

B2

Bay 4

Bay 6

C0 C1

C0 C1

L4

L7

2 2 2 2 2 2 2 2
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

0 0 0 0 0 0 0 0
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

DIR2 SAN Fabric

DIR1 SAN Fabric

HBA 1

L7

Bay 1
C0 C1

L8

L3

4 4 4 4 4 4 4 4
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

6 66 6 6 6 6 6
3 33 3 0 0 0 0
0 12 3 0 1 2 3

HBA 1

The same native


backend ports should not be
shared for both direct host connectivity
and SVC connectivity as shown here.
The correct way to bypass SVC is to use
other backend ports not connected to the
SVC

HBA 1

HBA 2

Node 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

HBA 2

Node 4

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn 4 Node SVC MAX Vdisk 4096

5005076801
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

Copyright IBM Corporation 2014

port1=110529e
port2=120529e
port3=130529e
port4=140529e

R2

R4

1 1 1 1 1 1 1 1
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3

Bay 3

Bay 5

Bay 7

C0 C1

C0 C1

C0 C1

R3

R4

R7

R8

5555 5 555
0000 3 333
0123 0 123

3 3 33 3 3 3 3
0 0 00 3 3 3 3
0 1 23 0 1 2 3

R3

R7

7 7 7 7 7 77 7
0 0 0 0 3 33 3
0 1 2 3 0 12 3

DIR2 SAN Fabric

DIR1 SAN Fabric

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4

Application Host Server

L7

Bay 1
C0 C1

port1=110528b
port2=120528b
port3=130528b
port4=140528b

A2

B2

Application Host Server

The same native


backend ports should not be
shared for both direct host connectivity
and SVC connectivity as shown here.
The correct way to bypass SVC is to use
other backend ports not connected to the
SVC

Right or Wrong Game/Storage - SVC


SVC Correct Example

SVC Incorrect Example

The same port on the back-end is being used for host and SVC

DS8K Right I/O Enclosures

DS8K Left I/O Enclosures


Bay 2
C0 C1

Bay 0
C0 C1
L2

L4

L3

Bay 6

C0 C1

L4

L7

2 2 2 2 2 2 2 2
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

0 0 0 0 0 0 0 0
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

Bay 4

C0 C1
L8

L3

4 4 4 4 4 4 4 4
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

6 66 6 6 6 6 6
3 33 3 0 0 0 0
0 12 3 0 1 2 3

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

HBA 2

Node 4

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn 4 Node SVC MAX Vdisk 4096

5005076801
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

port1=110529e
port2=120529e
port3=130529e
port4=140529e

R2

R4

1 1 1 1 1 1 1 1
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3

Bay 5

Bay 7

C0 C1

C0 C1

R3

R4

R7

R8

5555 5 555
0000 3 333
0123 0 123

3 3 33 3 3 3 3
0 0 00 3 3 3 3
0 1 23 0 1 2 3

R3

port1=110528b
port2=120528b
port3=130528b
port4=140528b

A2

DS8K Right I/O Enclosures

DS8K Left I/O Enclosures

Bay 3

C0 C1

R7

7 7 7 7 7 77 7
0 0 0 0 3 33 3
0 1 2 3 0 12 3

Bay 2
C0 C1

Bay 0
C0 C1
L2

L4

L3

B2

Bay 4

Bay 6

C0 C1

C0 C1

L4

L7

2 2 2 2 2 2 2 2
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

0 0 0 0 0 0 0 0
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

DIR2 SAN Fabric

DIR1 SAN Fabric

HBA 1

L7

Bay 1
C0 C1

L8

L3

4 4 4 4 4 4 4 4
3 3 3 3 0 0 0 0
0 1 2 3 0 1 2 3

6 66 6 6 6 6 6
3 33 3 0 0 0 0
0 12 3 0 1 2 3

HBA 1

The same native


backend ports should not be
shared for both direct host connectivity
and SVC connectivity as shown here.
The correct way to bypass SVC is to use
other backend ports not connected to the
SVC

HBA 1

HBA 2

Node 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

HBA 2

Node 4

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn 4 Node SVC MAX Vdisk 4096

5005076801
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

Copyright IBM Corporation 2014

port1=110529e
port2=120529e
port3=130529e
port4=140529e

R2

R4

1 1 1 1 1 1 1 1
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3

Bay 3

Bay 5

Bay 7

C0 C1

C0 C1

C0 C1

R3

R4

R7

R8

5555 5 555
0000 3 333
0123 0 123

3 3 33 3 3 3 3
0 0 00 3 3 3 3
0 1 23 0 1 2 3

R3

R7

7 7 7 7 7 77 7
0 0 0 0 3 33 3
0 1 2 3 0 12 3

DIR2 SAN Fabric

DIR1 SAN Fabric

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4

Application Host Server

L7

Bay 1
C0 C1

port1=110528b
port2=120528b
port3=130528b
port4=140528b

A2

B2

Application Host Server

The same native


backend ports should not be
shared for both direct host connectivity
and SVC connectivity as shown here.
The correct way to bypass SVC is to use
other backend ports not connected to the
SVC

3 factors determine right/wrong


number of paths per vdisk device
Example of correct/incorrect number of datapaths with Host to SVC
zoning
Which Side is right?

Copyright IBM Corporation 2014

Summary - 3 factors determine


right/wrong paths
Example of correct/incorrect number of datapaths with Host to SVC zoning
Correct datapathing has 3 factors
Proper zoning
Proper SVC Host definitions (SVC logical config of the host def)
Proper redundancy for the SVC preferred /non preferred pathing

Copyright IBM Corporation 2014

Right or Wrong ?
fscsi5=10000000C9C0DC7F
fscsi7=10000000C9C0B3DB

fscsi0=10000000C9C0A984
fscsi2=10000000C9C0E0E0
Zone for p770_1_vio1a_d1 Zone for p770_1_vio1a_d2

10000000c9779a4a
500507680110B374
500507680130B374
500507680110B363
500507680130B363
500507680110B371
500507680130B371
500507680110B335
500507680130B335

b03vio101
NRPOKVIO1A

10000000C9C0E0E0
500507680110B374
500507680130B374
500507680110B363
500507680130B363
500507680110B371
500507680130B371
500507680110B335
500507680130B335

d1

d2

d3

d4

Zone for p770_1_vio1a_d3_SVC

Zone for p770_1_vio1a_d4_SVC

10000000C9C0DC7F
500507680120B374
500507680140B374
500507680120B363
500507680140B363
500507680120B371
500507680140B371
500507680120B335
500507680140B335

10000000C9C0B3DB
500507680120B374
500507680140B374
500507680120B363
500507680140B363
500507680120B371
500507680140B371
500507680120B335
500507680140B335

DIR2 SAN Fabric

DIR1 SAN Fabric

Over subscribed SVC to Host


HBA Zoning causing to many
datapaths
P1 P2 P3 P4
HBA 1

HBA 2

Node 1

P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

P1 P2 P3 P4
HBA 1

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn
5005076801
port1=10B374
port2=20B374
port3=30B374
port4=40B374

HBA 2

Node 4

4 Node SVC MAX Vdisk 4096

port1=10B363
port2=20B363
port3=30B363
port4=40B363

port1=10B371
port2=20B371
port3=30B371
port4=40B371

port1=10B335
port2=20B335
port3=30B335
port4=40B335

CF8

DEV#: 3 DEVICE NAME: hdisk3 TYPE: 2145 ALGORITHM: Load


Balance
SERIAL: 600507680181059BA000000000000005
==========================================================
State
Mode
Select
Errors
SVC Host DefinitionsPath# Adapter/Path Name
0
fscsi0/path0
OPEN NORMAL
558254
0
id:1
1*
fscsi0/path1
OPEN NORMAL
197
0
name:P770_1_vio1A
2*
fscsi0/path2
OPEN NORMAL
197
0
10000000C9C0B3DB
3
fscsi0/path3
OPEN NORMAL
493559
0
10000000C9C0DC7F
4
fscsi2/path4
OPEN NORMAL
493330
0
10000000C9C0E0E0
5*
fscsi2/path5
OPEN NORMAL
197
0
6*
fscsi2/path6
OPEN NORMAL
197
0
10000000C9C0A984
7
fscsi2/path7
OPEN NORMAL
493451
0
8
fscsi5/path8
OPEN NORMAL
492225
0
9*
fscsi5/path9
OPEN NORMAL
197
0
10*
fscsi5/path10
OPEN NORMAL
197
0
11
fscsi5/path11
OPEN NORMAL
492660
0
12
fscsi7/path12
OPEN NORMAL
491988
0
13*
fscsi7/path13
OPEN NORMAL
197
0
14*
fscsi7/path14
OPEN NORMAL
197
0
15
fscsi7/path15
OPEN NORMAL
492943
0

Copyright IBM Corporation 2014

Wrong - Causes Over Subscription


fscsi5=10000000C9C0DC7F
fscsi7=10000000C9C0B3DB

fscsi0=10000000C9C0A984
fscsi2=10000000C9C0E0E0
Zone for p770_1_vio1a_d1 Zone for p770_1_vio1a_d2

10000000c9779a4a
500507680110B374
500507680130B374
500507680110B363
500507680130B363
500507680110B371
500507680130B371
500507680110B335
500507680130B335

b03vio101
NRPOKVIO1A

10000000C9C0E0E0
500507680110B374
500507680130B374
500507680110B363
500507680130B363
500507680110B371
500507680130B371
500507680110B335
500507680130B335

d1

d2

d3

d4

Zone for p770_1_vio1a_d3_SVC

Zone for p770_1_vio1a_d4_SVC

10000000C9C0DC7F
500507680120B374
500507680140B374
500507680120B363
500507680140B363
500507680120B371
500507680140B371
500507680120B335
500507680140B335

10000000C9C0B3DB
500507680120B374
500507680140B374
500507680120B363
500507680140B363
500507680120B371
500507680140B371
500507680120B335
500507680140B335

DIR2 SAN Fabric

DIR1 SAN Fabric

Over subscribed SVC to Host


HBA Zoning causing to many
datapaths
P1 P2 P3 P4
HBA 1

HBA 2

Node 1

P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

P1 P2 P3 P4
HBA 1

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn
5005076801
port1=10B374
port2=20B374
port3=30B374
port4=40B374

HBA 2

Node 4

4 Node SVC MAX Vdisk 4096

port1=10B363
port2=20B363
port3=30B363
port4=40B363

port1=10B371
port2=20B371
port3=30B371
port4=40B371

port1=10B335
port2=20B335
port3=30B335
port4=40B335

CF8

DEV#: 3 DEVICE NAME: hdisk3 TYPE: 2145 ALGORITHM: Load


Balance
SERIAL: 600507680181059BA000000000000005
==========================================================
State
Mode
Select
Errors
SVC Host DefinitionsPath# Adapter/Path Name
0
fscsi0/path0
OPEN NORMAL
558254
0
id:1
1*
fscsi0/path1
OPEN NORMAL
197
0
name:P770_1_vio1A
2*
fscsi0/path2
OPEN NORMAL
197
0
10000000C9C0B3DB
3
fscsi0/path3
OPEN NORMAL
493559
0
10000000C9C0DC7F
4
fscsi2/path4
OPEN NORMAL
493330
0
10000000C9C0E0E0
5*
fscsi2/path5
OPEN NORMAL
197
0
6*
fscsi2/path6
OPEN NORMAL
197
0
10000000C9C0A984
7
fscsi2/path7
OPEN NORMAL
493451
0
8
fscsi5/path8
OPEN NORMAL
492225
0
9*
fscsi5/path9
OPEN NORMAL
197
0
10*
fscsi5/path10
OPEN NORMAL
197
0
11
fscsi5/path11
OPEN NORMAL
492660
0
12
fscsi7/path12
OPEN NORMAL
491988
0
13*
fscsi7/path13
OPEN NORMAL
197
0
14*
fscsi7/path14
OPEN NORMAL
197
0
15
fscsi7/path15
OPEN NORMAL
492943
0

Copyright IBM Corporation 2014

Right or Wrong?

Copyright IBM Corporation 2014

Right or Wrong?

Copyright IBM Corporation 2014

Right or Wrong?

Copyright IBM Corporation 2014

Wrong, Shows as Degraded Paths

Copyright IBM Corporation 2014

Whats wrong?
Fabric_A Zone Definitions

Fabric_B Zone Definitions

Zone 101_a

Zone_101_d

A1=10000000c97549bc
P1=50050768011052ca
P1=50050768011052b7
P1=500507680110529e

B2=10000000c9771aae
P1=50050768011052ca
P1=50050768011052b7
P1=500507680110529e

P1=500507680110528b

P1=500507680110528b

Zone 101_c

b03vio101
b03vio101

A1

A2

A3

B1

B1=10000000c9771aaf
P1=50050768014052ca
P1=50050768014052b7
P1=500507680140529e
P1=500507680140528b

Zone_101_b
A2=10000000c97549bd
P1=50050768013052ca
P1=50050768013052b7
P1=500507680130529e
P1=500507680130528b

DIR2 SAN Fabric

DIR1 SAN Fabric

Port to Fabrics
not evenly distributed
SVC Host Definitions
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 1

HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn
5005076801
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

HBA 2

Node 4

id:2
name:b03vio100_apvg
10000000C97549BD
10000000C97549BC
id:3
name:b03vio100_dbvg
10000000C9771AAF
10000000C9771AAE

4 Node SVC MAX Vdisk 4096


port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

port1=110529e
port2=120529e
port3=130529e
port4=140529e

port1=110528b
port2=120528b
port3=130528b
port4=140528b

Copyright IBM Corporation 2014

In this example we show


2 SVC Host definitions, for 1 host,
with uneven port to fabric distribution
and 4 zones
As you can see...A1 will failover to A2,
on the same fabric..this is a Single point
of Falure (SPoF)
See the next page for correct zoning
where there are multiple Host HBAs,
zoned to the SVC

Wrong
Fabric_A Zone Definitions

Fabric_B Zone Definitions

Zone 101_a

Zone_101_d

A1=10000000c97549bc
P1=50050768011052ca
P1=50050768011052b7
P1=500507680110529e

B2=10000000c9771aae
P1=50050768011052ca
P1=50050768011052b7
P1=500507680110529e

P1=500507680110528b

P1=500507680110528b

Zone 101_c

b03vio101
b03vio101

A1

A2

A3

B1

B1=10000000c9771aaf
P1=50050768014052ca
P1=50050768014052b7
P1=500507680140529e
P1=500507680140528b

Zone_101_b
A2=10000000c97549bd
P1=50050768013052ca
P1=50050768013052b7
P1=500507680130529e
P1=500507680130528b

DIR2 SAN Fabric

DIR1 SAN Fabric

Port to Fabrics
not evenly distributed
SVC Host Definitions
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 1

HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn
5005076801
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

HBA 2

Node 4

id:2
name:b03vio100_apvg
10000000C97549BD
10000000C97549BC
id:3
name:b03vio100_dbvg
10000000C9771AAF
10000000C9771AAE

4 Node SVC MAX Vdisk 4096


port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

port1=110529e
port2=120529e
port3=130529e
port4=140529e

port1=110528b
port2=120528b
port3=130528b
port4=140528b

Copyright IBM Corporation 2014

In this example we show


2 SVC Host definitions, for 1 host,
with uneven port to fabric distribution
and 4 zones
As you can see...A1 will failover to A2,
on the same fabric..this is a Single point
of Falure (SPoF)
See the next page for correct zoning
where there are multiple Host HBAs,
zoned to the SVC

Right
Fabric_A Zone Definitions

Fabric_B Zone Definitions

Zone 101_a

Zone 101_a

A1=10000000c97549bc
P1=50050768011052ca
P1=50050768011052b7
P1=500507680110529e

b03vio101_a

A1

b03vio101_b

B1

A2

B2

B1=10000000c9771aaf
P1=50050768014052ca
P1=50050768014052b7
P1=500507680140529e

P1=500507680110528b

P1=500507680140528b

Zone_101_b

Zone_101_b

A2=10000000c97549bd
P1=50050768013052ca
P1=50050768013052b7
P1=500507680130529e
P1=500507680130528b

B2=10000000c9771aae
P1=50050768012052ca
P1=50050768012052b7
P1=500507680120529e
P1=500507680120528b

DIR2 SAN Fabric

DIR1 SAN Fabric

The host definitions in the SVC


should be defined as 2 hosts
and 4 zones in the Fabrics

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 1

HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

HBA 2

Node 4

iogrp 0

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn
500507680
port1=11052ca
port2=12052ca
port3=13052ca
port4=14052ca

4 Node SVC MAX Vdisk 4096


port1=11052b7
port2=12052b7
port3=13052b7
port4=14052b7

port1=110529e
port2=120529e
port3=130529e
port4=140529e

SVC Host Definitions


id:2
name:b03vio101_a
10000000C97549BC
10000000C9771AAF
id:3
name:b03vio101_b
10000000C97549BD
10000000C9771AAE

port1=110528b
port2=120528b
port3=130528b
port4=140528b

Copyright IBM Corporation 2014

In this example we show


2 seperate SVC Host definitions and
2 Fabric zones, per host per Fabric
Totaling 4 zones
This allows proper failover from Primary
to Alternate nodes within the SVC iogrp.

Right or Wrong?
Fabric_A Zone Definitions
A1=10:00:00:00:c9:3f:75:37
A2=10:00:00:00:c9:3f:80:54
A3=10:00:00:00:c9:42:89:ac
A4=10:00:00:00:c9:42:80:16

b03vio210

A1 A2 A3 A4 B1 B2 B3 B4

DIR2 SAN Fabric

DIR1 SAN Fabric

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 1

HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

iogrp 0
2048 LUNs max

2048 LUNs max

5005076801

HBA 2

Node 4

iogrp 1

wwpn

Fabric_B Zone Definitions


B1=10:00:00:00:c9:3f:74:16
B2=10:00:00:00:c9:3f:f7:2f
B3=10:00:00:00:c9:42:b4:4b
B4=10:00:00:00:c9:42:b6:18

4 Node SVC MAX Vdisk 4096

SVC Host Definitions


id:8
name:b03vio210_allvg
10000000C942B618
10000000C942B44B
10000000C9428016
10000000C94289AC
10000000C93FF72F
10000000C93F7416
10000000C93F8054
10000000C93F7537

Copyright IBM Corporation 2014

In this example we show


1 SVC Host definitions and
2 Fabric zones, per host per Fabric.
This does not allow proper failover from
Primary to Alternate nodes within the
SVC iogrp.
See the next page for proper zoning

Right
b03vio210_1

A3

B2

A2

Fabric_B Zone Definitions


B1=10:00:00:00:c9:3f:74:16
Fabric_A Zone Definitions
A1=10:00:00:00:c9:3f:75:37

Fabric_B Zone Definitions


B2=10:00:00:00:c9:3f:f7:2f

Fabric_A Zone Definitions


A3=10:00:00:00:c9:42:89:ac

SVC Host Definitions

HBA 2

Node 1

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1

HBA 2

Node 2

HBA 1

HBA 2

Node 3

HBA 1

iogrp 1

2048 LUNs max

2048 LUNs max

wwpn

HBA 2

Node 4

iogrp 0
5005076801

B4

Fabric_B Zone Definitions


B4=10:00:00:00:c9:42:b6:18

DIR2 SAN Fabric

DIR1 SAN Fabric

HBA 1

A4
B3
Fabric_A Zone Definitions
A4=10:00:00:00:c9:42:80:16

Fabric_B Zone Definitions


B3=10:00:00:00:c9:42:b4:4b

Fabric_A Zone Definitions


A2=10:00:00:00:c9:3f:80:54

P1 P2 P3 P4

b03vio210_4

b03vio210_3

b03vio210_2

B1

A1

id:1 name:b03vio210_1
10000000C93F7537
10000000C9428016
id:2 name:b03vio210_2
10000000C93F8054
10000000C93FF72F
id:3 name:b03vio210_3
10000000C94289AC
10000000C942B44B

4 Node SVC MAX Vdisk 4096


id:4 name:b03vio210_4
10000000C93F7416
10000000C942B618

Copyright IBM Corporation 2014

In this example we show


4 seperate SVC Host definitions and
2 seperate Fabric zones, per host
per Fabric
This does allow proper failover from
Primary to Alternate nodes within the
SVC iogrp.

Session summary
You should know:
Zoning is one of the most important configuration to proper:

Failover
Redundancy
Resiliency
Performance

Understand best practice zoning for 4 port SVC models


Understand best practice zoning for 8 port SVC models CG8
Understand best practice zoning for 12 port SVC models DH8
Understand best practice zoning for Dual Core Fabric to SVC
Hopefully enjoyed learning Right and Wrong Zoning Scenarios
by playing along
Had Fun

Copyright IBM Corporation 2014

Thank you!

For you interest


and attendance

Copyright IBM Corporation 2014

SVC Zoning Evolution

IBM SVC Zoning - Best Practices 101

Backup Slides

Copyright IBM Corporation 2014

86

Agenda

SVC Zoning, Volume Pathing and Multipathing testing


Most common questions
Understanding the physical and logical
General SVC Best Practice Zoning Concepts
SVC Zone Concepts
Zoning evolution with SVC Model evolution
Storage to SVC Zoning
Host HBA to SVC zoning (preferred/non preferred pathing)
Dual Host HBAs
Multiple Host HBAs
Host HBA multipathing settings and testing

Zone Types for 8 port Nodes


MM/GM and heartbeat zones for 8 port nodes

SVC Cluster zoning for Dual Core Fabrics examples


Cluster Zone 4 port vs. 8 port nodes
Copyright IBM Corporation 2014

Most common zoning questions - BP


recommendations
What is the max number of WWNN/WWPNs devices?
Always check the Max Limit configuration URL for the most current
updates
Search for the term "configuration limits and restrictions" in the Search
support and downloads search box at the following website:
www.ibm.com/storage/support/2145

Currently 1024 WWNN per cluster and 1024 WWPNs per cluster
Recommendation - The more wwpns per wwnn, the more throughput up
to 16

How many IO connections/zones per storage device to the


SVC should I zone?
16 from any one storage device unit zoned with all SVC node ports

All SVC nodes must see same set of LUNs from disk controller
Otherwise degraded mode on controller and/or MDisks

Copyright IBM Corporation 2014

Most common zoning questions - BP


recommendations
Should I mask host connections through zoning or SVC
configurations?
Best practice is to use the default value of ..111111111111 (all ports
enabled) and control masking through zoning
Use host type Target Port Group (tpgs) for Solaris hosts, (hpux) for
HP/UX, (openvms) for HP AlphaServer/HP Intergrity and (generic) for
everything else
Separate disk and tape IO on host HBAs

What works better, 4 or 8 paths per Vdisk?


Recommendation is 4 paths per Vdisk

Copyright IBM Corporation 2014

Most common zoning questions - BP


recommendations
How many Iogrps should I map to a host? 4?
Recommendation is to size per throughput and number of hosts per
cluster
May need to isolate/sement host activity/chatter from one Iogrp to
another
For more horsepower spread IO across more LUNs from more
Iogrps

Can a host have only one connection to the SVC ?


Dual host HBA connections

Copyright IBM Corporation 2014

Three (3) types of Port Masking


There are three(3) types of Port masking to be aware of with the new code and DH8
Host port masking
Default port masking, with 32 bits, used to be "1111" now it is "...111111111111". reading from
right to left...1 for each of the 12 ports. Leaving the default of 1 enables the flow of IO to pass
through the node port, "0" means to block IO on the port.
Benefits
zoning changes
Less complexity for Vdisk IOgrp moves,
Changes introduced in the SAN, or the hosts, with less complexity

The MM/GM port mask


Allows only MM/GM replication traffic to flow through the enable ports,
Note: Other traffic can flow here from other connected devices, but defeats the purpose of dedicating
MM/GM traffic . We dedicate and use port 5&6 for MM/GM (partner_fc_port_mask) so the mask would be
000000110000

The Intra-cluster communication port mask


Also dedicated at 0000110000 for 12 ports meaning ports 7 & 8 are enabled with 1's and the rest
are blocked.
Bottom-line: We have so much field experience behind these best practices, that this is what we recommend. The most
common reoccurring issue is when LUNs are moved around between IOgrps or shared to clusters, the zoning gets
redone, but most SME's forget about or don't understand the port mask and so don't change it and then the redundancy is
lost when change is made, resulting in IO blocked and outages...this we made it a best practice to reduce the complexity
of managing the LUN to host mapping.
Copyright IBM Corporation 2014

Disk Controller Best Practices


Maximum of 1024 WWNNs
EMC DMX/SYMM, All HDS and SUN/HP HDS clones use one WWNN
per port; each appears as a separate controller to SVC
Map LUNs through up to 16 FA ports
Results in 16 WWNNs/WWPNs used out of the max of 1024

IBM, EMC Clariion, HP, etc. use one WWNN per subsystem; each
appears as a single controller with multiple ports/WWPNs
Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of
1024

Greater Qdepth is archived with fewer Mdisks

Copyright IBM Corporation 2014

92

Disk Controller Best Practices


DS4K/5K EMC Clariion/CX
Both have preferred controller architecture
SVC honors this configuration

Use minimum of 4 and preferably 8 ports or more up to maximum of


16
More ports equate to more concurrent I/O driven by SVC
Support for mapping controller A ports to Fabric A and controller B
ports to Fabric B or cross connecting ports to both fabrics from both
controllers
IBM DS4000 does not support a storage system A and storage
system B in the same zone
Later is preferred to avoid AVT/Trespass occurring if a fabric or all
paths to a fabric fail

SVC supports SVC queue depth change for CX models


Drives more I/O per port per Mdisk

93

Storage Zones - General SVC Best


Practice Zoning Concepts
Create two cluster zones (do not include MM/GM and intracluster traffic ports)
Never span zones to include more than one Backend storage
device!
Never put Host OS ports, SVC ports and Backend Storage
ports together in the same zone
Instead
Create zones with Host ports and SVC ports
Create zones with Backend and SVC ports
Never use the same DS8K ports or any native back-end port for
connectivity to SVC and an attached host
If SVC is attached to the DS8K or other native back-end devices and the DS8K
or other back-end device is using native GM (not SVC GM) then dedicate
appropriate back-end ports specifically for GM, not to be used for attaching any
other device, whether Host Server, SVC or other connectivity relationships.
Copyright IBM Corporation 2014

Host Zones
Best Practices Concepts
Too many paths to a Vdisk
If the recommended number of paths to a Vdisk are exceeded, path
failures may not be recovered in the required amount of time
Causes excessive I/O waits, resulting in application failures
Under certain circumstances, it can reduce performance
Note: 8 paths are supported but 4 are optimum for SDD/SDDDSM/SDDPCM

SVC host zones


There must be a single zone for each host port. This zone must
contain the host port, and one port from each SVC node that the host
will need to access. While there are two ports from each node per
SAN fabric in a usual dual-fabric configuration, make sure that the
host only accesses one of them.
Check with the Host System Administrators to receive output from the SDD or
PCM path query device commands on each host to verify proper pathing
between the SVC nodes and host systems
Note: It is a supported configuration to have eight paths to each VDisk, but this design
provides no performance benefit, and it does not improve reliability or availability by any
significant degree
Copyright IBM Corporation 2014

Host Zones
SVC Preferred Node Scheme
Hosts with four (or more) Host Bus Adapters (HBAs)
Takes a little more planning.
Because eight paths are not an optimum number, you must instead
configure your SVC Host Definitions (and zoning) as though the single
host is two or more separate hosts
During Vdisk assignment, alternate which Vdisk is assigned to one of
the pseudo-hosts, in a round robin fashion (a pseudo-host is nothing
more than another regular host definition in the SVC host config. Each pseudohost will contain 2 unique host WWPNs, 1 WWPN mapped to each fabric)
Note: A pseudo-host, is not a defined function or feature of the SVC. If you
need to define a pseudo-host, you are simply adding another host id to the
SVC host config. Instead of creating one host id with 4 WWPNs, you would
define 2 hosts with 2 WWPNs. This is now the reference for the term pseudohost.
Note: Be careful not to share the Vdisk to more than two adapters per host,
so as to not oversubscribe the number of datapaths per vdisk per host

Copyright IBM Corporation 2014

General SVC Best Practice Zoning


Concepts
Dual Core Fabrics require a breakout in cluster zoning to
separate SVC node ports on core1 from core2
Prevents/Minimizes IO from spanning/routing across ISLs and Edge
switches in a Core/Edge design
SVC node ports plugged into dir1 core1 should reside in a separate zone
from SVC node ports plugged into dir1 core2.
For MM/GM zones, take advantage of spreading the IO across both
cores through careful zoning

Ensure the Intra-cluster/heartbeat zone is also isolated properly between


each fabrics dual cores (dont span heartbeat across cores in one fabric)

Note: - Never make zoning changes on redundant Fabrics at the same time
Make changes on one fabric and wait 30 min in-between
Please see the following link for zoning BPs
Cisco:
http://g25aciwas09.con.can.ibm.com:9080/ram/assetDetail/generalDetails.faces?guid=5EB2DDF9-7568-368A-7F99-1BCFFF85E472

Brocade:
http://g25aciwas09.con.can.ibm.com:9080/ram/assetDetail/generalDetails.faces?guid=57A9649D-A04B-5931-05DA-F389F697D4D3
Copyright IBM Corporation 2014

New Build- Positioning for HBA


redundancy
Logical port #s reflecting the embedded WWPN
Blue ports go to one Fabric
Red Ports go to the other Fabric

Logical port with wwpn # embedded


Physical port number

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

New Build - Physical to Logical Port


numbering for 12 port nodes
When creating a new cluster on DH8 etc then you will get a slot and port encoding
for all cards with all ports logically left to right.
Logical port #s reflecting the embedded WWPN

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

New Build - Physical to Logical Port


numbering
Logical port #s reflecting the embedded WWPN
When Write Data Rate > 3GB/sec per IOgrp

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

New Build - Physical to Logical Port


numbering for 12 port nodes

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

New Build Numbering When Write Data


Rate > 3GB/sec per IOgrp

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

New CG8 Dual Core Zones


Make the local_fc_port_mask (Node to Node/heartbeat) = 10001000
Make the partner_fc_port_mask (MM/GM
= 01000100

Copyright IBM Corporation 2014

Port Layout provided Keith Williams

Correct way to make MM/GM Zone,


new implementation 7.1.x
Make the local_fc_port_mask (Node to Node/heartbeat) = 000010001000
Make the partner_fc_port_mask (MM/GM
= 000001000100

104

Potrebbero piacerti anche