FICON For CCIEstudy May

FICON
101
CCIE SAN Study Group
By Ozden Karakok May 2011
FICON
2010 Cisco Systems, Inc. All rights reserved.
Objective & Agenda

At the end of this session, attendees will be able to answer what is FICON is all about! FICON Concepts Terminology Topologies Addressing Frames
FICON Applications MDS Requirements FICON Manager FICON Control Unit Process (FCD) SAN-OS Mainframe
Configuring FICON

FICON
SAN-OS 3.0 Update Troubleshooting FICON Summary

FICON Terminology
FICON
IBM Mainframe Systems

Current IBM mainframe systems:
S/390 G5 S/390 G6 zSeries 800 zSeries 890 zSeries 900 zSeries 990 z9 Ent. Class z9 Business C. z10 Ent. Class z10 Business C.
4
OS/390 VM VSE/ESA MVS z/OS z/VM Linux
FICON
Introduction to FICON
FICON Protocol Layer
FICON is Fibre Channel FC-4 layer protocol It is at the same layer as FCP protocol Fibre Channel Single Byte Command Code mapping protocol is described in FCSB3, FC-SB4 FC-SB2 is an older version. FC-SB3-4 is superset.
Ref: www.t11.org
Physical Variant
ULP (Upper Level Protocol)
SCSI 3/4
Single Byte Command Code Sets
FC - 4 FC - 3
SCSI Command Set Mapping (FCP)
FC-SB2/FC-SB3 (FICON)
Common Services FC - 2 FC - 1 FC - 0
FICON
Framing Protocol/Flow control
Encode/Decode (8/10)
FICON Terminology
A Channel is a physical entity/element of the processor, acting as the interface between the CPU and the CU
A Channel Image instantiates multiple, logical channels behind a physical channel
A Channel Path includes the interface adapter, physical fiber pair, and up to two intervening FICON switches/directors between the channel and the CU A Channel Path Identifier (CHPID) is a single hexadecimal byte value identifying each channel path to the system
FICON
FICON Terminology
Outboard I/O processors called Control Units (CUs)
CUs can be stand-alone or integrated in peripheral devices CUs are identified by a two-byte hexadecimal CUNUMBER A Control Unit Image instantiates multiple, logical CUs behind a physical CU
A Device Address (aka Unit Address) is a decimal value that selects a specific device on a given CU image
FICON
FICON Terminology
A Channel is an entity typically mapped to an N-port in a host computer which performs access to I/O devices
M A I N F R A M E
IBM
Channel Channel Images Control Unit Image

There can be multiple Channel Images behind one N port, each of which represent a separate logical channel.
N Port
A Control Unit is the interface to I/O devices such as printer, disks etc.
N Port
Legend
Physical Logical
FICON
Control Unit (CU)
There can be multiple Control Unit images behind each Control Unit
8
FICON Terminology
A Logical Partition (LPAR) is a subset of mainframe resources, dedicated to implementing a particular programming environment
Each LPAR functions independently of other LPARs and is protected against the actions of poorly-behaved LPARs Resources such as CHPIDs (Channel Path Identifier) may be shared between LPARs
FICON
MIF
LPARs can share channels Multiple Image Facility (MIF)
MIF LPAR 1
N_Port CU
Channel
N_Port
LPAR 2
LPAR n
N_Port CU
FICON
10
FICON Terminology
An Output Port is the FICON director or switch port that connects to the CU
In a switched point-point environment, it can be a single byte hexadecimal value identifying the director/switch port number connected to the CU FICON port numbering schemes are proprietary to the switch vendor/ model; typically different than whats used for FC config The port number may also be added with the switch domain ID to form a two-byte Output Port value if supported by the CU In a cascaded environment, Output Port must be a two-byte value derived from the non-Entry Switch
FICON
11
FICON Cascaded Topology

Output Ports Entry Switch Non-entry switch
CH = Initiator CU = Target
Switch ID = Domain ID
Portaddress = int fc X/Y
FICON
12
FICON Terminology
The Input Output Control Program (IOCP) is a mainframe application that defines the Input Output Control Data Set (IOCDS), which the mainframe uses to define & identify control units, devices, channel paths, channels and their relationships to each other. I/O definition file (IODF) All of the terms weve just covered are required IOCP/ IOCDS parameters
FICON
13
Objective & Agenda

FICON Concepts Terminology Topologies Addressing Frames
FICON Applications MDS Requirements FICON Manager FICON Control Unit Process (FCD) Configuring FICON SAN-OS Mainframe SAN-OS 3.0 Update Troubleshooting FICON Summary

FICON
14
FICON Topologies
A Point-to-Point (direct connection) A Switched Point-Point Environment results when there is one FICON director/switch in the channel path between the channel and CU A Cascaded Environment results when two FICON directors/ switches are connected in series along the channel path between the channel and CU An Entry Switch is the FICON director or switch connected to the channel via the channel path
In a cascaded environment, the Entry Switch is first switch encountered in the channel path from the channel to the CU The Entry Switch is identified by a single hexadecimal byte equal to its FC domain ID
FICON
15
FICON Topologies (Contd)

Point to Point
FICON Channel
In Point to Point topology, channel and control unit are directly connected. There is no switch between them.
FICON Control Unit
FICON
16

FICON Channel FICON Channel
Single Switch Fabric Path Configuration

In Single Switch Fabric path configuration, all channels and control units are attached to a single switch This is called switched point-topoint configuration.
FICON Control Unit

FICON
FICON Control Unit

FICON Control Unit

17

FICON Channel FICON Channel
Two Switch Fabric Path Configuration

In Two Switch Fabric path configuration there may be more than 2 switches in the fabric, but only one hop between any pair of channel and control units. Multiple ISLs between connected switches are allowed. However, the fabric should perform load balancing on these ISLs.
FICON Control Unit

FICON
FICON Control Unit

FICON Control Unit

18
FICON Addressing
8
Link address
Static in FICON fabric (usually 0)
Link Address
FICON defines concept of link address. Each channel and control unit have one unique link address. In older versions, link address used to be single byte and could only be used in single switch environment FC-SB3 defines two byte link addresses which allow more than one switch in FICON path. Link addresses are mapped to FCID as depicted in figure.
Domain
Area
Port
Single Byte Link address to FCID mapping

16
Link address
Static in FICON fabric (usually 0)
Domain
Area
Port
Two Byte Link address to FCID mapping (for Cascaded environments)

FICON 2010 Cisco Systems, Inc. All rights reserved.
19
HCD Point to Point
FICON
20
HCD Cascaded
FICON
21
Using DM to identify Port Addresses

9513 - 16 ports per slot 9509 - 32 ports per slot
Notice FICON port numbers are displayed

22
Objective & Agenda


FICON
23
FICON Channel Initialization
CH to Switch
Same Well Known Fibre Channel Addresses
CH to CU
FICON
24
FICON Control Unit Initialization
Defined In FC-FS
Can register both SCSI and FC-SB FC-4 types
Defined In FC-GS
port-wwn (vendor) :50:05:07:63:00:c2:04:2b (IBM) node-wwn :50:05:07:63:00:c0:04:2b class :2,3 node-ip-addr :0.0.0.0 ipa :ff ff ff ff ff ff ff ff fc4-types:fc4_features:scsi-fcp fcsb2-ch-cu fcsb2-cu-ch [snip]
FICON
25
QSA Query Security Attributes

Used by Channel to verify Fabric Binding and Insistent Domain IDs is configured on MDS Only used on Cascade environment
QSA = 0x7E 00 00 00 Revision field = 0x00 00 00 01

26
QSA Accept
QSA Accept sent if Security Features enabled
Bit Meaning 0 1 2 3 4 5 6
0000 0011 Bit 0 & 1 are on 7654 3210 BIT numbers
Fabric Binding Static DomainID Soft Zoning Hard Zoning SLAP SRP MS CT Authentication
7-31 Reserved
27
FICON
ESA Exchange Security Attributes

Used between Switches (ISL) to validate QSA information Passed during ISL link initialization Address by and to MDS Fabric Controller 0xFF FF FD ESA frame command code is 0x32 00 00 00
272.210534 272.210594 272.241828 272.242168 ff.ff.fd -> ff.ff.fd ff.ff.fd -> ff.ff.fd ff.ff.fd -> ff.ff.fd ff.ff.fd -> ff.ff.fd 0xa 0xffff SW_ILS 0x32000000! 0xa 0xa FC Link Ctl, ACK1! 0xa 0xa SW_ILS SW_ACC (0x32000000)! 0xa 0xa FC Link Ctl, ACK1!
ESA Class F Ack ESA Accept Class F Ack
FICON
28
RNID Request Node Id Data

Used to determine characteristics of node at other end
FICON
29
RNID Accept
Accept is broken into two sections:
Common ID Field - shows pWWN and nWWN. Ignored by FICON Specific ID Data shows SB-2, FCID, Name, Manufacturer, Model..
30
LIRR, RLIR, DRLIR

LIRR (Link Incident Record Registration) used to register for RLIR (Registered Link Incident Record) similar to SCR RLIR used by N-Ports to send link incidents to registered ports - similar to RSCN DRLIR (Distribute Registered Link Incident Records) used to send incidents to entire fabric Switch receives DRLIR and send RLIR to registered NPorts LIRR sent to all Switches in path and Control Unit Sent to 0xFF FF FA (MDS Management Server)
31
ELP Establish Logical Path LPE Logical Path Established

ELP - Sent from
Channel to Control Unit to verify optional parameters and verify logical path LPE Sent from Control Unit to Channel as verification of logical path and pass optional parameters LACK Close exchange
FICON
32
Frame Format
FC-SB-3 Frame Format
Bytes: 4 Idles
S O F
24
FC Header
32
FICON Header
0 - 8160 Data Payload
4
E CRC O F
4 Idles
Bit 0
Reserved IUI Reserved
7 8
CH_ID DH_FLGS Device Address
15 16
Reserved
23 24
CU_ID Reserved CCW Number Token
31
SB-3 Header IU Header
Command / Data / Status / Control / Link Header Reserved IU Count LRC DIB Data Count
DIB Header
FICON
33
FICON Read Operation
FICON Channel
Command DIB (READ) Command DIB (RSP) Data DIB Data DIB Status DIB
FICON CU
FCP Initiator
FCP_READ FCP_DATA FCP_DATA FCP_RSP
FCP Target
FICON Read Operation
FCP Read Operation
FICON
34
FICON Write Operation
FICON Channel
Command DIB (WRITE) Command DIB (RSP) Data DIB Data DIB Status DIB
FICON CU
FCP Initiator
FCP_WRITE FCP_XFER_RDY FCP_DATA FCP_DATA FCP_RSP
FCP Target
FICON Write Operation
FCP Write Operation
FICON
35
Flow Control
FICON uses standard FC-2 BB_Credit flow control FICON also uses IU Pacing: Similar to Fibre Channel EE_Credits IU Pacing credit is sent to the channel by the CU Indicates the maximum number of IUs a channel can send before a command response IU is expected
IU Pacing CRR
Channel N_Port
CMR Frames
F_Port E_Port E_Port F_Port
Frames R_RDYs R_RDYs FC-2 BB_Credit
CU N_Port
FC-2 BB_Credit
36
Objective & Agenda


FICON
37
Linux Support
IBM zSeries supports Linux LPARs: Allows Linux to access ESCON/FICON storage Linux also supports open systems FCP: Same adapters used for FICON and FCP Different firmware versions Applications: CPU-intensive workloads FICON Server/storage FICON consolidation LINUX FCP NPIV (like MIF for FCP channels)
z/OS FICON
FICON
FC FC
FC FC
FC
FC
FC
FC
zSeries server
38
Disaster Recovery Applications

Data replication over FICON: Backup, snapshot, and disaster tolerance Local and remote replication Globally Dispersed Parallel Sysplex (GDPS) w/ IBM XRC IBM PtP VTS GDPS w/ IBM PPRC (runs over FCP) HDS HXRC
FICON
39
Disaster Recovery Applications

XRC - eXtended Remote Copy
FICON
40
Channel Extension
Channel extension is different than distance extension:
Distance extension simply means extending FICON over a WAN Channel extension involves protocol translation Channel extenders proxy some commandse.g. writesto increase performance over distance MDS does not do channel extension in SAN-OS 3.X
FICON
41
Objective & Agenda


FICON
42
MDS FICON Requirements

Feature
Predictable and persistent FCID allocation
Explanation
Since Link addresses are mapped to FC-ID, FICON requires FC-Ids to be predictable and persistent. FICON also requires uniform port byte (last byte in FC-ID) in FICON fabric.
Support for new LIRR, RLIR, and DRLIR ELS
Link Incident Record (LIR) ELS reports link level events in the fabric to the channel N-port. Examples of link incidents are loss of signal or synchronization, primitive sequence timeout, Invalid primitive sequence, NOS received Concept similar to RSCN
Report Node Identification (RNID) ELS Large Buffer to Buffer Credit Allocation
RNID is an ELS command, which can have common and specific Node data. FICON defines a specific node data field format. FICON requires 60 B2B credit for 100 Km and 6 B2B credit for 10 Km. MDS 9000 FICON supports up to 255 B2B Credits per Port + extended buffer credits
FICON
43
MDS FICON Requirements

Feature
Insistent Domain ID
Explanation
Since Link addresses are mapped to FC-ID, FICON requires FC-Ids to be predictable and hence Domain ID to be static in cascaded FICON switches.
In-order Delivery
FICON requires in-order delivery for class 2 and class 3 frames.
Fabric Binding and EFMD ELS
Fabric Binding feature prevents unauthorized switches joining the fabric based on switch WWNs. Exchange Fabric Membership Data (EFMD) is used to distribute this information fabric wide.
Security Attributes (QSA and ESA)
FICON requires Fabric Binding and Insistent Domain ids to be enabled in the entire fabric. Query Security Attribute (QSA) is used by Channels to ensure these features are turned on. Exchange Security Attribute (ESA) is used by switches to validate these attributes during E port bringup.
FICON
44
Objective & Agenda


FICON
45
What is FICON manager?

FICON Manager Running process that manages functionality required to support FICON in a VSAN, doing the necessary checks for enabling FICON and holding all FICON specific configuration information.
FICON
46
FICON Manager
FICON Manager is responsible for:
- Providing CMI interface for configuration - Serialization of FICON config - Maintain and apply FICON configuration files - Port Number Allocation and assignment - Port Address Allocation and assignment - Consistency Checks during enabling of FICON - Maintaining port numbers in FICON VSANs - Interact with ACL Manager for necessary ACLs during port bringup - Exchanging new ELS During E-port bringup
FICON
47
FICON Manager Role
CLI Mainframe FCD (CUP)

FICON Manager
SNMP
GUI
FC2
Others
Vsan Manager
Port Manager
FICON
48
FICON Configuration Files

Up to 16 configuration files per FICON VSAN
- Restricted to 8 alphanumeric characters
Shared lock for file access Proprietary IBM format IPL contains the following attributes for each implemented port:
- Port Address Name - Host Control - Block - Prohibit Dynamic Connectivity Mask
- show ficon vsan <x> will show saved IPL files
FICON
49
Port Number and Port Address

Implemented/Unimplemented Port Number
Refers to any available port in the chassis Reflected in the table to the right
Slot
1 2 3 4 5 6 7 8 9 10 11 12 13 Max Physical* Port Channels
91xx 92xx 9506

0 - 39 0 - 31 32 - 63 0 - 31 32 - 63 64 - 95 96 - 127 ---
9509
0 - 31 32 - 63 64 - 95 96 - 127 --128 - 159 160 - 191 192 - 223
9513
0 - 15 16 - 31 32 - 47 48 - 63 64 - 79 80 - 95 --96 - 111 112 - 127 128 - 143 144 - 159 160 - 175
Installed/Uninstalled Port
Only ports participating in FICON VSANs are considered installed i.e. If port 4 is in a FICON VSAN on a 16 port line card in a chassis, then ports 0-3, 5-31 are considered uninstalled
Port number is the physical location on the switch Port address is:
- - - - Port name Block/Unblock state Connectivity attributes Same as port number unless port swapped
256 possible port number/addresses for 9509 (8 bits, some unimplemented) 0xFE and 0xFF reserved An FCIP tunnel must be explicitly bound to a port number for FICON
- Available port numbers depends on the slot the IPS blade is located in
A Port Channel must be explicitly bound to a port number for FICON

- Available port numbers depends on chassis show ficon first-available port-number reveals next portnumber free for allocation
40 40-55
64 64-79
128 128-143
224 224-249
528 224 - 249

50
FICON
Using DM to identify Port Addresses

9513 - 16 ports per slot 9509 - 32 ports per slot
Notice FICON port numbers are displayed

51
FICON States
Various states for FICON STATE Enabled Disabled Up Down Meaning
User has configured FICON on a given VSAN. User configures a non-FICON vsan. FICON enabled and all prerequisites met FICON enabled, but one or more prerequisites not satisfied
FICON
52
FICON Prerequisites
Before FICON is up, FICON manager must check prerequisite conditions:
In-order delivery enabled Fabric Binding Insistent Domain ID No conflicting persistent FCIDs - Does not allow user configuration for reserved range: D.0.0 thru D.MaxPa.0 where: D is domain MaxPa is max Port Addr implemented in FICON
FICON
53
FICON Port Initialization

Blocked/Unblocked State - A blocked port is configured down by Port manager, at request of FICON manager - Blocked ports will continue to transmit an Off-Line State (OLS) primitive sequence. - Similar to shut/no shut, as blocked port must be reinitialized - Exception is TE port: - Removal of the VSAN from the VSAN allowed list - Port Channels - Members get port channel configuration (block/unblock) Prohibit State - FICON manager programs ACLs to allow communication between certain ports and prohibit others - Similar concept to zoning - Port remains up
FICON
54
FICON Port Initialization ELS

Replies to Query Security Attributes (QSA)
- For Fx, Responsible for sending: - Fabric Binding - Insistent Domain ID - In-order delivery
Sends Exchange Security Attributes (ESA)

- Upon bring up of (T)E Ports
Sends Request Node Identification (RNID)

- Sent after QSA reply
FICON
55
FICON Port Swap Support

Port Swap feature
- Must be supported for FICON directors - Enables swapping configuration on two physical ports - FICON manager swaps all config except Port # - Ports left in shut state followingno shut required
FICON Configuration Including Port Address, Block State, etc.
FICON Configuration Including Port Address, Block State, etc.
Physical Port A
Physical Port B
56
Objective & Agenda


FICON
57
FICON Control Unit Port (CUP)

CUP protocol provides inband management of switch from mainframe In order to be classified as FICON director, CUP protocol needs to supported. CUP specification is proprietary to IBM and is available only through special license agreement with them. CUP can configure a port to be prohibited from another port, blocked, etc. Complimentary to SNMP, CLI, and is serialized through CMI FICON Manager holds all configuration and FICON specific information; FCD acts as a client
SANOS 3.0(2) View of FICON IPL File
FICON
58
FICON Control Device

FCD process is responsible for:
(FCD)
- Acquiring persistent pWWN for Control Device - Uses Key-Based Chassis Specific (KCS) WWNs - Uses VSAN based nWWN - Acquiring FCID from FPort server - 0xFE is well defined address required - FCID will be <domain, 0xFE, 00> for Control Device - Device allegiance based Serialization of FICON config - Support for both SB-3 configuration and query commands
FICON
59
FICON Port Configuration

Block Prohibit Port Address Name Port Swap Service Required Port Statistics Block all communication from a port. In this state port is down and will not send any information. Each port (identified by port address) can block any other port. This is done via ACLs. Each port (address) can have a 24 character port address name. Reassigns port address, connectivity attributes and port address name of ports by swapping. This is set when FICON Director determines that some service is required on the port. Port is kept offline. Cleared when required service performed. The MDS needs to maintain detailed statistics on each port. The statistics are reported based on port numbers and not port addresses
FICON
60
HCD - CUP
FICON
61
FCD Control Device

An example of an FCD Control Device on a FICON vsan:
switch2# show ficon control-device sb3 vsan 1977 Control Unit Image:0x80b9df4 VSAN:1977 CU:0x4dfe00 CUI:0 CUD:0 CURLP:(nil) ASYNC LP:(nil) MODE:0 STATE:1 CQ LEN:0 MAX:0 PRIMARY LP: VSAN:0 CH:0x0 CHI:0 CU:0x0 CUI:0 ALTERNATE LP: VSAN:0 CH:0x0 CHI:0 CU:0x0 CUI:0 Control Unit Image:0x80bcbec VSAN:1978 CU:0x4efe00 CUI:0 CUD:0 CURLP:(nil) ASYNC LP:(nil) MODE:0 STATE:1 CQ LEN:0 MAX:0 PRIMARY LP: VSAN:0 CH:0x0 CHI:0 CU:0x0 CUI:0 ALTERNATE LP: VSAN:0 CH:0x0 CHI:0 CU:0x0 CUI:0
Domain ID = 77 (0x4d) Well known port = 0xFE pWWN assigned
switch2# show flogi database vsan 1977 --------------------------------------------------------------------------INTERFACE VSAN FCID PORT NAME NODE NAME --------------------------------------------------------------------------sup-fc0 1977 0x4dfe00 22:00:00:05:30:00:46:e0 27:b9:00:05:30:00:46:df Total number of flogi = 1.
62
Objective & Agenda


FICON
63
CLI Configuration of FICON

An example of using the setup ficon script:
switch1# setup ficon
--- Ficon Configuration Dialog --This setup utility will guide you through basic Ficon Configuraton on the system. Press Enter if you want to skip any dialog. Use ctrl-c at anytime to skip all remaining dialogs. Would you like to enter the basic configuration dialog (yes/no)[yes]:yes Enter vsan [1-4093] :3 vsan 3 does not exist, create it? (yes/no)[yes]:yes Enable ficon on this vsan? (yes/no)[yes]:yes The following options required for ficon will be enabled: In-order-delivery Fabric binding Configure domain-id for this ficon vsan (1-239) :3 Would you like to configure ficon in cascaded mode: (yes/no)[no]:yes swwn from Configure peer wwn (hh:hh:hh:hh:hh:hh:hh:hh): 20:00:00:0d:ec:0e:5b:80 remote switch Configure peer domain (1-239) :4 Would you like to configure additional peers: (yes/no)[no]: # show wwn switch Enable SNMP to modify port connectivity parameters? (yes/no)[yes]: Disable Host from modifying port connectivity parameters? (yes/no)[no]: Enable active=saved? (yes/no)[yes]:no Would you like to configure additional ficon vsans (yes/no)[yes]:no The following configuration will be applied: vsan database vsan 3 in-order-guarantee vsan 3 fcdomain domain 3 static vsan 3 fabric-binding enable fabric-binding database vsan 3 swwn 20:00:00:0d:ec:0e:5b:80 domain 4 fabric-binding activate vsan 3 zone default-zone permit vsan 3 ficon vsan 3 no active equals saved
FICON
64

A FICON port-channel configuration example:
switch1# conf t Enter configuration commands, one per line. End with CNTL/Z. MDS9506(config)# int port-channel 1 Remember that PortChannels need to be assigned FICON port numbers, so in the next command we are determining what the next available FICON port number is. switch1(config-if)# do show ficon first-available port Port number 128(0x80) is available Next step is to bind the FICON port number, set the port type to an ISL (E_Port), and set it up so that only VSANs 1 and 10 can trunk across the PortChannel. switch1(config-if)# switch1(config-if)# switch1(config-if)# switch1(config-if)# ficon portnumber 0x80 switchport mode e switchport trunk allowed vsan 1 switchport trunk allowed vsan add 10
Next we move interfaces fc1/9 and fc2/1 into PortChannel 1 switch1(config-if)# int fc1/9 switch1(config-if)# channel-group 1 force fc1/9 added to port-channel 1 and disabled please do the same operation on the switch at the other end of the port-channel,then do "no shutdown" at both ends to bring them up switch1(config-if)# int fc2/1 switch1(config-if)# channel-group 1 force fc2/1 added to port-channel 1 and disabled please do the same operation on the switch at the other end of the port-channel,then do "no shutdown" at both ends to bring them up
FICON
65

A FICON port-channel configuration example continued:
Next we no shut interfaces fc1/9, fc2/1, and PortChannel 1. switch1(config-if)# int fc1/9, fc2/1 switch1(config-if)# no shut switch1(config)# int port-channel 1 switch1(config-if)# no shut Next we repeat the configuration steps on the switch2. switch2(config)# int port-channel 1 switch2(config-if)# do show ficon first-available port-number Port number 64(0x40) is available swich2(config-if)# ficon portnumber 0x40 switch2(config-if)# switchport trunk allowed vsan 1 switch2(config-if)# switchport trunk allowed vsan add 10 swich2(config-if)# int fc1/1 switch2(config-if)# channel-group 1 force fc1/1 added to port-channel 1 and disabled please do the same operation on the switch at the other end of the port-channel, then do "no shutdown" at both ends to bring them up switch2(config-if)# int fc1/9 switch2(config-if)# channel-group 1 force fc1/9 added to port-channel 1 and disabled please do the same operation on the switch at the other end of the port-channel, then do "no shutdown" at both ends to bring them up switch2(config)# int fc1/1, fc1/9 switch2(config-if)# no shut switch2(config)# interface port-channel 1 switch2(config-if)# no shut switch2(config-if)# end
FICON
66
CLI RLIR
MDS-1# show rlir ?
erl history internal recent statistics Show Established Registration List Show link incident history Show RLIR internal Show recent link incident Show RLIR Statistics
MDS-1# show rlir internal vsan 3

Internals for VSAN: 3 -----------------------VSAN state Ficon State Fabric Name R_A_TOV Local Domain Remote Domains : : : : : : Active Up 20:03:00:05:30:00:51:1f 10000 ms 0x3(3) None
MDS-1# show rlir statistics vsan 3

Statistics for VSAN: 3 ------------------------Number Number Number Number Number Number Number Number Number Number Number Number of of of of of of of of of of of of LIRR received LIRR ACC sent LIRR RJT sent RLIR sent RLIR ACC received RLIR RJT received DRLIR received DRLIR ACC sent DRLIR RJT sent DRLIR sent DRLIR ACC received DRLIR RJT received = = = = = = = = = = = = 0 0 0 0 0 0 0 0 0 0 0 0
FICON
67
CLI RNID and Port attributes
MDS-1# show ficon vsan 5 portaddress 0x21

Port Address 33(0x21) is up in vsan 5 Port number is 33(0x21), Interface is fc1/3 Port name is Port is not admin blocked Prohibited port addresses are 250-253,255(0xfa-0xfd,0xff) Admin port mode is FX Port mode is F Peer is type 002064 model 104 manufactured by IBM
MDS-1# show ficon vsan 5 portaddress brief installed

----------------------------------------------------------------------------Port Port Interface Admin Status Oper FCID Address Number Blocked Mode ----------------------------------------------------------------------------0x40 0x40 port-channel 5 off up TE -0xfe 0xfe -off up ---
FICON
68
CLI FICON File Configuration

switch1# show run ficon vsan 2 file IPL switch1(config)# ficon vsan 2 switch1(config-ficon)# ? FICON Configuration: active Configure automatic saving of ficon configuration code-page Ficon configuration for codepage on a VSAN do EXEC command end Exit from configure mode exit Exit from this submode file Ficon file configuration on a VSAN host Enable host control of ficon no Negate a command or set its defaults portaddress Ficon configuration for portaddress on a VSAN snmp Enable snmp control of ficon switch1(config-ficon)# portaddress 1 switch1(config-ficon-portaddr)# ? FICON config for portaddress: block Block a range of portaddress do EXEC command end Exit from configure mode exit Exit from this submode name Give name for a portaddress no Negate a command or set its defaults prohibit Prohibit communication with a range of portaddress
FICON
69
Creating FICON VSAN from DM
FICON
70
Creating FICON VSAN from FM
FICON
71
Creating/Modifying FICON Files from DM
FICON
72
FICON Configuration File (IPL)

Edit IPL file, Block Prohibit Name Ports
FICON
73
Creating Port Channel through FM
Determine the ISLs to add to the Port Channel
Now select the Port Channel Wizard
FICON
74
Creating Port Channel through FM (contd)

Select a Port number for the logical PC
Fill in Port Channel Information
Port Channel is now created
FICON
75
Showing Port Channel through DM
Select Interface->Port Channels
All Port channel information is displayed
FICON
76
Swapping FICON Ports through DM
Select Ports to swap
Select FICON->Swap Selected Ports
FICON
77
Swapping FICON Ports through DM (contd)
Notice the swapped ports
FICON
78
FICON VSAN Options
Offline SW = Enables ability for Host to disable FICON or offline MDS Sync Time = Enables ability for Host to sync MDS time with Host Port Control Host = Enables ability for Host to Name, Prohibit, Block MDS ports Port Control SNMP = Enables ability for Device Manager to Name, Prohibit, Block MDS ports Active=Saved = Enables ability for IPL (ficon running configuration) to be saved automatically and immediately
FICON
79
FICON Load Balancing

Packets of a flow (SID,DID) or an exchange (SID, DID, OXID) are received in the order they were sent. Some applications can not handle Out Of Order exchanges FICON is one such application. In fact FICON uses two unique exchanges, one for the entire outbound exchange (For example a write chain) with an OXID allocated by the CH. The CU sends back a command response with a different OXID representing the inbound exchange
FICON
80
FICON Load Balancing

The FICON load balancing tool has been enhanced in the Fabric Manager 3.0 release.
FICON
81
Objective & Agenda


FICON
82
Configuring Devices on Mainframe

Consider the following example and IOCP definition
FICON
83

Configuring CHPID 89 online HCD
SDSF TSO Command Input
FICON
84

Varying devices online
SDSF TSO Command Input HCD
Vary devices D40 through D74 online. I/O errors occur because the volumes at these addresses are not initialized. V D40-D74,ONLINE IEF503I UNIT 0D50 I/O ERROR IEF503I UNIT 0D51 I/O ERROR ... Following command automatically executed based on the range that was varied on in previous command. D U,,,0D40,00052,L=OZI-Z VARY RANGE DISPLAY IEE457I 10.55.06 UNIT STATUS 198 UNIT TYPE STATUS VOLSER VOLSTATE 0D40 3390 O PPRD40 PRIV/RSDNT 0D41 3390 O PPRD41 PRIV/RSDNT 0D50 3390 OFFLINE /RSDNT
FICON
85
Displaying Devices on Mainframe

Notice the online state for configured CHPID 86 HCD
TSO
Notice the availability of devices configured in HCD HCD HCD TSO
FICON
86

Displaying CHPID 86 and 89 Matrix
52 Devices Unit = 3390 Begins at D40
CUP Device Unit = 2032 at well-known 0xFE
FICON
87

Displaying Path Matrix for Device D40
CHPID with corresponding Destination Link Address Entry Link Address
FICON
88
Objective & Agenda


FICON
89
New FICON Port Numbering scheme

Port-numbers are user-configurable Duplicates allowed
- Port-number 1 can be assigned to fc1/1 and fc2/1
Assigned on per-slot or logical basis

ficon slot <x> assign port-numbers <range> ficon logical-port assign port-numbers <range>
Default configuration
- No change for 1, 2, 6 and 9 slot chassis
- 13 slot chassis will have 11 slots w/ 16 port-numbers each
HA compatibility
FICON
90
Configuring Port-Numbers
Consider the following 9509 with LC modules in slots 2, 3, 4, and 8 switch1# sh Mod Ports --- ----2 24 3 8 4 16 5 0 6 0 8 4 module Module-Type -------------------------------1/2/4 Gbps FC Module IP Storage Services Module 1/2 Gbps FC Module Supervisor/Fabric-1 Supervisor/Fabric-1 10 Gbps FC Module
Model -----------------DS-X9124 DS-X9308-SMIP DS-X9016 DS-X9530-SF1-K9 DS-X9530-SF1-K9 DS-X9704
Status -----------ok ok ok active * ha-standby ok
Finding out what is assigned this example show the defaults for a 9509
switch1# show ficon port-numbers assign ficon slot 1 assign port-numbers 0-31 ficon slot 2 assign port-numbers 32-63 ficon slot 3 assign port-numbers 64-95 ficon slot 4 assign port-numbers 96-127 ficon slot 7 assign port-numbers 128-159 ficon slot 8 assign port-numbers 160-191 ficon slot 9 assign port-numbers 192-223 ficon logical-port assign port-numbers 224-249
FICON
91
Unreserve FICON port-numbers from an unused slot, slots 1, 7, & 9 in this case switch1# show ficon port-numbers assign ficon slot 1 assign port-numbers 0-31 ficon slot 2 assign port-numbers 32-63 ficon slot 3 assign port-numbers 64-95 ficon slot 4 assign port-numbers 96-127 ficon slot 7 assign port-numbers 128-159 ficon slot 8 assign port-numbers 160-191 ficon slot 9 assign port-numbers 192-223 ficon logical-port assign port-numbers 224-249 switch(config)# no ficon slot 1 assign port-numbers 0-31 switch(config)# no ficon slot 7 assign port-numbers 128-159 switch(config)# no ficon slot 9 assign port-numbers 192-223
Reserve port-numbers for slot 2, a 24 port card
switch(config)# ficon slot 2 assign port-numbers 128-151
FICON
92
The output should reflect the current status for all the ports switch(config)# do show ficon vsan 2 portaddress brief | inc fc2 0x80 0x81 0x82 0x83 0x84 0x85 0x86 0x87 0x88 0x89 0x8a 0x8b 0x8c 0x8d 0x8e 0x8f 0x90 0x91 0x92 0x93 0x94 0x95 0x96 0x97 0x80 0x81 0x82 0x83 0x84 0x85 0x86 0x87 0x88 0x89 0x8a 0x8b 0x8c 0x8d 0x8e 0x8f 0x90 0x91 0x92 0x93 0x94 0x95 0x96 0x97 fc2/1 fc2/2 fc2/3 fc2/4 fc2/5 fc2/6 fc2/7 fc2/8 fc2/9 fc2/10 fc2/11 fc2/12 fc2/13 fc2/14 fc2/15 fc2/16 fc2/17 fc2/18 fc2/19 fc2/20 fc2/21 fc2/22 fc2/23 fc2/24 off off off off off off off off off off off off off off off off off off off off off off off off not not not not not not not not not not not not not not not not not not not not not not not not installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed installed -------------------------------------------------
FICON
93
Also the running config will show the newly reserved ports
Switch-9509(config)# do sh run | incl assign no ficon slot 1 assign port-numbers ficon slot 2 assign port-numbers 128-151 force no ficon slot 7 assign port-numbers no ficon slot 9 assign port-numbers
FICON
94
Assign logical port-numbers for FCIP interfaces and Port-channels switch(config)# ficon logical-port assign port-numbers 16-31,224-249
Logical ports are shown in a show running config
Switch-9509(config)# do sh run | incl assign no ficon logical-port assign port-numbers no ficon slot 1 assign port-numbers ficon slot 2 assign port-numbers 128-151 force no ficon slot 7 assign port-numbers no ficon slot 9 assign port-numbers ficon logical-port assign port-numbers 16-31, 224-249 force
FICON
95
Objective & Agenda


FICON
96
Fabric Binding Troubleshooting

First, verify that identical Fabric Binding Databases are active on both switches Debug information:
debug fabric-binding all (collect from both switches) Activate the fabric binding database on both switchs while the debug is running. fabric-binding activate vsan X Flap the port-channel or ISL interface (Only if customer can take the outage)
Collect the following

show fabric-binding statis vsan X (x=vsan 500) show fabric-binding efmd statistics vsan X show fabric-binding database vsan X show fabric-binding database act vsan X show fabric-binding internal global show fabric-binding internal info vsan X show fabric-binding internal event show fabric-binding internal efmd event interface fcx/x or port-channel x
FICON
97
CUP Troubleshooting
Collect the following:
show tech-support details show ficon control-device sb3 show ficon control-device internal info show ficon control-device internal event-history vsan x (x=ficon vsan) show ficon control-device internal event-history errors show ficon control-device internal event-history msgs show ficon control-device internal event-history vsan <x> sh ficon internal event-history errors sh ficon internal event-history msgs sh ficon internal event-history vsan <x>
From the Mainframe:

MVS SYSLOG before, during, and after the failure. Output from D IOS,FICON
FICON
98
Basic Troubleshooting
Troubleshoot port problems (performance, failing) like any other case. Assume the switch is the problem and rule it out! attach mod x term len 0 show hardware internal debug-info interface fc x/x clear asic-cnt list-all-devices clear asic-cnt all clear asic-cnt device-id 3 From Supervisor show flogi internal event interface fcx/y (for both CH and CU interfaces) show port internal event-history interface fcx/y show ficon internal event-history vsan x interface fcx/y. show ficon internal event-history vsan 1 show ficon internal event-history msgs show ficon internal event-history errors show ficon internal info
FICON
99
Basic Troubleshooting
The list of commands that are useful from SAN-OS(later NX-OS) CLI: Setup ficon ficon slot X assign port-numbers 48-63 show ficon port-numbers assign slot 3 show ficon port-numbers assign logical-port show ficon vsan X portaddress brief show ficon vsan vsan-id file name filename portaddress show ficon vsan 20 portaddress 8 counters ficon swap portnumber old-port-number new-port-number ficon swap interface old-interface new-interface clear ficon vsan 1 allegiance ficon vsan 2 apply file SampleFile ficon vsan vsan-id offline ficon vsan vsan-id online show ficon control-device sb3 vsan X show flogi database vsan X
FICON
100
Basic Troubleshooting continued

Gather the following Mainframe displays before and after any failures or recreates D M=DEV(nnnn) D M=CHP(mmm) Collect the MVS SYSLOG before, during, and after the failure. Collect any messages that may appear on Hardware Console, for example, "LINK LEVEL ERROR FICON PLOGI ELS ERROR",
FICON
101
Summary / Review
FICON uses the Single-Byte Command Code Sets-3 (SB-3) and Single-Byte Command Code Sets-4 (SB-4) Three types of FICON topologies: Point-to-point, Switched point-topoint (via a FICON Director) ,Cascaded FICON Directors (through two FICON Directors) For FICON, there is a maximum of 1 hop from CH to CU Allow for fabric-binding between FICON fabrics Utilize setup ficon command Utilize GUI (FM - DM) for basic troubleshooting / configuration
FICON
102
References
FICON Planning and Implementation Guide: http://www.redbooks.ibm.com/abstracts/sg246497.html?Open MDS FICON Configuration Guide: http://www.cisco.com/en/US/docs/switches/datacenter/ mds9000/sw/4_1/configuration/guides/cli_4_1/ficon.html MDS Fabric Manager Configuration Guide http://www.cisco.com/en/US/docs/switches/datacenter/ mds9000/sw/4_1/configuration/guides/fm_4_1/ficon.html
FICON
103
Question and Answer
Thanks for attending this session and Good Luck in your CCIE Journey
FICON
104
FICON
105

FICON For CCIEstudy May

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

FICON For CCIEstudy May

Caricato da

Copyright:

Formati disponibili

FICON

By Ozden Karakok May 2011

2010 Cisco Systems, Inc. All rights reserved.

Objective & Agenda

SAN-OS 3.0 Update Troubleshooting FICON Summary

2010 Cisco Systems, Inc. All rights reserved.

IBM Mainframe Systems

OS/390 VM VSE/ESA MVS z/OS z/VM Linux

2010 Cisco Systems, Inc. All rights reserved.

ULP (Upper Level Protocol)

Single Byte Command Code Sets

SCSI Command Set Mapping (FCP)

Framing Protocol/Flow control

2010 Cisco Systems, Inc. All rights reserved.

2010 Cisco Systems, Inc. All rights reserved.

Channel Channel Images Control Unit Image

Control Unit (CU)

2010 Cisco Systems, Inc. All rights reserved.

2010 Cisco Systems, Inc. All rights reserved.

2010 Cisco Systems, Inc. All rights reserved.

2010 Cisco Systems, Inc. All rights reserved.

FICON Cascaded Topology

Portaddress = int fc X/Y

2010 Cisco Systems, Inc. All rights reserved.

2010 Cisco Systems, Inc. All rights reserved.

Objective & Agenda

2010 Cisco Systems, Inc. All rights reserved.

FICON Topologies (Contd)

FICON Control Unit

2010 Cisco Systems, Inc. All rights reserved.

FICON Topologies (Contd)

Single Switch Fabric Path Configuration

FICON Control Unit

FICON Control Unit

FICON Control Unit

FICON Topologies (Contd)

Two Switch Fabric Path Configuration

FICON Control Unit

FICON Control Unit

FICON Control Unit

Static in FICON fabric (usually 0)

Single Byte Link address to FCID mapping

Static in FICON fabric (usually 0)

Two Byte Link address to FCID mapping (for Cascaded environments)

HCD Point to Point

2010 Cisco Systems, Inc. All rights reserved.

2010 Cisco Systems, Inc. All rights reserved.

Using DM to identify Port Addresses

Notice FICON port numbers are displayed

Objective & Agenda

FICON Channel Initialization

Same Well Known Fibre Channel Addresses

2010 Cisco Systems, Inc. All rights reserved.

FICON Control Unit Initialization

Can register both SCSI and FC-SB FC-4 types

2010 Cisco Systems, Inc. All rights reserved.

QSA Query Security Attributes

QSA = 0x7E 00 00 00 Revision field = 0x00 00 00 01

2010 Cisco Systems, Inc. All rights reserved.

ESA Exchange Security Attributes

ESA Class F Ack ESA Accept Class F Ack

2010 Cisco Systems, Inc. All rights reserved.