Module 3

BROCADE Silk Worm

Generations of Brocade

Gen-5 8Gbps Gen-4 4Gbps Gen-3 2Gbps Gen-2 1Gbps Gen-1 1Gbps Bandwidth increased generation by generation(1Gbps 8 Gbps)

Outdated in 1998

Hardware components of Switch

RAM

ASIC FP-ROM

In normal systems the components are: : - CPU - RAM - Hard-Disk In embedded systems : - CPU - RAM - FP-ROM (In place of HDD) In Brocade switches - HDD is replaced by FP-ROM. - The processor is called ASIC - RAM is SD-RAM

ASIC (Application Specific Integrated Circuit)

ASIC (Application Specific Integrated Circuit): is a processor, specially designed for Brocade switches and has the entire functional logics of switch embedded in it, and more powerful than generic switch. There are five generations in ASIC: ASSIC Generation -1 Generation -2 Generation -3 Generation -4 Generation -5 Stitch Loom Bloom, Bloom-2 Golden Eye Contour

GENERATIONS OF ASIC
Gen-2 : Loom ASIC - Supports 4 ports - Speed 1Gbps - Frame Processing > 1.5 secs. GEN-3:Bloom ASIC - Supports 8 ports - Speed 2 Gbps - Auto negotiable - Frame Processing < 1.5 secs. GEN-3:Bloom II ASIC(2001 2006) - Supports 8 ports - Speed 2 Gbps - Auto negotiable - Frame Processing < 1.5 secs. Gen-4 : Golden-Eye (2005 2007) - Supports 16 ports - Speed 4 Gbps - Auto negotiable up to 1 Gbps - Frame Processing < 1 sec Gen-4 : Golden-Eye ( 2007 to -----) - Supports 16 ports - Speed 8 Gbps - Auto negotiable up to 2 Gbps - Frame Processing < 1 sec

Capacities of RAM and FP-ROM

Depending on the speed of the switch the Capacities of RAM changes, basically SD-RAM is used: The capacities are as follows: 32mb,64mb,128 mb,256mb .. The capacity of FP-ROM is initially very low as the cost of flash chips are expensive those days. Now the prices of Flash chips are slashed down and the flash chips with huge capacities are placed in the switch depending on the functional requirements. 32mb,64mb,128 mb,256mb n

FP-ROM
In embedded system FP-ROM plays major role as hard disk does in the normal computer. FP-ROM plays major role in switch, it is the heart of a switch. To operate an embedded system an application and Real time Operating system to organize the application are required.

Application
FOS.x VXWORKS & LINUX

RTOS

RTOS (Real Time Operating System)

RTOS is more powerful than normal Operating System, It consists of kernel, drivers and modules. In Brocade switches up to Bloom ASIC the RTOS is VXWORKS, later from Bloom 2 Linux RTOS is introduced.

VXWORKS: - It is a powerful RTOS that never crashes - It is very thin in size - It has almost all kinds of device drivers , maintained in binary file format Prior to Bloom II the cost of FPROM was high, later the prices slashed down the size of chip is not a cost factor anymore. Linux RTOS: - It is a powerful RTOS with more advanced features - It is thick in size - It has almost all kinds of device drivers in RPM format

Fabric OS (FOS)
FOS is an application that runs on top of RTOS.
FOS changes according to the generations Generation -1 FOS 1.x Generation -2 FOS 2.x Generation -3 Bloom FOS 3.x Bloom II FOS 4.x Generation -4 FOS 5.x Generation -5 FOS 6.x FOS contains Licenses to support various functional aspects of switch Licenses: Web tools Fabric Trunking Fabric Watch Advanced Performance Monitoring Extended Fabric Remote switch

Licenses of Switch
Licenses are required to support various functional aspects of switch. LICENSES
WEB TOOLS Default licenses Fabric Zoning Trunking Fabric watch Advanced Performance monitoring Extended Fabric Remote Switch To manage switch using browser To join switch in a fabric To create zones in the fabric To join multiple ISLs and aggregate bandwidth To set traps using SNMP to monitor switch To monitor the performance of switch. To extend switch more than 250 meters. To extend switch beyond 5 kms

Contd
ISL( Inter switch Link) :ISL is a cable used to join two switches and extend the scope of fabric. Trunking: To aggregate the bandwidth between the two switches, to reduce the latency, two switches are joined using multiple ISLs between quads of switches. SNMP (Simple Network Monitoring Protocol): is used in network management systems to monitor network-attached devices for conditions that warrant administrative attention. SNMP is a component of the Internet Protocol Suite as defined by the Internet Engineering Task Force (IETF). It consists of a set of standards for network management, including an application layer protocol, a database schema, and a set of data objects.

Classification of Switches

LOW
S i Model o ts: S . / wo s SIC: loo S i Model o ts: S . / wo s SIC: loo

Mi

HIGH

S i Model o ts: S . / in SIC: loo II S i Model o ts: S . / in SIC: loo II

S i Model o ts: S . / in SIC: loo II

S i Model o ts: * = S . / in SIC: loo II

Classification of Switches

LOW
Switch Model: o ts S in

Middle

HIGH
Switch Model: o ts S in

Switch Model: o ts S in

Switch Model: o ts S in

Switch Model: o ts S in

High end switches a e hot pl ggable and highly available, they a e called co e switches o di ecto switches

Brocade MPR
MPR (Multi Protocol Router): Router supports all protocols like- FCP - IFCP - FCIP - ISCSI

MPR AP 7420 FOS XPath RTOS Netbsd FOS XPath cannot be upgraded to FOS 5.X, but Netbsd can be replaced With Linux

MPR AP 7500 FOS 5.x RTOS Linux Both FOS 4.x and XPath are consolidated and FOS 5.x is released, with RTOS Linux

LEDS of Switch
LEDs on Front Panel: - Front panel of the switch has ports arranged in Quads. - Up on every port, there are two LEDs - First LED is status LED, and the next LED speaks of speed. - When a cable is connected to switch the status LED glows - If a LED glows in green, Flogi is successful - If a LED glows in amber, Flogi is unsuccessful - If port is running in maximum speed the speed led glows else,it is stopped as it was initially.
Port STATUS LED RJ -45 Port Speed LED DB-

. . Port Front View of Switch

LEDS of Switch
LEDS on Rear panel: A single LED is seen at the center of the back panel of the switch. - When the switch is first turned on the Colour of the LED is Amber. Colour of LED Green Amber Green Amber Green Amber

If the switch boots Else If the fan of Switch works properly Else If the power supply is normal Else

LED Power

FAN

Power Button

Rear Panel

DIFFERENT WAYS TO ACCESS SWITCH

Switch can be accessed through various tools and services.

Telnet Secure Telnet SSH Web Tools SNMP SMI - S Fabric Manager( GUI tool of brocade) Brocade API (Freely downloadable) Hyper Terminal

ACCESS LEVELS OF SWITCH

There are four access levels in switch, they are: 1. User 2. Admin 3. Factory 4. Root 1. 2. 1. 2. User : To monitor the switch. Admin : To perform administrative tasks, e.g. firm ware upgrades, config upload download etc. Factory : To set factory defaults. Root : Set the switch to factory defaults and also all tasks can be executed in this level.

INSTALL AND SANITIZE

Every brocade switch have serial/console port and Ethernet (RJ-45) port ,apart from the other ports. Newly shipped brocade switch is provided with Ethernet cable, power cable and serial cable. Serial cable connects host to the DB- port (management/serial port) Ethernet cable connects host to the RJ-45 port (management/serial port) By default the IP address of brocade switch is 10.77.77.77 Default user name and password - User name: admin - Password: password If no default user name, password or IP address are provided need to contact the vendor.

CONTD
Connect the cable to HyperTerminal or Tipper 600 ,(default setting) Default port configuration - login to the switch User name : Admin Password : Password - Set the IP address >ipaddrset (follow the instructions and set IP address) >ipaddrshow (displays the IP address) >licenceshow (displays the licenses assigned to the switch) - check the environmental conditions >tempshow (displays temperatures at power supply, fans, CPU, RAM. Temperatures vary from component to component but ambiant temperature shoud be 3F to 5F

CONTD
Switch level ventilation >fanshow ( displays RPMs of each fan) The difference between the RPMs of fans should be narrow, if the difference is more the fan with less RPMs should be replaced.

Power supply checking >psshow displays the rate of power consumption

Switch parameters >timeout 5 (current time out value of telnet) generally set 5minutes >quietmode 1 (to stop alert infos) >switchshow (displays the summary of the switch)

Configure switch (initial configuration of switch is disruptive, disable the switch) >switchdisable >configure (change the required parameters like domain ID, PID etc) :d (save and quit) :c (quit without save) >switchenable (To enable the switch) > config show (Displays the parameters of the switch) > license show (To show the licenses of switch) >license add (To add the license) >switch beacon 1 (To turn on the LEDs) >switch beacon 0(To turn off the LEDs)

CONTD

Switch port level commands: >portcfgshow (Switch ports and their summary - global) >portshow <port number>(To show the information of an individual port) >portenable <port number>(To enable a required port) >portdisable <port number>(To disable a required port)

Contd
- To configure ports: Configure a port as loop port >portcfgLport <port number> <true 1/false 0> Configure a port as expansion port >portcfgEport <port number> <true 1/false 0> Configure a port as trunk port >portcfgtrunkport <port number> <true 1/false 0> Configure a port as fabric/default port >portcfgdefault <port number> Configure the speed of a port >portcfgspeed <port number>,<0> Auto negotiation <1>(1Gbps) <2>(2Gbps) <4>(4Gbps)

Backup & Restore Switch configuration

To upgrade the firmware of switch, the configuration of the switch should be taken back-up. Upgrade of firm ware is to upgrade the version of FOS. Once the upgrade is completed restore the configuration if necessary. Back-up the configuration of switch is non disruptive Restoring the switch is disruptive Taking the back-up of switch is to upload the configuration file to the FTP server ,as the back-up cannot be performed in the local switch
Config. txt Config. FTP txt Config. txt

>configupload (Not disruptive)

Server

>switchdisable >configdownload (disruptive)

Switch

Firmware upgrade
FOS 3. x / VXworks FOS is a single binary file of size 2mb. Once the firmware is upgraded the previous version is overwritten by new version and cannot be reverted FOS 4. x / Linux FOS is a zip file with a bundle of rpms of size 40 mb. Once firmware is upgraded, it will not over-write the previous older one. It divides FP-ROM into two partitions, primary and secondary. If the newer version is not supportive, we can revert the changes and retain to the older firmware. If the latest version is comprehensive ,the upgrade can be saved, and reboot is required for the changes to take effect.

Contd
Download the upgrades from the website to the FTP server and later down load to the switch for upgrade. The version difference to upgrade/downgrade the firmware should be two.

Firmware upgrade

FOS 4.0

FOS 4.2

FOS 4.4

Firmware Down grade

1. Download firmware from website to FTP server 2. Back-up/upload switch configuration to FTP server. 3. Download firmware to switch 4. Restore / Download switch configuration to switch from FTP server
Firmwar e Firmwar e

http://

Config.t xt

Config.t xt

3 2 Switch 4
Config.t xt Firmwar e

Security
Zoning : To create the virtual bus in a physical san to limit the scope of initiator and enhance security is called zoning. SAN ensures any to- any connectivity, which has advantages as well as disadvantages. Any host can access any Lun,that results in threat to the security. Security can be implemented at various levels across the fabric: - HBA Selective Lun Presentation - Storage sub system Lun Security / Lun masking - Switch Zoning

Selective Lun presentation

Initially the security is conducted by HBA driver, which enables selective lun presentation HBA driver detects the number of drives, that a host should be able to Access.e.g lpfc driver. According to our illustrated figure it is clearly visible that 2 hosts are connected to Jbod via hub. Now both the hosts can access all the drives in the Jbod, as initiator (HBA) probes and identifies the discs. Hosts accessing all the drives may arise complications like corruptions hangouts etc., to restrict the hosts from accessing all the discs and confine their authorization to access a particular set of discs, Selective lun presentation was introduced.

Hub Jbod with 20 discs

Selective lun presentation process

1. In this illustrated figure how the discs are presented to the host is disclosed. sd.conf 2. Host identifies number of presented discs through sd.conf. 3. In this example, 10 discs out of 20 in Jbod are presented to the host. 4. The information pertaining to the presented discs is mapped by host from sd.conf. 5. But HBA probes identifies and maps the entire discs in Jbod. 6. It is the responsibility of lpfc to suppress the access to unwanted discs by entering them entering their wwns and disabling them in lpfc.conf.

Lpfc.conf

HBA

Disadvantages of selective lun presentation

Selective lun presentation process is vast and those days lpfc is not dynamic and any new additions of discs should be entered manually in lpfc.conf files in entire host in the network which is practically impossible in the environments where there are more than two hundreds of hosts.

The next layer of HBA was introduced with dynamic lpfc drivers which are used by switches to be the member of zone to provide security to Luns, but this concept is confined to Jbod level. - lp8000 is static -lp9000 is dynamic. At switch level discs of Jbod are visible but not the luns of storage sub system. To provide lun security, the following techniques are developed. - Lun mapping - Lun security - Lun masking

Zoning
In a fabric there can be many number of storage sub systems as well as hosts with heterogeneous operating platforms and applications and conducted via devices like switches ,hubs etc. In this vast infrastructure ,if the access between hosts and storage sub systems is not aligned and restricted properly, this turns to be major threat to data. - By any chance two hosts within the fabric with two different operating environments access same lun,its catastrophic. - To a fabric up to 16000 devices can be connected, if a host reboots and initiator starts probing the devices, It takes the management host days together to boot. Considering the above adverse factors ,the concept of zoning has evolved, to restrict the access between hosts and storage subsystems and assign a few particular hosts to gain access to the storage sub systems Zoning : To create the virtual bus in a physical san to limit the scope of initiator and enhance security and performance is called zoning.

2 Z1 1

3 4 Z2 5

Zoning
Z1 Zone 1 Z2 Zone2
Zone1: Allows hosts 1 and 2 to access storage subsystem A. Remaining hosts are forbidden. Zone2: Allows hosts 4 and 5 to access storage subsystem B Remaining hosts are forbidden. Host 3 belongs to none of the both zones, it has no access to any of the storage sub systems.

SAN

TYPES OF ZONING
In brocade point of view we have two types of zones - Hardware enforced - Software enforced

HARDWARE ZONING
Static zone Works with domain ID and port number Frames are filtered by ASIC Secured but not flexible Lack of flexibility is the main disadvantage, for every change at switch level should be configured which may happen quiet often - Device zoned to a particular port cannot be a member of zone if it is connected to another port in a fabric - If a domain ID of a switch changes, zones will be disabled - If any switch component is replaced, the zones should be re-configured, but this is an exception for HBA - If HBA is replaced at host level need not re-configure

Hardware Zoning
zone

Host Storage 0 5 0 0 0 0 0 5 0 1 0 0

HOS T

0 5 0 2 0 0

Software Zoning

zone Host
HOS T

Storage

5 0 0 1 0 0

0 5 0 1 0 0 1 0 0 1 b 4 0 5 0 1 2 0 1 0 0 1 0 f 0 e 0 1 4 3

wwpn

SOFTWARE ZONING
In brocade switches software zoning traverses through two phases, one phase Is up to loom and another phase starts from bloom Software zoning at loom level -Dynamic zoning -Uses wwpn devices and wwnn of Software zoning from bloom level -Dynamic zoning -Uses wwpn and wwnn of devices -Session level verification by name server -Frame filtration by ASIC -flexible and secured Reconfiguration of zone required only when HBA is replaced

-Session level verification by name server -flexible

MIXED ZONING
Zoning that is performed using WWPNS as well as domain and port ids is mixed zoning. ASIC level frame filtration is not performed Session level verification is done It is less secured when compared to hardware zoning as well as software zoning and used in very small scale environments

ZONE ARCHITECHTURE
OBJECTS OF ZONE Member s
Members WWN numbers of Hba's , discs as well as storage sub systems are given alias names called members Zones Combination of multiple members is called Zone

Zones

Configs

Configs Combination of zones or a zone is called config. There may be any number of configs, but only one config is enabled at a time

Associating objects of Zone

2 Z1 1 3 4 Members 1 5 A 2

SAN

Zone 1 (Z1) 1 A Config cfg1 2

Zone 1 (Z1)

1 2 A

Associating objects of Zone

2 Z1 1

3 4 Z2 5

Members 1 A 2

Members 4 B Zone 2 (Z2) 4 B Config cfg2 5 5

SAN

Zone 1 (Z1) 1 A 2

Zone 1 (Z1)

Zone 2 (Z2)

1 2 A

4 5 B

Features of Zone
Zone allows grouping of SAN devices, which communication each other. Zone or a group of zones are bound to the configuration. Zone should include atleast one initiator member and one target member Zone is always fabric centric, but not switch centric.

Configuration Management
Configuration plays important role in the fabric Configurations are stored in FP ROM ASIC looks in to FP ROM and finds the configuration which is loaded in RAM This way switch internal hardware, CPU, RAM and FP ROM manage zones. Any time only one configuration is effective across the fabric. If another configuration is required ,never disable effective configuration, always enable required configuration upon the effective configuration. Upon enabling the new configuration on effective configuration, the current configuration will become dormant

Types of Configuration
Defined configuration Effective configuration Saved configuration Disabled configuration

Defined Configuration : configurations defined by the user to implement in a fabric is called defined configuration. Effective configuration : configuration which is enforced in a fabric Saved configuration : defined configuration + Effective configuration Disabled configuration : remove effective configuration from RAM

Types of configuration
RAM
Defined config (cfg1 + cfg2) Effective configuration cfg1 If the configuration is removed from RAM, it is called disabled.

>cfgdisable Disable the configuration

Saved Configuration (cfg1 & cfg2) + cfg1

AS IC

> cfgclear-Remove info from RAM & FPROM. > cfgsave -Save the configuration (To sanitize switch) > cfgenable - Enable configuration > cfgshow To see the all configurations

FP ROM

Addition a switch to the fabric

While adding a switch to the fabric many factors should be considered. The factors differ if the switch added is : 1) New switch to fabric 2) Existing switch with its own configuration

New switch to fabric

ISL RAM
Defined config (cfg1 Z1) Effective config Cfg1 Z1 Defined config (cfg1 Z1)

RAM
Effective config Cfg1 Z1

Saved Configuration (cfg1) + cfg1

AS IC

Saved Configuration (cfg1) + cfg1

AS IC

FP ROM
OLD SWITCH

FP ROM
NEW SWITCH

New switch to fabric

When a new switch is attached to the fabric, it has no configurations stored either in RAM or FP-ROM. Once the new switch is joined to the switch in fabric, the following propagations takes place: 1) Defined configuration in the inactive portion of RAM in old switch is propagated to inactive portion of the RAM in New switch. 2) Effective configuration in the active portion of RAM in old switch is propagated to active portion of the RAM in New switch 3) Later the configurations are propagated to FPROM

Thus the new switch added to the fabric participates in enforcing existing effective configuration

Addition of switch with own configurations

Before Connecting switches RAM
Defined config (cfg1 Z1) Effective config Cfg1 Z1 Defined config (cfg2 Z2)

RAM
Effective config Cfg2 Z2

Saved Configuration (cfg1) + cfg1

AS IC

Saved Configuration (cfg2) + cfg2

AS IC

FP ROM
OLD SWITCH

FP ROM
Existing Switch with own configuration

After Connecting switches RAM

Defined config (cfg1 Z1) (cfg2 Z2) Effective config Cfg1 Z1 Defined config(cfg1 Z1) (cfg2 Z2)

RAM
Effective config Cfg1 Z1

Saved Configuration (cfg1)(cfg2) + cfg1

AS IC

Saved Configuration (cfg2)(CFG1) + Cfg1

AS IC

FP ROM
OLD SWITCH

FP ROM
Existing Switch with own configuration

After Connecting switches RAM

Defined config (cfg1 Z1)(cfg2 Z2) (cfgall(z1,z2) Effective config Cfgall(z1,z2)

RAM
Defined config (cfg1 Z1)(cfg2 Z2) (cfgall(z1,z2))
Effective config

(cfgall(z1,z2))

Saved Configuration (cfg1)(cfg2)(cfgall ) + cfgall

AS IC

Saved Configuration (cfg2)(CFG1)(cfg all) + Cfg1

AS IC

FP ROM
OLD SWITCH

FP ROM
Existing Switch with own configuration

Addition of switch with own configurations

While joining a switch with own configuration to the new switch following conflicts may arise: - Zone content conflicts - Zone type conflicts - Zone name conflicts The above conflicts have to be resolved by identifying them, by using the third party software tools specially designed to diagnose the conflicts mentioned above. Once the conflicts are resolved join the switch, the connection may not be established as the effective configurations conflict, to resolve the mismatch. 1) Disable the E-port of new switch to disable connectivity between switches >portdisable <port number> 2) Disable the configuration of new switch to erase the configurations in RAM. >cfgdisable

Addition of switch with own configurations

3) Enable the E-port to open communication between switches >port enable <port number> Once the port is enabled the communication between the two switches open up and the configuration of new switch is propagated to the existing switch and loaded and saved in FPROM. Immediately new defined configuration is loaded to the inactive partition of RAM. The defined as well as effective configurations are propagated to their respective partitions of RAM on the new switch.

Inter Switch Link (ISL)

ISL (Inter Switch Link): - The cable that merges two switches are called ISL. - Used for routing and load balancing in fabric. - Allows exchange of switch parameters between two switches. - ISL is divided into 8 channels, through which data is transferred. - Control packets and data packets are sent through ISL for data integrity. The 8 channels of ISL are (VC-0 VC-7) - The priorities are set to channels to be identified by ASIC . - Based on such priorities buffers are assigned. - ISL has 27 buffers allocated to channels depending upon the requirement and data flow.

Virtual Channels of ISL

VC0 VC1 VC2 VC3 VC4 VC5 VC6 VC7 VC0 VC1 VC2 VC3 VC4 VC5 VC6 VC7

S w I t C h 1

S w I t C h 2

S w I t C h 1

VC0 VC1 VC2 VC3 VC4 VC5 VC6 VC7

Vc0 priority -1 Transfers Link control frames Vc1 priority -2 Link control and link control Vc2 priority -3 Transfers data frames Vc3 priority -3 Transfers data frames Vc4 priority -3 Transfers data frames Vc5 priority -3 Transfers data frames Vc6 priority -4 Multicast traffic Vc7 priority -4 Broad-cast traffic

Priorities of Buffer
ISL has 27 buffers that has been shared among the Virtual Channels based on the priority and requirement level. Priority 1 Priority 2 Priority 3 Priority 4 level level level level One buffer Zero buffer Six buffers One buffer 1*1 1*0 6*4 1*2 1 0 24 2

ISL Subscription Ratio - As suggested by the brocade for every three hosts ISL should be used for better performance. - 3:1 - Ratio Sum of all devices bandwidth : Sum of all ISLs bandwidth (Number of device ports : Number of E ports e.g. 3:1,7:1,15:1) - Maximum number of hops suggested between devices using ISL is 7 - In a fabric core edge - topology is followed.

CORE EDGE TOPOLOGY

Switch Switch Core switch Switch

Switch

Switch

Switch

Storage

Principal Switch/Sub-ordinate switch

In a fabric 239 switches can be connected Only one switch can be a principal switch in a fabric ,that maintains domain integrity. Remaining switches in the fabric are called sub-ordinate switches principal switch maintains information of all the sub-ordinate switches. principal switch updates its time from NTP server and pushes the same to the sub-ordinate switches. During reconfiguration there is a chance for down time of atleast 10 seconds. From FOS 4.x static configuration of switch was introduced to avoid down time.

Reconfiguration of Fabric
Reconfiguration of fabric takes place when a new switch is joined to the fabric. If ISL is connected between the switches they immediately merge, and messages will be generated. To establish communication between two switches the 24 bit dynamic address is required The first octet consists of domain ID which should be unique for both the switches, independently It takes 10 seconds for principal switch to ensure uniqueness of Domain ID Any switch at the time of booting is a sub ordinate switch

Merge two switches with same domain ID

principal Switch Domain ID 1 Sub-ordinate switch Domain ID 1

- Once the sub-ordinate switch is powered on, principal switch send Domain ID list to sub ordinate switch. Now domain ID is conflicting Sub-ordinate switch updates its domain ID which ever is available Sends the confirmation of updated domain ID to the principal switch

Merge two principal switches

principal Switch Domain ID 1 principal switch Domain ID 1

Both are principal and both have same domain IDs Now there is a race condition, as exchanging domain Id is conflicting So the connection is segmented as domain overlapped. To resolve this issue - Login to switch - Disable the switch - Set the configuration

Merge two principal switches

principal Switch Domain ID 1 principal switch Domain ID 2

Both the switches are up and are running Domain IDs are unique and different But both are principal. Connect the ISL cable The switch with least wwn is considered as principal switch

Exchange Parameters
1. 2. When two switches are connected using ISL cable the following process takes place Ports on both the switches to which the ISL is connected are reconfigured to expansion ports E-port Once configured switches generate ELP (Exchange Link Parameters) - if the parameters match switches proceed to next process - if parameters mismatch a message is generated stating ELP- rejected - find out the parameter that conflicts and rectify the parameter Exchange fabric parameters such as static & dynamic addresses Exchange WWN principal switch sends DIA ( Domain Identifier Address) Up on receiving DIA sub-ordinate switch sends confirmation RDIA (Received Domain Identifier Address) Once the above parameters are exchanged the switch becomes the member of fabric

1. 2. 3. 4.

Exchange Parameters
Principal Switch E-Port

ISL

Exchange Link Parameter Exchange Fabric Parameter Exchange WWN Domain Identifier Address Received Domain Identifier Address E-Port Sub-ordinate Switch

Fabric Shortest Path First

Fabric shortest path first: - It is first proposed by Brocade - Now it is available with almost all SAN switches - It is basically routing protocol What is Path? - Distance between two points - Reference point is required (hops) - Involves source and destination What is route? - It is a part of a path - A map between in port to out port. - Finally reaches the destination

FSPF
FSPF down loads routing table to ASIC. The cumulative cost of all ISLs between source and target is called ISL cost Cost is calculated based on the speed
2Gpbs 4Gpbs 1Gpbs

500

250

750

The total ISL cost 250+500+750 1500 ASIC routes switch to switch frames using lowest cumulative costs of all available traversal links Trunks with aggregated higher band-width are favored over 1 Gbps FSPF implemented using FOS FSPF does load balance at path level

ISL Cost
Each ISL has a metric cost based up on band width Cumulative cost based up on sum of all costs of all traversed ISL Path selected is the one with the minimum cost Once the path is selected the path is used until it fails ROUTE : - A map used to reach the next hop between import and out port with in a path. - Only 7 hops are allowed between source and destination switch. 500

50 0

75 0 500

50 0

25 0 500

LINK COST METRIC

Link cost may be dynamic or static By default it is set dynamic Link cost set on out port Link cost is not bi-directional The default metrics are 1000 at 1Gbps 500 at 2 Gbps 250 at 4 Gbps Link cost and bandwidth are inversely related .

DYNAMIC LOAD SHARING

Allows load sharing across multiple equal cost paths Provide better utilization of ISL and increase performance Its selects the path in round robin way using algorithm developed by brocade The sharing is switch centric If DLS is set and any host is removed the other host will shift to next ISL a Round robin sharing b c

DYNAMIC LOAD SHARING

1 2 3 1 2 3 1 2 3

TRUNKING

Combines multiple ISLs in to a single ISL Minimum two ISLs are required Maximum limit is four ISLs in one trunk group should be from the same quad Trunk does frame level load balance Supports multiple trunk between switches

TRUNKING

ISL

Frames

Contd
Preserves in-order delivery Trunking license key is required on both the sides of ISL Ports must be operating at the speed of 2Gbps Between switches different lengths of cables are used Difference between longest ISL and shortest ISL is called diskew value

IOD (In order delivery)

Frames are delivered in order in spite of being traversed across multiplepath. If any changes occurs in the topology, the principal ISL goes down. the frames may go out-of order, during a brief time during the RCF. The default behavior is to automatically enable out-of-order frames during fabric topology changes. To reroute frames after a fabric topology change and ensure in order delivery.

- iodreset : If an E-Port route goes down ,frames are transmitted across a new route or re-uses previous route. - iodset : Once the Para meter is set ,the frame are delivery in order - iodshow : To show if the IOD is set or not