HP/H3C Advanced Enterprise Networking Intelligent Resilient Framework (IRF)Operation and Configuration

2010

Contents
Section 1 IRF Principle and Implementation Section 2 Basic Configuration of the IRF Section 3 IRF Maintenance Section 4 Split Stack Detection

IRFv2: Members, Roles and Topology

Daisy Chain
Master
IRF-port 1 IRF-port 2

Slave
IRF-port 1 IRF-port 2

Slave
IRF-port 1 IRF-port 2

Slave

Ring

Master
IRF-port 1 IRF-port 2

Slave

IRF-port 2

IRF-port 1

IRF-port 1

IRF-port 2

IRF-port 2

IRF-port 1

Slave

Slave

IRFv2 IRF Connections

IRFv2 systems are connected using any 10 GbE interface:

CX4 SFP+ XFP XENPAK

Inexpensive Local Connection cables are available for CX4, SPF+ and XFP ports.

Basic Concept - UNIT ID

UNIT ID identifies the unit/switch uniquely in the IRF Fabric. Auto numbering: The default Unit ID need not be configured by the user and the Unit ID can be automatically allocated through the fabric cable connection. Manual numbering: In some special cases, the Unit ID can be manually modified through the command line. 1 2 3 4

String connection

Ring connection

4
1

Basic Concept - Master/Slave

The Master Unit is elected from the Fabric to accomplish important operations (such as routing delivery). Slave: Other Units except Master. The Unit with the highest priority number is the master.
Master
1

Slave
2

Slave
1

Slave

IRF Merge: Master Election

1. The current master wins, even if a new member has a higher priority. (When a new member is added, IRF merge does not happen.) 2. A member with a higher priority wins. 3. A member with the longest system up-time wins. (The precision of the system up-time is ten minutes.) 4. A member with the lowest bridge MAC address wins.

3Com Confidential 7

Basic Concept IRF-Port

Each unit has a logical IRF-Port defined Physical interfaces are placed into the logical IRF-Port IRF-Ports connect from Port 1 to Port 2 from device to device

2 Downlink port connects the uplink port

Basic Concept System Events

IRF system events indicate the global events that possibly occur in the IRF, which need be processed by each Unit and module to ensure that the IRF can normally run upon these events. Simple system events include Unit Up and Unit Down Complicated system events include Join, Leave, Merge and Split These events can basically describe all the changes in IRF. Master
1

Slave
2

Slave
1

Slave

Basic Concept - Merge

Merge: Two IRF systems are merged to form a new IRF system. Re-numbering may be caused in the case of auto numbering. Two Masters will compete to gain a new Master. Configuration comparison: Units configured differently from the Unit with the highest priority number will be restarted. Master
1

Master
2

merge
4 3

Slave

Slave

IRFv2 Feature Comparison Summary

Feature
Max Nr. of 10GbE ports per IRF Port Display command for IRF ports IRF port traffic statistics IRF Split Detection (MAD) Auto member id change (when member id collision happens)

3Com 4800G H3CS5500-EI

2 No No Yes

H3C S5800 H3C S5820X

4 Yes Yes Yes

3Com S7900E H3C S7500E

8 Yes Yes Yes

Yes

No

No

Contents
Section 1 IRF Principle and Implementation Section 2 Basic Configuration of the IRF Section 3 IRF Maintenance Section 4 Split Stack Detection

IRF Principle and Implementation Configuration

All configurations are divided into global configuration and local configuration. Global configuration includes layer 3 interface, IP address, routing protocol and security feature. It is efficient in the whole Fabric. Local configuration mainly includes the port parameters and it is efficient in a local Unit only. One Fabric ensures that the global configurations are identical when the system is running.

How to implement it? 1. Begin with as minimal of a configuration as possible to ensure the global configurations are identical. 2. The global configuration will be broadcast to each Unit for synchronous execution and the local configuration will be sent to the relevant Unit for execution.

IRF Principle and Implementation Configuration Comparison

Begin with a minimal configuration on all devices. Preferred to reset savedconfiguration and reboot prior to starting. Configuration comparison will be performed at of system start, new Unit joining or merge. The Unit with the highest priority serves as the reference in the configuration comparison. All other devices are re-written except for the local port and IRF configuration.

Fabric configuration Global configuration Local allocation Global configuration

Fabric configuration Global configuration Local allocation Global configuration

UNIT1

UNIT2

Basic Configuration Steps

1. On the chassis-based switches, convert them to IRF-mode 2. Choose a switch to be the master switch 3. Assign a high IRF priority to the master and ensure its Member ID is 1. 4. Assign a different Member ID to each one of the other switches in the IRF group and reboot them. 5. Shut down all the physical ports that will be used for the IRF connectivity. 6. Assign the physical ports to the IRF logical interface 7. Enable the physical ports used with IRF (cont.)
3Com Confidential 15

Basic Configuration Steps (cont.)

8. Save the configuration of each device and turn them off. 9. Physically connect the IRF links to build the IRF fabric.
(See the next slide for connection guidelines) Wait until the boot process is complete before turning on the next device. This will guarantee that this unit will become the master.

10. Turn on the unit that needs to be the master (Member-id=1)

11. Repeat the process for each member (turn on and wait). This step is called device insertion. Always turn on a device connected to other devices that are already up and running.

3Com Confidential 16

IRF Connections
IRF Port Numbering
Syntax: irf-port x/y
X is the member number Y is the logical port number

Logical Ports
IRF logical port 1 of one device must be connected to IRF logical port 2 of the next device Connecting IRF-ports of the same logical number will prevent the devices to recognize each other as members of the same IRF group

IRF Configuration Example:

irf-port 1/2 irf-port 1/1 irf-port 2/2 irf-port 2/1 irf-port 3/2 irf-port 3/1

Master Member 1

Slave Member 2
3Com Confidential 17

Slave Member 3

IRF Configuration Master 1 (Step 1)

On Chassis Systems, you must first enable IRF mode (only applies to the 7500E and highernot the 5820x and 5500-EI): [H3C] chassis convert mode irf This command will convert the device to IRF mode and the device will reboot. Are you sure? [Y/N] y

The device reboots automatically to switch its operating mode. (To reverse this command enter the undo chassis convert mode command)

IRF Configuration Master 1 (Steps 2 and 3)

Assign the IRF Priority of the switch and save the configuration: [H3C] irf member 1 priority 32
Where priority can be an integer value from 1 to 32, 1 being the lowest 32 being the highest priority. Highest priority take precedence over a lower priority.

[H3C] save

IRF Configuration Master 1 (Step 5)

Shutdown the ports that are to operate as IRF-Ports: [H3C] interface TenGigabitEthernet 1/3/0/25 [H3C-Ten-GigabitEthernet1/3/0/25] shutdown [H3C-Ten-GigabitEthernet1/3/0/25] quit [H3C] interface TenGigabitEthernet 1/3/0/26 [H3C-Ten-GigabitEthernet1/3/0/25] shutdown [H3C-Ten-GigabitEthernet1/3/0/25] quit
Notice the extra number in the interface designation. The first number (1/) is the Chassis ID (member number) associated by IRF.

IRF Configuration Master 1 (Step 6)

Assign the ports to their respective IRF-Port: [H3C] irf-port 1/2 [H3C-irf-port 1/2] port group interface TenGigabitEthernet 1/3/0/25 [H3C-irf-port 1/2] port group interface TenGigabitEthernet 1/3/0/26 [H3C-irf-port 1/2] quit Note: This is Logical IRF-Port #2. This port MUST be connected to another System 2s logical port #1

IRF Configuration Master 1 (Step 7 and 8)

Re-activate the assigned ports: [H3C] interface TenGigabitEthernet 1/3/0/25 [H3C-Ten-GigabitEthernet1/3/0/25] undo shutdown [H3C] interface TenGigabitEthernet 1/3/0/26 [H3C-Ten-GigabitEthernet1/3/0/25] undo shutdown [H3C-Ten-GigabitEthernet1/3/0/25] quit [H3C] irf-port-configuration active %May 5 23:18:40:124 2010 H3C STM/4/MERGE: IRF merge occurs and the IRF system does not need to reboot. [H3C] save
1

IRF Configuration Slave 2 (Step 1)

On Chassis Systems, you must first enable IRF mode: [H3C] chassis convert mode irf This command will convert the device to IRF mode and the device will reboot. Are you sure? [Y/N] y The device reboots automatically to switch its operating mode. A configuration register is set to indicate IRF mode To reverse this command enter the undo chassis convert mode command

IRF Configuration Slave 2 (Steps 2, 3, and 4)

Renumber the unit to the second IRF device: [H3C] irf member 1 renumber 2 Warning: Renumbering the switch number may result in configuration change or loss. Continue? [Y/N]: y [H3C] quit <H3C> reboot
You can validate the IRF unit number using the display irf command. Note: The member number on the 7500 and higher series can be 1 to 4. Today you CANNOT assign a S7510E as unit number 4 and if you set a 7506E to unit 4, the last two card slots will NOT operate!!!
1

IRF Configuration Slave 2 (Step 5)

Shutdown the ports that are to operate as IRF-Ports: [H3C] interface TenGigabitEthernet 2/3/0/25 [H3C-Ten-GigabitEthernet2/3/0/25] shutdown [H3C-Ten-GigabitEthernet2/3/0/25] quit [H3C] interface TenGigabitEthernet 2/3/0/26 [H3C-Ten-GigabitEthernet2/3/0/25] shutdown [H3C-Ten-GigabitEthernet2/3/0/25] quit
Notice the extra number in the interface designation (2/). This is the Chassis ID (member number) associated by IRF.

IRF Configuration Slave 2 (Step 6)

Assign the ports to their respective IRF-Port: [H3C] irf-port 2/1 [H3C-irf-port 2/1] port group interface TenGigabitEthernet 2/3/0/25 [H3C-irf-port 2/1] port group interface TenGigabitEthernet 2/3/0/26 [H3C-irf-port 2/1] quit This places both ports into the Logical IRF-Port of 2/1 Note: that this is Logical IRF-Port #1. This port MUST be connected to another switchs logical port #2
1

IRF Configuration Slave 2 (Step 7 and 8)

Re-activate the assigned ports and save the configuration: [H3C] interface TenGigabitEthernet 1/3/0/25 [H3C-Ten-GigabitEthernet1/3/0/25] undo shutdown [H3C] interface ten-gigabitethernet 1/3/0/26 [H3C-Ten-GigabitEthernet1/3/0/25] undo shutdown [H3C-Ten-GigabitEthernet1/3/0/25] quit [H3C] irf-port-configuration active %May 5 23:16:57:541 2010 H3C STM/4/MERGE: IRF merge occurs and the IRF system needs a reboot. [H3C] save [H3C] quit <H3C> reboot
1

IRF Configuration Completion

When you connect the two switches together:
Chassis switches - The unit with the lowest priority should request at the command line to be rebooted Stackable switches The unit with the lowest priority will automatically be rebooted!!!

Once the unit(s) come back up, they will be in full IRF mode. This can be validated with the display irf command

Contents
Section 1 IRF Principle and Implementation Section 2 Basic Configuration of the IRF Section 3 IRF Maintenance Section 4 Split Stack Detection

IRF Troubleshooting

Operation Display the information of the whole Fabric Display the Fabric topology management information Display the IRF configuration display irf

Command

display irf topology

display irf configuration

display irf configuration

[H3C] display irf configuration MemberID 1 2 NewID 1 2 IRF-Port1 Ten-GigabitEthernet1/2/0/1 Ten-GigabitEthernet1/2/0/2 disable IRF-Port2 disable Ten-GigabitEthernet2/2/0/1 Ten-GigabitEthernet2/2/0/2

display irf

<H3C> display irf Switch Slot Role Priority CPU-Mac *+1 0 Master 1 00e0-fc0a-15e0 2 1 Slave 1 00e0-fc0f-8c02 -------------------------------------------------* indicates the device is the master. + indicates the device through which the user logs in. The Bridge MAC of the IRF is: 000f-e26a-58ed Auto upgrade : no Mac persistent : always Link-delay timer : 0 ms Domain ID : 30

display irf topology

<H3C> display irf topology Topology Info -----------------------------------------------------------------------IRF-Port1 IRF-Port2 Switch Link neighbor Link neighbor Belong To 1 DIS -UP 2 00e0-fc0a-15e0 2 UP 1 DIS -00e0-fc0a-15e0

Contents
Section 1 IRF Principle and Implementation Section 2 Basic Configuration of the IRF Section 3 IRF Maintenance Section 4 Split Stack Detection

Basic Concept Split Stack

Split: One IRF Fabric is split into one or several IRF systems due to disconnection of its internal links. Each new IRF system will elect its own new Master. Layer 2 and layer 3 protocol conflicts probably will occur upon the split.

Master
1

Master
2 MAD

Switch

Slave

Slave
1

IRF Split Stack Problems

Layer-2:
STP loops Forwarding Duplicate switch MAC addresses Port! Possible duplicate forwarding of frames Forwarding Port!

Layer-3:
Duplicate IP addresses Possible duplicate packetsboth routers forwarding packets Broken IRF Link Master! Master!

3Com Confidential 36

Multi-Active Detection (MAD): Split Detection Protocols

MAD using LACP:

Uses a distributed Bridge Aggregation interface connected to a 3rd device to exchange MAD information (only supported by H3C devices) To support this function LACP has been extended with MAD specific TLV fields.

MAD using BFD:

a special VLAN with ports in each member must be configured and each member device must be configured with an MAD IP address. These addresses are invisible for the rest of the network and no routing interface can be attached to an MAD/BFD enabled VLAN.

Recommendation: Use BOTH!

3Com Confidential 37

MAD: Collision Handling and Failure Recovery

Collision Handling
The part that contains the device with the lowest member-id remains in Normal state and the other goes into Recovery state. The ports of a device in recovery state become blocked. The administrator can exclude some ports from becoming blocked.

Failure Recovery
When the IRF link is back online, the IRF system detects that the IRFports are up and triggers the Recovery process. During the recovery, the part of the IRF that was in recovery state is rebooted to be re-inserted into the IRF.

3Com Confidential 38

MAD and LACP Process

Requirements:
IRF link is separate from LACP link Switches are directly connected on LACP link LACP link can be used for normal data functions LACP Link Blocked Port

When the two masters see each other on the LACP link and the IRF link is down The lowest member ID goes into a Normal state The other member goes into a Recovery state

Broken IRF Link Device in Recovery state

Device in The Recovery member disables its Normal state other ports until the stack is re-merged

3Com Confidential 39

IRF Configuration Enable LACP MAD

Master
1

Master
2
MAD

Switch

To enable MAD over LACP:

[H3C]interface bridge-aggregation 2

Slave

Slave
1

[H3C-Bridge-Aggregation2] link-aggregation mode dynamic [H3C-Bridge-Aggregation2] mad enable [H3C-Bridge-Aggregation2] quit Add ports 1/3/0/2 and 2/3/0/2 to the aggregation interface and they are dedicated to the LACP MAD detection for Switch One and Switch Two. [H3C] interface gigabitethernet 1/3/0/2 [H3C-GigabitEthernet1/3/0/2] port link-aggregation group 2 [H3C-GigabitEthernet1/3/0/2] quit [H3C] interface gigabitethernet 2/3/0/2 [H3C-GigabitEthernet2/3/0/2] port link-aggregation group 2
1

MAD and BFD Process

Requirements:
Used when an intermediate layer-3 device is between the two IRF switches BFD is used via a layer-3 link (UDP connection) BFD TCP Link Intermediate Layer-3 Device Blocked Layer-3 processing

When the two masters see each other on the BFD UDP connection and the IRF link is down The lowest member ID goes into a Normal state The other member goes into a Recovery state The Recovery member stops processing layer-3 traffic

Broken IRF Link Device in Normal state Device in Recovery state

3Com Confidential 41

IRF Configuration Enable BFD MAD

MasterMAD Master
1 2

[H3C] vlan 3 [H3C-vlan3] port gigabitethernet 1/3/0/1 [H3C-vlan3] port gigabitethernet 2/3/0/1 [H3C-vlan3] quit

Slave

Slave

Create VLAN-interface 3 and configure the MAD IP address for the interface. [H3C] interface vlan-interface 3 [H3C-Vlan-interface3] mad bfd enable [H3C-Vlan-interface3] mad ip add 192.168.2.1 24 chassis 1 [H3C-Vlan-interface3] mad ip add 192.168.2.2 24 chassis 2 [H3C-Vlan-interface3] quit
1

display mad
<H3C> display mad MAD LACP enabled. MAD BFD enabled. # Display the detailed MAD configuration. <H3C> display mad verbose Current MAD status: Detect Excluded ports(configurable): GigabitEthernet 2/1/0/2 GigabitEthernet 2/1/0/3 Excluded ports(can not be configured): Ten-GigabitEthernet1/2/0/25 Ten-GigabitEthernet2/2/0/26 MAD enabled aggregation port: Bridge-Aggregation2 MAD BFD enabled interface: Vlan-interface10 mad ip address 10.0.0.2 255.255.0.0 chassis 1 mad ip address 10.0.0.3 255.255.0.0 chassis 2
1