VLAN Technology

Technology - VLAN is a method used to logical segment a physical network (IEEE 802.1Q standard) Network separation is achieved by isolation on layer 2 connectivity Delivery of Ethernet packets is restricted to members that belong to the same VLAN A VLAN tag information (VID) is added to network frames either by the host or by the switch Advantages Provides more flexible network deployment over traditional network technology Simplifies network administration and configuration Can improve network performance by separating a network into different segments

VLAN Tagging
Layer 2

Application

data
data

A chunk of data

Layer 2

TCPIP (AIX)

An IP packet (max.1500 byte)

Layer 2

Ethernet Adapter

data

An untagged Ethernet frame (max. 1516 bytes) An a tagged Ethernet frame (max. 1532 bytes)

Network switch or AIX VLAN device driver or power hypervisor

data

AIX VLAN Configuration (VLAN Unaware Hosts)

VLAN definition at switch level (standard configuration) - VLANs are defined at switch level by the network administrator (frequently used) - VLAN tag is interpreted by the switch - PVID (Port VLAN ID = default VLAN ID) is added to packets entering switch and removed before packets are delivered to host VLAN unaware host could be confused by receiving a tagged Ethernet frame (drop and indicate a frame error)
Frame A

AIX/Linux VLAN unaware host

Frame B

Network Interface Layer3

en 0

VLAN n

Frame A

Frame B

Network switch with PVID Definition

AIX VLAN Configuration (VLAN Aware Hosts)

AIX can define VLANs based on any physical Ethernet adapters - VLANs are defined at AIX level by the system administrator - VID tag is added to Ethernet frames by AIX logical device (VALN device driver) VLAN tag is interpreted by the logical device - Switch must be configured with tagged port

AIX VLAN aware host en 1 ent 1

VLAN n and p

Frame A

Frame B

en 0

Tagged or untagged frames

Frame A

Frame B

Network switch with PVID and additional VID Definition

Dynamic VLAN Registration Protocol

Generic Attribute Registration Protocol (GARP) - Generic link layer protocol that allows different applications to propagate information between switches - packets sent for GARP protocols are called Bridge Protocol Data Units(BPDU) GARP VLAN Registration Protocol(GVRP) - GVRP is a GARP application that allows for the dynamic registration of VLANs over networks - Reduce the work required by network administrators The network automatically learns about the VLAN topology - Makes setup easier - Only the host needs to be configured with VLAN , rather than all switch ports - GVRP support with virtual i/o server version 1.4 (fix pack 9.1) GVRP is implemented with in the existing shared Ethernet adapter(SEA) driver code - GVRP announce to the physical network the VLANs that have been statically configure on the SEAs virtual adapter - To use GVRP when creating an SEA
$ mkdev sea ent0 vadapter ent1 default ent1 defaultid 1 attr gvrp=yes

Physical Network to PowerVM Virtulation

POWER Hypervisor Ethernet switch: Network virtualization Interpretation communications External network communications

POWER system using Physical adapter

POWER 5 or POWER 6 SYSTEM

AIX LPAR
enx

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR enx

AIX LPAR

AIX LPAR

AIX LPAR

POWER Hypervisor
Physical Device PCI Eth or LHEA port

POWER Hypervisor

VLAN
VIOS LPAR

Enterprise Network

Enterprise Network

External Network Access with or without VIOS

Routing with Gateway LPAR

Bridging with SEA

AIX LPAR
enx

AIX LPAR

AIX LPAR

AIX LPAR
TCPIP ROUT ING

AIX LPAR
enx

AIX LPAR

AIX LPAR

AIX LPAR

POWER Hypervisor
Enterprise Network

POWER Hypervisor
Enterprise Network

VIOS LPAR

VLANs

Physical Devices

Multiple Virtual Switch (POWER6 only)

Multiple Virtual machines can be defined in IBM System p6 --Vswitches are defined at system level(Hypervisor) One VSwitch (VSwitch0) is defined by default Create a switch from HMC --select server->configuration->virtual network management VLANS are not able to communicate through different Vswitches No VLAN predefinition Create a Virtual Ethernet and select a Vswitch --Select the Vswitch and define the PVID and additional V Ds

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

entx

entx

entx

entx

entx

POWER Hypervisor V switch A

VLAN X

V switch B
VLAN X

V switch C
VLAN X
VIOS LPAR

Power6 Server

Physical Device

Enterprise Network

Bridging to External Network using SEA Feature

Bridging
- VIOS partition uses SEA for external network access - One virtual Ethernet adapter for one or many VLANs on the VIOS
AIX LPAR
en0 (if)

AIX LPAR
en0 (if)

AIX LPAR
en0 (if)

AIX LPAR
en0 (if)

POWER6 Server

POWER6 Server

ent0 (virt) PHPY ent1 (virt)

ent0 (virt)

PVID 118

ent0 (virt)

PVID 218

ent0 (virt)

PVID 318

PHPY

VIOS

VIOS

ent1 (virt)

ent2 SEA
ent0 (phy)

ent2 SEA
ent0 (phy)

PVID=99 Added VLANs VIDs=118,216,318

To other switch

To other switch

Physical Ethernet Adapter for the SEA backing device

SEA backing device supports: - PCI Ethernet adapter. - Logical host Ethernet Adapter (LHEA)port with promiscuous mode. The only logical port configurable is number 1 on the physical port Corresponding IVE physical port is dedicated for SEA use All the IVE switch bandwidth is assigned to the SEA

AIX LPAR

en0 (if)
ent0 (virt)

POWER6 Server

PHYP
VIOS
ent1 (virt)

ent2 SEA
PCI adapter Or LHEA port with Promiscuous mode

Switch
Two IVE physical ports set to promiscuous mode are required to improve the network connectivity of the SEA.

Accessing External Network: IVE (Power versus SEA

Using VIO Shared Ethernet Adapter Using Integrated Virtual Ethernet

VIOS
SEA

AIX
Virtual Ethernet Driver

AIX
Virtual Ethernet Driver

LINUX
Virtual Ethernet Driver

AIX
LHEA Driver

AIX
LHEA Driver

LINUX
LHEA Driver

Virtual Ethernet switch

Hypervisor

Network Adapter

Integrated Virtual Ethernet

LAN IVE communicates directly to logical partitions (LPARs). -- It is physical device, it must before partition mobility. -- Each IVE physical port is separated network Switch. -- It reduces the interaction with POWER Hypervisor. -- 10Gbps IVE feature has better performance than virtual Ethernet. -- IVE supports a maximum of 16 or 32 LPARs. IVE reduces the need for physical Ethernet adapters.

Network Availability: Methodology

Define which level of availability is needed --Accepted network downtime (from millisecond to hour) --Network maintenance timeframe Find corresponding SPOFs that can be eliminated --Hardware and software single point failure(SPOF) Identify which type of availability feature could be used --Hardware (physical adaptor or equipment) --Hypervisor(virtual adapters) --AIX, Linux, or VIOS Implement availability options /features --step-by-step process implementation to minimize down time Test failover and recovery conditions --Check mechanism used to failover and recover Verify disk configuration and partition migration dependencies. --Example : Disk MPIO requires Dual VIOS configuration.

Network Feature Used for Availability

Hardware - Redundant Ethernet adapters - Dual switch configuration AIX - NIB (Ether Channel backup - Link aggregation (Ether Channel) - TCPIP multipath routing with Dead Gateway detection IBM POWER Systems - Virtual Ethernet adapter and virtual hypervisor switch - Virtual I/O server or gateway partition - Virtual I/O server used as a backup network access - Dual shared Ethernet adapter with NIB at the client - SEA failover feature

Network availability = Mix of several features

Availability Solutions: External Access

SPOFs/Resilience --Physical Ethernet Adapter --VIOs partition --External switch port / switch Solutions for availability --VIOS physical link aggregation --LHEA ports link aggregation --Single VIOS as Network Backup --Dual VIOS Multipath routing with DGD on client NIB at client SEA Failover
POWER Server
AIX Client
Virt Ent

AIX Client
Virt Ent

VIOS 1
Shared Ethernet adapter Physical Ent

Switch

Solutions: VIOS physical Link Aggregation

Link Aggregation at VIOS - one primary adapter with one backup adapter Two different switches without extra configuration - Several primary adaptors with one backup All primary adaptors must be connected to the same physical switch Extra configuration on switch is needed - Easy to setup and manage - No special configuration on client partition - single or multiple LANs at client partition Resilience - VIOS partition is a point of Failure Performance - Network bandwidth is increased

Phys Ent

Phys Ent

switch

switch

Solutions: AIX Link Aggregation with LHEA ports

Link aggregation at AIX VIOS - one primary adapter with one backup adapter Two different switches without extra configuration - Several primary adapters with one backup All IVE physical ports corresponding to the primary adapters must be connected - Set up the external address to ping LA parameter - No need for VIOS - Multiple LANs requires VLAN tags at client partition. Resilience - IVE adapter is a point of failure. Use multiple IVE adapters(only p57/02CECs) Performance - Low latency and high network bandwidth

POWER Server

AIX Client
LINK Agent Primary
backup

ent

ent

ent

LHEA

LHEA

LHEA

LHEA port

LHEA port

LHEA port

IVE Adapter
Phys Port Phys Port Phys Port

Switch

Switch

Single VIOS configuration as network Backup

Use a VIOS to provide network Backup path
Complexity -- Requires configuration on client(NIB) -- Need to ping outside the client initiate NIB failover resilience -- protects against single switch port / switch /Ethernet adapter failure Throughput / scalability -- High bandwidth applications may benefit from the physical adapter -- Backup performance limited to a single Ethernet adapter and VIOS CPU Notes -- Useful for multiple LPARs configuration -- NIB does not support tagged VLANs on physical LAN
POWER Server AIX Client
NIB
Phys Ent Virt Ent

AIX Client
NIB
Phys Ent Virt Ent

VIOS 1
Shared Ethernet adapter Phys Ent

Switch

Switch

Dual VIOS Configurations: Three solutions

Routing: Two shared Ethernet adapters on different VLANS with multi path routing on client partition

POWER Server AIX Client

Virt Ent

AIX Client
Virt Ent

NIB: Two shared Ethernet adapters in different VLANs with NIB on the client partition Shared Ethernet adapter failover feature. Benefits: --Bridging solutions are easy to setup --No specialized configuration on switch component --Needs two physical Ethernet adapters (one per virtual I/O server

VIOS 1
Shared Ethernet adapter Phys Ent

VIOS 2
Shared Ethernet adapter Phys Ent

Switch

Switch

Needs additional CPU, RAM, and disk for second VIOS partition

Dual VIOS Configuration with NIB at Client

Considerations - No VLAN tagging (only untagged packets) - Need two virtual Ethernet adapters at client partition - Special client partition configuration (one NIB for each AIX network) - Address to ping must be reliable Benefits: - Allows manual load balancing configuration of client through both VIO servers - No specific configuration needed on external physical switch
POWER Server AIX Client NIB
Virt Ent Virt Ent

AIX Client NIB

Virt Ent Virt Ent

VIOS 1
Shared Ethernet adapter Phys Ent

VIOS 1
Shared Ethernet adapter Phys Ent

Switch

Switch

Dual VIOS with NIB at client: Single LAN (Details)

en2 (if) ent2 NIB ent0 virt ent1 virt backup en2 (if) ent2 NIB ent0 virt ent1 virt backup

Client LPAR 1

Client LPAR 2

Dual VIOS configuration with SEA failover Feature : A Best Practice

Considerations - Is supported at VIOS 1.2 and above - Has no load balancing for single configuration - Supports VLAN tagging Benefits - Easy client partition configuration - Network configuration simplified compared to the NIB Remarks - The control channel is critical for VIO server synchronization. only one SEA must be active at a time - Two parameters for the support: Priority Control channel

Heart beat

SEA failover Feature: Testing

Manual failover: - set ha_mode to standby on primary: the SEA is expected to fail over >chdev dev ent2 attr ha_mode=standby - Reset ha_mode to auto on primary :the SEA is expected to failover VIOS shutdown: - Reboot VIOS on primary: the SEA is expected to failover - when primary VIOS comes up again: the SEA is expected to fail back VIOS error: - Deactivate primary VIOS on HMC: the SEA is expected to failover Active and boot VIOS: the SEA is expected to fail back Physical link failure: - Unplug the link of the physical adapter on the primary: the SEA is expected to fail over Reconnect the link of physical adapter on primary :the SEA is expected to failback Reserve the boot sequence: - shut down both VIO servers Boot the standby VIOS : the SEA is expected to become active on standby. Boot the primary VIOS: the SEA is expected to fail back

Migrating from NIB configuration to SEA failover

Converting from older NIB method

Use DLPAR to add required virtual adapters - control channel, trunk - Minimize downtime NIB configuration removed at later time - Clients IP address is associated with the NIB - Downtime when moving IP address to another interface (short planned downtime on the Client)

Multiple LANs in the client partition: Availability solution Multiple LANs solution is driven by VLAN tagging rules: If only untagged frames are propagated to external network
infrastructure (VLAN unaware hosts) - No VLAN ID administration at AIX and supervisor level - Need to segment traffic in Hypervisor (thru PVID configuration) - Multiple gateway s or SEAs and multiple LANs in virtual switch - Easy network administration, complex virtual architecture. If tagged frames are supported by external switch - AIX and Hypervisor VIDs must be defined according enterprise network architecture - No additional gateway or SEA for multiple VLANs configuration

Thank you