ACI

MIGRATION AND
IMPLEMENTATION
Rene Raeber
Dis+nguished Engineer, Datacenter EMEAR
IEEE-802.1 Architect

Mercator

The ques:on is:

Migrating from Current to New!

Nexus 9500
Spine
10G
40G BiDi
10G
VTEP

1G
Server

Nexus 9300
TOR

VTEP

VTEP

VTEP

1G &
10G
Server

There must be a middle road!

5

Things we would like to understand how to do

Extend ACI to
WAN/DCI

AVS

vSwitch

Let me just run

my network (but
fix my Flooding,
Mobility,
Configuration,
Troubleshooting
challenges)

AVS

vSwitch

Extend ACI to local

hypervisors
Interconnect to existing
DC Networks

Extend ACI to to existing Nexus

installations via a full ACI VXLAN
Switching Enabled Hypervisor and
remote ACI Physical Leaf
6

The Power of Datacenter Networks

The Power of Datacenter Networks

Guiding Principles

Allow for gradual migration of existing classic topologies they will not
go away overnight

Facilitate the 40Gig market transition

Adopt and allow for integration of overlay technologies such as VXLAN

Consider ACI for Green-Field environments or environments looking for

increased operational flexibility

Hypervisors come in different flavors and encapsulation styles

Still need WAN services!

9

A World of many options

1
2

Border Leafs

VXLAN Based Fabric

AVS
4

VTEP

3
5

VXLAN Enabled
Hypervisor

VTEP
VXLAN Enabled
Hypervisor

Classic POD (Mix of N9K and classic platforms in StandAlone)

ACI based network
VXLAN based hypervisor
Remote Leaf (2H2015)
DCI

Service Interconnect to
ASR9K/N7K WAN/DCI
VTEP

10

Add Nexus 9000 to Existing Nexus 2000-7000 Fabric

Deploy standalone Nexus 9000 into existing Nexus Fabric to add network capacity.
Existing

Nexus 2K-7K
Fabric

What you get:

Nexus 9000 Switches

40 Gig capability with QSA for backward capability
Programmability through various APIs (python/puppet/chef)
Nexus 9000

N1Kv

Leverage existing APIs, cloud orchestration/automation tools

Power savings and lower TCO, specifically in N9500 chassis

11

Add an ACI POD

Deploy ACI Fabric in parallel with existing Nexus Fabric. Connect via L2/L3.

ACI
Fabric

Existing

Nexus 2K-7K
Fabric

APIC

L2 or L3 Connection

Nexus 9000

N1Kv

12

Extending ACI Policy to Servers on Existing Fabric

Deploy/upgrade AVS & Remote Leaf N9300 in existing Nexus Fabric.
Extend ACI Policy model over existing Nexus Fabric, allowing apps on existing Nexus Fabric to realize benefits of ACI.

ACI
Fabric

Existing

Nexus 2K-7K
Fabric

APIC

L2 or L3 Connection
PROFILE

*Nexus
9300
Nexus
9000

PROFILE

PROFILE

AVS

*Remote Leaf s/w 2H CY15

ACI POLICY

AVS

13

INTEGRATION / MIGRATION

REMOTE VTEP (PHYSICAL) VIA NEXUS 9300

Classical L2
ACI Infra / L3
Why is this extra box in the middle ?
One could connect the ACI spines to
the pair of N7K Aggregation switches
as full mesh. Still the same results,
but harder to scale when adding more
ACI spines;
Cabling mismatch (40GE on the ACI
side and 10GE on the Nexus side);
Route within the fabric for full remote
VTEP switching (versus route via
outside from the border leaf);

VTEPExternal

AVS

OVS

Why are these links called ACI Infra ?

By the time of vLeaf full switching
\support, the ACI Infra links will be
used to bootstrap the remote VTEP
(physical or virtual);
The APIC VTEP address is then only
reachable through the link at the
spines (and not via Border Leaf)

14

INTEGRATION / MIGRATION

REMOTE VTEP (PHYSICAL) VIA NEXUS 9300

ACI Spines primary forwarding related features are:
- Directory/Proxy Service;
- Multicast Root;
IP Forwarding
- IP Forwarder;

Classical L2
ACI Infra / L3

Directory/Proxy & Multicast root

Services Located in ACI Spine

VTEPExternal

AVS

OVS
15

INTEGRATION / MIGRATION

REMOTE VTEP (PHYSICAL AND VIRTUAL) FULL ACI SWITCHING

What changes ?
- VTEP internal;
- ACI Infra at remote links;

Classical L2
ACI Infra / L3

VTEPInternal

AVS

OVS
16

INTEGRATION / MIGRATION

REMOTE VTEP (PHYSICAL AND VIRTUAL) FULL ACI SWITCHING

Classical L2
ACI Infra / L3

Grow the ACI Fabric as needed

AVS

OVS
17

INTEGRATION / MIGRATION

REMOTE VTEP (PHYSICAL AND VIRTUAL) FULL ACI SWITCHING H1CY15

Classical L2
ACI Infra / L3

and add further services and nodes at ACI

AVS

OVS

AVS

OVS

OpenStack KVM
Juno (basic)
K release (full)

FCS

FCS
18

19

Integration Scenario

Customer Selected ACI for his existing workloads.

Need to interconnect ACI to the existing infrastructure

Need to move (migrate) workloads
Very likely scenario & needs to be easy.

20

The Migration Steps

1.

Extend L2 into ACI

2.

Configure ACI for this L2 extension

3.

Create new EPG and contracts for the workloads to move into

4.

Move Workloads

5.

Move HSRP Default Gateway over to ACI

6.

Turn off the Existing Network

Easy.
21

Step 1: Connect Fabric to Existing Network

Functionally we are expanding the VLANs into ACI.

Existing Design

ACI Fabric

HSRP
Default GW
VLAN 10 / Subnet 10
EPG-10 = VLAN 10

VM

VM

VM

22

Step 2: Connect Fabric to Existing Network

The ACI Infra Admin creates the Leaf interface policy (speed,
CDP, LLDP etc) for the port.

The ACI Tenant Admin uses that port for the migration (see later).
APIC
Existing Design
Lets call this Tenant Red

HSRP
Default GW
VLAN 10 / Subnet 10

VM

VM

VM

23

Now its virtual !

vvvvvvvvvvv

24

Step 3: Configure ACI in preparation for the migration

(EPG equals VLAN)
Tenant Red
Context Red

Always need a Tenant & Context

For the migration:

Bridge Domain 10
Subnet 10

EPG-10

Bridge Domain 20
Subnet 20

EPG-20

create a Bridge-Domain for each

VLAN & define the subnet.
Create EPG and assign it the correct
subnet and VLAN.

Per Bridge-Domain:

We dont want ACI to route this subnet

yet, the existing HSRP gateways
remain the default gateway for now.
Disable Unicast Routing and Enable
flooding
25

Step 3 (continued): Configure ACI Bridge Domain settings

Tenant Red

Temporary Bridge Domain specific

settings while we are using the
HSRP gateways in the existing
network.

Select Forwarding to be Custom

which allow

Context Red
Bridge Domain 10
Subnet 10

EPG-10

Enable Flooding of L2 unknown unicast

Enable ARP flooding

Disable Unicast routing

26

Step 3 (cont.): Create EPG

EPG=VLAN model

Create EPG

Link it to the right vCenter (VMM)

This allows APIC to create DVS

switches on ESXi and ensures
correct signaling between APIC/
vCenter
Connect EPG to the port connected
to existing network. Specify VLAN.

Interface Policy was already set by

ACI Infra Admin in Step 2.

Bridge Domain 10
Subnet 10

EPG-10

VMM Domain - vCenter

Static Binding to port (vlan-10)

Interface Policy

Leaf 2
Port 3
27

Step 3 (cont.): Create EPG (expanding to multiple EPGs)

EPG=VLAN model

Bridge Domain 20

Bridge Domain 10
Subnet 10

EPG-10

Subnet 20

EPG-20

VMM Domain - vCenter

VMM Domain - vCenter

Static Binding to port (vlan-10)

Static Binding to port (vlan-20)

Interface Policy

Leaf 2
Port 3 (trunk)

28

Step 4: Migrate Workloads

APIC point of view, the policy model
EPG 10
P

VM

VM

VM

APIC

VMs will need to be connected to new Port

Group under APIC control (AVS or DVS).
Existing Design

HSRP
Default GW
VLAN 10 / Subnet A

VM

VM

VM

29

Step 5: Complete the Migration

Change BD settings back to normal for ACI mode

Change BD settings back to default.

No Flooding

Unicast Routing enabled.

30

FEX Topology Support Roadmap

Standalone and ACI
Active/Standby
Teaming

Straight Through
(Single Homed)

vPC (Dual Homed)

EvPC

Nexus 9300
Standalone

6.1(2)I2(3)

6.1(2)I2(3)

Target 1HCY15

Future

Nexus 9300
ACI Leaf

Supported at FCS

Brahmaputra 1HCY15

Target 2HCY15

Future
31

Organization Implications
Cisco Infrastructure Team Journey

STORAGE

SECURITY

NETWORK

ARCHITECTURE

DESIGN

IMPLEMENTATION

OPERATIONS

Network
Virtual Teams

COMPUTE

UC/Video

Infrastructure as a Service

32

32

APIC Screen shots

33

Normative

ACI Application Centric Infrastructure

APIC Application Policy Infrastructure Controller
DFA Distributed Fabric Automation
VDP Virtual Station Interface Discovery Protocol
VXLAN - Virtual eXtensible Local Area Network
VXLAN Segment - VXLAN Layer 2 overlay network over which VMs communicate
VXLAN Overlay Network - another term for VXLAN Segment
VXLAN Gateway - an entity which forwards traffic between VXLAN and non-VXLAN environments
VTEP - VXLAN Tunnel End Point - an entity which originates and/or terminates VXLAN tunnels
VLAN - Virtual Local Area Network
VM - Virtual Machine
VNI - VXLAN Network Identifier (or VXLAN Segment ID)
ACL - Access Control List
ECMP - Equal Cost Multipath
IGMP - Internet Group Management Protocol
PIM - Protocol Independent Multicast
SPB - Shortest Path Bridging
ToR - Top of Rack
TRILL - Transparent Interconnection of Lots of Links

34