Bird's Eye View on

Neutron & SDN

Xin Wu @ Big Switch Networks

Logical View: Same as Wireless Router at Home

1.1.1.2

192.168.1.1/24
host1

host2

tenant
A
router

1.1.1.3

external
network

tenant
B
router

192.168.2.1/24

192.168.1.1/24

192.168.2.1/24

host3

host4

host6

host5

Physical View
spine

leaf1

leaf2

host2

vswitch1

host1

host3

host4

host6

server1

host5

Tenant Router Is Missing

Key Question: Where to Implement Tenant Router?

Option 1: one software router on a server
Option 2: distributed router on switches

Option 1: One Software Router on a Server

openstack
controller

spine

leaf1

SDN controller

host2

vswitch1

host1

host3

host4

leaf3

leaf2

host6

server1

host5

vswitch2

tenant A router

tenant B router

server2

openstack
controller

1.

cr

ea

te

VM

&

at

ta
c

VM

to

vs
w

itc

2.

cre
po ate/u
rt,
vla pdate
n,
ma serv
c, I erid
P,
ne ,
two

rk

Extremely Simplified Control Plane Flow

SDN
controller

vswitch

3. program flow entries

agents

Option 1: One Software Router on a Server

Pros: server only

Option 1: One Software Router on a Server

Cons 1: cannot support non-vm workloads
Solution: offload tunnel to physical switch
openstack
controller

spine

leaf1

SDN controller

host2

vswitch1

host1

host3

host4

leaf3

leaf2

host6

server1

host5

vswitch2

tenant A router

tenant B router

server2

Option 1: One Software Router on a Server

Cons 2: suboptimal routing
Solution: distributed virtual routing (DVR)
openstack
controller

spine

SDN controller

leaf1

leaf2

host2

vswitch1

host1

host3

host4

host6

server1

host5

Key Question: Where to Implement Tenant Router?

Option 1: one software router on a server

Option 1: One Software Router on a Server

Pros: server only (no longer the case for non-vm workloads)
Cons 1: cannot support non-vm workloads offload tunnel to physical switch
Cons 2: suboptimal routing distributed virtual routing

Option 2: Distributed Router on Switches

Pros 1: Support both vm and non-vm workloads
Pros 2: Always optimal forwarding/routing
openstack
controller

spine

SDN controller

leaf1

leaf2

host2

vswitch1

host1

host3

host4

host6

server1

host5

Applications that Drives Neutron and SDN Evolvement

1.

NFV DPDK, SR-IOV

2.

Docker 4-tier networking

NFV Intel x86 Data Plane Development Kit (DPDK)

NFV: networking function running in VM
OVS/linux bridge: expensive interrupt and data copy between kernel and NIC
NUMA: non-uniform memory access

NIC

NIC

memory

memory

core

core

core

core

CPU1

CPU2

NUMA1

NUMA2

firewall VM 1

firewall VM 2

NIC
kernel

vswitch
kernel

firewall VM 1

NIC

firewall VM 2

user space vswitch

firewall VM 1

firewall VM 2

openstack
controller

1.

cr

ea

te

NF

VM

&

at
ta
c

SDN
controller

it t
o

NU

A1

vswitch

3. program flow entries

2.

cre
po ate/u
rt,
vla pdate
n,
ma serv
c, I erid
P,
ne ,
two

rk

DPDKs Impact on Control Plane

agents

NFV Single Root I/O Virtualization (SR-IOV)

SR-IOV NIC
DMA between NIC and VM
No CPU is involved
NIC

NIC (physical)

vswitch
kernel
(virtual)
firewall VM 1
firewall VM 1

firewall VM 2

firewall VM 2

SR-IOVs Impact on Control Plane

openstack
controller

1. create NFV VM &

assign virtual NIC to it
agents

2. create/update server-id,
port, vlan, mac, IP, network

SDN
controller

3. program flow entries

physical
switch

vswitch

Applications that Drives Neutron and SDN Evolvement

1.

NFV DPDK, SR-IOV

2.

Docker 4-tier networking

Docker on Physical Server: Solved Problem

spine

leaf1

leaf2

host2

vswitch1

docker1

docker3

docker4

docker6

server1

host5

Docker on VM
Solution 1: run Kubernetes on top of VMs separate IP space
Solution 2: SDN controller manages vswitches in VMs
openstack
controller

spine

leaf1

SDN controller

leaf2

host2

vswitch1

vswitch2
docker1

vm3

docker2 vm1

server1

host5

Neutron & SDN

Where to implement tenant router?
option 1: one software router on a server
option 2: distributed router on switches

Killer application
NFV DPDK, SR-IOV
Docker 4-tier networking