1 Copyright 2011, Oracle and/or its affiliates.

All rights
reserved.
 ORACLE
PRODUCT
LOGO

Solaris 11 Networking Overview

Sebastien Roy, Senior Principal Engineer
2 Solaris Core OS, Oracle
Copyright 2011, Oracle and/or its affiliates. All rights
reserved.
 Topics

Stack Architecture (Crossbow)

Networking and Zones
Configuration
Additional Feature Highlights

3 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Crossbow: Built-in Network Virtualization and
Resource Control
Network
Virtual NICs, Virtual Switching, Network in a Box
Virtualization

Bandwidth Built-in QOS: bandwidth limits for data links and on a per-
Partitioning flows basis

Resource Constraint traffic processing to CPUs or CPU pools

Control dedicated to zones

Real-time usage and history for VNICs, hardware

Observability
resources, and traffic flow

Parallel traffic from hardware to applications, Dynamic

Scalability
Polling, NUMA I/O

4 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Parallel Network Virtualization Architecture
Virtualization and QoS designed-in
Independent Hardware Lanes with
dedicated resources (CPUs, I/O
threads, interrupts): from the NIC to
applications
VNIC behaves just like a regular
NIC (link speed, stats, MAC
address)
Hardware and software fanouts for
best scalability
Adaptive polling mode depending
on load

5 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Network Resource Control

Set bandwidth limit on a VNIC

(virtual link speed)
QoS integrated in the core
stack, no separate component to
configure
Constrain the CPUs used by
VNICs or data links by CPU ids
or pool names
Integrated with Solaris resource
#dladmcreatevniclnet0\
management and zones pmaxbw=100Mvnic0

6 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Controlling and Observing Flows
Control the Un-Controllable

Built-in QoS can be applied to

traffic flows specified by the
administrator
Managed by flowadm(1M) and
specified by source and
destination IP addresses, protocol,
port number, etc.
Flows can be observed in real time
with flowstat(1M), or a history can
be obtained using extended
accounting

7 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Highly Available VNICs

Link Aggregation provides

transparent failover and
increased throughput to VNICs
and zones
Compliant with IEEE 802.3ad
IP Multipathing (IPMP) can also
be used, but needs to be
configured from within zones

8 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Virtual Switching

A virtual switch is created automatically Use etherstubs instead of physical NICs

when VNICs are configured
Build virtual switches that are independent
Virtual switches allow VNICs to from any hardware
communicate with each other and with
hosts on the network As many as you want on a single host

9 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Private Virtual Network

Use a virtual switch to

build a private network
Use a zone to firewall the
private network, and route
with physical network
Virtual router/firewall has
very small footprint

10 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Virtual Multi-Tiered Architecture

11 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Simulating Network Latencies

12 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Data Center Modeling and
Network Consolidation with Oracle Solaris 11

Built-in Network Functions Network Virtualization

Routing, Firewall, Bridging, Virtual switching,
Integrated Load Balancer, Virtual NICs, QOS
VRRP

Solaris Zones Resource controls

lightweight, small footprint CPU pools, NUMA I/O,
memory capping

13 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Infiniband and Zones
Infiniband (IB) is the backplane for
Engineered Systems SPARC
SuperCluster, Exadata, Exalogic
IP over IB partitions are the IB
equivalent of Ethernet VNICs
IB P_KEY is the equivalent of a Ethernet
VLAN
Can apply same network resource
control to IB partitions
Allows mixing and matching of VNICs
and IB partitions in a zone

14 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 VLAN Separation

VNICs can be assigned a VLAN id

Virtual switch provides VLAN
separation
Local traffic between VNICs
Traffic to and from external hosts
Extend VLAN separation from
physical network into virtual switch

15 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Dynamic VLAN Provisioning
Elastic and Isolated Virtual Networks in the Cloud

Global zone dynamically sends

updates to switch when VLANs
are configured on physical NIC
Switch updates VLANs
associated with each port
Messages are sent only from
global zone
Data link protection can be used
to block attempts from non-global
zone to add unauthorized VLANs
Based on IEEE 802.1d standard

16 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Data Center/Converged Ethernet

I/O Consolidation (LAN, SAN, IPC) on Ethernet

Foundation IEEE protocols (LLDP, DCBX,
Example: Fibre Channel over Ethernet PFC) implemented in Solaris 11

Partition traffic between multiple independent Support for Intel 82599 10 Gigabit Ethernet
lanesonthewire Controller (Niantic)

17 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 NUMA I/O: Out of the Box I/O Scalability
Co-locate I/O kernel threads,
interrupts, and DMA buffers with
devices for best performance
Out-of-the-box, no tuning
required
I/O topology discovery for
SPARC T3/T4 and Intel
platforms, e.g. Sun Fire x4800.
Built-in support for Network
Stack and Infiniband RDSv3.
Extensible.

18 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Cloud-Scale Networking With Solaris 11
Network Virtual NICs (VNICs), Virtual switching, Hardware-assisted virtualization, Automatic
Virtualization VNICs for zones, SR-IOV Integration, VLAN isolation, Anti-spoofing protection

Resource
Integrated QOS, Bandwidth limits, Mapping to CPUs or CPU pools for isolation
Control
Parallel stack, NUMA I/O Framework, SR-IOV Integration, Dynamic Polling, Buffer
Performance Management, Pre-mapped buffers, Kernel Socket API, 4x Lower latency vs KVM,
Converged Ethernet

Built-in Network
Routing, Firewall, Load Balancing, VRRP, Bridging
Funtionality

IPMP re-architecture, Vanity naming, Automatic IP configuration, Centralized IP

Management administration, Centralized data link administration, Consolidated data link
properties, GLDv3 unification for legacy drivers
Real-time data link, hardware, and flow statistics. History integrated with extended
Observability
accounting. Capture local traffic through through virtual switch and IP loopback path.
Committed GLDv3 APIs, pluggable TCP congestion algorithms, IP Filter Hooks,
APIs
Kernel socket API

19 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Topics

Stack Architecture (Crossbow)

Networking and Zones
Configuration
Additional Feature Highlights

20 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Exclusive IP By Default

Networking administrative experience matches that of

physical system
Network virtualization at the link-layer using VNICs
Creating VNICs in the global zone can be problematic
Global zone datalink namespace may not accommodate non-global
zone configuration (e.g. consolidation or migration scenarios)
Network SLA for the zone (i.e. the VNIC properties) need to migrate
with the zone

21 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Automatic Network Resource (anet)

New zonecfg anet resourceisanautomaticVNIC

VNIC is automatically created when the zone boots
VNIC is automatically destroyed when the zone halts
VNIC is created within the non-globalzonesdatalink
namespace (net0, net1,bydefault)
In Solaris 11, each zone has its own separate datalink namespace
Solves difficult p2v and zone migration problems
Included in the system default zone template

22 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Resource Management With anet

anet properties define VNIC attributes for resource

management
Bandwidth limit
MAC address
Allowed IP addresses
VNICinheritszonesconfiguredresourcepoolbydefault
Because anet is a zonecfg resource, it can be migrated
with the zone

23 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Topics

Stack Architecture (Crossbow)

Networking and Zones
Configuration
Additional Feature Highlights

24 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Datalink Administration (dladm)

Configure link-layer objects and their properties

Physical links
VNICs
VLANs
link aggregations
Physical link naming
Physical links named automatically (net0, net1, etc.)
Dissociate interface names and configuration from particularities of
physical hardware

25 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 IP Administration (ipadm)

Configure network-layer objects and their properties

IP interfaces
IP addresses
IPMP groups
generic protocol properties
Replaces use of ifconfig and /etc/hostname.<intf> files
Replaces use of ndd for TCP/IP tuning

26 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Network Configuration Profiles

Administrator chooses configuration policy by selecting

profile
Automatic
System automatically determines which interfaces to configure, and
configures with DHCP
DefaultFixed
All aspects of interface configuration is done manually with command-line
tools (dladm, ipadm, etc.)
Profiles administered with netcfg(1M) and netadm(1M)

27 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Topics

Stack Architecture (Crossbow)

Networking and Zones
Configuration
Additional Feature Highlights

28 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 IPMP Enhancements

IPMP groups configured using ipadm(1M)

IPMP meta-interface holds data addresses
New ipmpstat(1M) command to display IPMP status
Transitive probing
Probing without test addresses
Useful for using IPMP with zones to prevent address space
exhaustion

29 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 IP Host Model Configuration

End-system model (ES model) as defined in RFC 1122

Strong
Accept incoming packet if dst address is assigned to input interface
Output interface selected based on src address
Weak
Source priority
Configured with ipadm hostmodel property

30 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Integrated Load Balancer (ILB)

Standalone in-kernel L3/L4 load balancer

Multiple load-balancing algorithms
Built-in and pluggable server health-check
Configured using the ilbadm(1M) command

31 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 VRRP

Virtual Router Redundancy Protocol

Provides high-availabilityofroutersthroughredundancy
and fail-over
Can be used with ILB to provide redundancy for load balancer
Configured with vrrpadm(1M)

32 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
 Bridging

Create bridged LANs between multiple physical Ethernet

links (including link aggregations)
Protocols supported
STP (Spanning Tree Protocol)
TRILL (Transparent Interconnect of Lots of Links)
Bridge instances configured using dladm(1M)

33 Copyright 2011, Oracle and/or its affiliates. All rights

reserved.
34 Copyright 2011, Oracle and/or its affiliates. All rights
reserved.