NETWORK VIRTUALIZATION

SEMINAR REPORT 2009-2011

In partial fulfillment of Requirements in Degree of Master of Technology In SOFTWARE ENGINEERING

SUBMITTED BY

NICY K.S

DEPARTMENT OF COMPUTER SCIENCE COCHIN UNIVERSITY OF SCIENCE AND TECHNOLOGY KOCHI 682 022

COCHIN UNIVERSITY OF SCIENCE AND TECHNOLOGY KOCHI 682 022 DEPARTMENT OF COMPUTER SCIENCE

CERTIFICATE
This is to certify that the seminar report entitled NETWORK VIRTUALIZATION is being submitted by NICY K.S in partial fulfillment of the requirements for the award of M.Tech in Software Engineering is a bonafide record of the seminar presented by her during the academic year 2009.

Dr.Sumam Mary Idicula Reader Dept. of Computer Science

Prof. Dr.K.Poulose Jacob Director Dept. of Computer Science

ACKNOWLEDGEMENT

First of all let me thank our Director Prof: Dr. K. Paulose Jacob, Dept. of Computer Science, who provided with the necessary facilities and advice. I am also thankful to Dr. Sumam Mary Idicula, Reader, Dept. of Computer Science, for her valuable suggestions and support for the completion of this seminar. With great pleasure I remember Mr. G. Santhoskumar for his sincere guidance. Also I am thankful to all of my teaching and non-teaching staff in the department and my friends for extending their warm kindness and help. I would like to thank my parents without their blessings and support I would not have been able to accomplish my goal. Finally, I thank the almighty for giving the guidance and blessings.

CONTENTS
1. INTRODUCTION 2. OVERVIEW AND STATE OF THE ART 2.1 From VPNs to VNs 2.2 Basic components 2.3 Business roles 3. DESIGNING VN-ENABLED NETWORKS 3.1 Infrastructure Provider requirements 3.2 VNID, VLID, virtual link aggregation and scalability 3.3 QoS and isolation 4. BUILDING VIRTUAL NETWORKS 4.1 The VNP InP interface 4.2 VN admission control and mapping of virtual resources 4.3 Network reoptimization 5. I/O VIRTUALIZATION 5.1 Private I/O Devices 5.2 Shared I/O Devices 6. NETWORK VIRTUALIZATION IN SOFTWARE 6.1 Xen Network Virtualization Architecture 6.2 Multiqueue Networking Interfaces 6.3 CDNA Network Virtualization Architecture 7. CONCLUSION 8. REFERENCES 1 2 2 3 6 7 7 8 10 12 12 13 15 17 17 19 20 20 22 24 28 29

Abstract
The interest in network virtualization has been growing steadily among the networking community in the last few years. Network virtualization opens up new possibilities for the evolution path to the Future Internet by enabling the deployment of different architectures and protocols over a shared physical infrastructure.The deployment of network virtualization imposes new requirements and raises new issues in relation to how networks are provisioned, managed and controlled today. In the last few years, the concept of network virtualization has attracted a great deal of attention from industry and research fora. A network virtualization renaissance has been originated mainly from the realization that it can provide a platform upon which novel network architectures can be built, experimented and evaluated, freed from legacy technology constraints. Furthermore, network virtualization has been heralded as the keystone of a new global Internet architecture, a new component enabling the coexistence of multiple networking approaches and deployment of new network technologies and advanced applications. In addition, virtualization is expected to provide a clean separation of services and infrastructure and facilitate new ways of doing business by allowing the trading of network resources among multiple providers and customers. Several proposals for network virtualization architecture or for the utilization of network virtualization in several contexts have been put forward.

Abstract
The interest in network virtualization has been growing steadily among the networking community in the last few years. Network virtualization opens up new possibilities for the evolution path to the Future Internet by enabling the deployment of different architectures and protocols over a shared physical infrastructure.The deployment of network virtualization imposes new requirements and raises new issues in relation to how networks are provisioned, managed and controlled today. In the last few years, the concept of network virtualization has attracted a great deal of attention from industry and research fora. A network virtualization renaissance has been originated mainly from the realization that it can provide a platform upon which novel network architectures can be built, experimented and evaluated, freed from legacy technology constraints. Furthermore, network virtualization has been heralded as the keystone of a new global Internet architecture, a new component enabling the coexistence of multiple networking approaches and deployment of new network technologies and advanced applications. In addition, virtualization is expected to provide a clean separation of services and infrastructure and facilitate new ways of doing business by allowing the trading of network resources among multiple providers and customers. Several proposals for network virtualization architecture or for the utilization of network virtualization in several contexts have been put forward.

Presented by Nicy K.S MTech SE Dept.of Computer Science CUSAT

Network Virtualization

1.INTRODUCTION
In the last few years, the concept of network virtualization has attracted a great deal of attention from industry and research fora. Although it is not a strictly new concept, a network virtualization renaissance has been originated mainly from the realization that it can provide a platform upon which novel network architectures can be built, experimented and evaluated, freed from legacy technology constraints. Furthermore, network virtualization has been heralded as the keystone of a new global Internet architecture, a new component enabling the coexistence of multiple networking approaches, thus overcoming the present Internet ossification problem and stimulating the development and deployment of new network technologies and advanced applications. In addition, virtualization is expected to provide a clean separation of services and infrastructure and facilitate new ways of doing business by allowing the trading of network resources among multiple providers and customers. Up to now the concept of network virtualization has been explored essentially in testbeds, on a limited scale. Several proposals for network virtualization architecture or for the utilization of network virtualization in several contexts have been put forward. However, so far, the impact of network virtualization on the underlying physical infrastructure has been relatively overlooked. In particular, the feasibility of the network virtualization concept in large scale operator networks has not been adequately assessed. There are multiple challenges associated with the deployment of network virtualization in operator infrastructures. A few examples are how to enforce isolation between different virtual networks, how to conciliate a wide range of requirements of different virtual networks, how to fulfill carrier-class level reliability, how to guarantee scalability.

Dept. Of Computer Science,CUSAT.

Network Virtualization

2. OVERVIEW AND STATE OF THE ART

2.1 From VPNs to VNs In general, virtualization provides an abstraction between user and physical resources, so that the user gets the illusion of direct interaction with those physical resources. In the last few years, significant advances in operating system virtualization technologies have been achieved, with contributions from major players which has enabled the use of virtualization in a growing number of contexts and applications. Major router vendors are also following this trend and it seems likely that network virtualization will become a reality in operator networks in a not too distant future. The concept of network virtualization is not new it was materialized in the past with network based VPNs, which have been a highly successful approach to provide separate virtual networks over a common physical infrastructure. One may wonder why full-blown network virtualization is in fact required if the VPN model has been so successful to deploy the concept of virtual network (VN). VPNs fulfill the basic goal of providing different logical networks over a shared infrastructure, but suffer from a few limitations, among others: All virtual networks are based on the same technology and protocol stack, which precludes the coexistence of different networking solutions; A real isolation of virtual network resources is not possible, by default; A clean separation of the roles of infrastructure provider and VPN service provider is not possible and in practice they are played by the same entity. Network virtualization goes a step further by enabling independent programmability of virtual networks. This means that a virtual network is no longer necessarily based on IP, or any other specific technology, and any kind of network architecture can in principle be built over a virtual network. Another important strength of virtualization is the native capability to handle multi-provider scenarios and hide the specificities of the network infrastructure, including the existence of multiple administrative domains. Although some workaround solutions exist, this has traditionally been a complicated issue for VPNs.

Dept. Of Computer Science,CUSAT.

Network Virtualization Finally, network virtualization provides a real isolation of virtual networks sharing the same infrastructure, for example by means of different operating system instances, and not just an illusory isolation, as provided by VPNs. 2.2 Basic components Network virtualization is composed of two main components link virtualization and node virtualization. Link virtualization allows the transport of multiple separate virtual links over a shared physical link. A virtual link is often identified explicitly by a tag, but can also be identified implicitly, by a time slot or a wavelength, for example. A wide variety of the standard link virtualization techniques available in todays Internet (e.g. ATM, Ethernet 802.1q, MPLS) may in principle be used for this purpose. The basic elements of network virtualization are shown in Figure 2.2.1. At the substrate level, the substrate node is network equipment capable of supporting virtual nodes by means of any virtualization technology. A single substrate node typically contains a number of virtual nodes.

Fig.2.2.1 Network Virtualization Model

Dept. Of Computer Science,CUSAT.

Network Virtualization The virtual node is another central element in virtual network architecture. The concept of node virtualization has been materialized, to some extent, by virtual routers. Although the practical realization of the concept has had a few variations, in broad terms, a virtual router appears for all purposes (e.g. configuration, management, monitoring, troubleshooting) like a dedicated physical router. However, in most cases, a virtual router provides an illusion of isolation, rather than a real isolation, as it lacks dedicated memory, processing and I/O resources. Node virtualization is based on isolation and partitioning of hardware resources. Physical resources of a substrate node (e.g.CPU, memory, storage capacity, link bandwidth) are partitioned into slices and each slice is allocated to a virtual node according to a set of requirements. Recent developments in operating system virtualization have permitted significant advances in terms of fairness and performance. Virtualization of substrate nodes, in combination with virtualization of links interconnecting those substrate nodes, enables the creation of virtual networks, functionally equivalent to a physical network. A substrate node terminates one or multiple physical links, which may in turn carry multiple virtual links. Correspondingly, a virtual node terminates one or multiple virtual links. For scalability reasons, virtual links may be handled per aggregate, rather than per virtual link, as will be discussed later on. Some substrate nodes, in addition to terminate virtual links, are also able to forward traffic transported in virtual links in a transparent way (this is the case of node b in Figure 2.2.2)

Fig 2.2.2 Basic Network Virtualization Elements

Dept. Of Computer Science,CUSAT.

Network Virtualization

From a functional point of view, substrate nodes can be classified in four major groups, according to the type of functionality that they provide: Edge nodes: located at the edge of the substrate network, these nodes are mainly points of presence (PoP) of the network infrastructure provider. Typically, these edge nodes are connected to end users, either directly or through an access network provider, which may be itself virtualized, or n t; geographical location is usually a key attribute of this kind of nodes. Core VN-capable nodes: support virtualization, i.e. can host virtual nodes and can also support non-VN traffic (again, node b in Figure 2.2.2). A core VN-capable node may be connected to edge nodes, or to other core nodes, either VN-capable or not. Core transport nodes: do not support node virtualization, are typically core nodes used to transport traffic. Border nodes: are connected to neighbor border nodes located in different network domains.

Fig2.2.3 Types of Substrate Node

Dept. Of Computer Science,CUSAT.

Network Virtualization

It should be noted that these functions are not mutually exclusive for example, node b in Figure 2.2.2 supports both core VN-capable node and core transport node functions. Figure 2.2.3 shows a scenario with two network domains, A and B, and the four types of network nodes identified above.. 2.3 Business roles In general, network virtualization model includes three different business roles: The Infrastructure Provider (InP) deploys and runs the network physical resources, and partitions them into isolated virtual slices, using some virtualization technology. Typically, these resources are offered to another service provider, and not to end users (but the InP customer might as well be a corporation using the virtual network for its internal use, rather than to build commercial end user services). The InP has a vision of the resources allocated to each VN, but not of the protocols running inside. The virtual network provider (VNP) is responsible for finding and composing the adequate set of virtua resources from one or more infrastructure providers, in order to fulfill the virtual network operator request. The VNP leases slices of the virtualized infrastructure from one or more InPs and puts them together. Strictly speaking, what the VNP provides is not a network yet, but just an empty container where the virtual network operator builds the protocols that make up the VN. The virtual network operator (VNO) deploys any protocol stack and network architecture over a VN, independently of the underlying physical network technologies. The VNO operates, maintains, controls and manages the virtual network. Ideally, the fact that resources are virtual, rather than physical, should not imply any major impact from an operational point of view. Thus, the role of the VNO should be indistinguishable from that of any operator running a native network infrastructure. VNOs have a unified view of the network, regardless of the multiple infrastructure domains on which it is built.

Dept. Of Computer Science,CUSAT.

Network Virtualization

3. DESIGNING VN-ENABLED NETWORKS

3.1 Infrastructure Provider requirements In a network virtualization scenario, an InP must fulfill generic network infrastructure provider requirements, as well as new requirements specifically related with virtualization of network resources, as detailed in the following paragraphs. Robustness: The network should continue to operate in the event of node or link failure. A virtual network should provide carrier-class reliability (typically in the order of 99,99% to 99,999%). Manageability: The InP must have a view of the underlying physical topology, operational state, provisioning status, and other parameters associated with the equipment providing the virtual network. Traffic and network resource control: Traffic engineering and management techniques performed by the InP must not constrain the basic operation of a VN in any way. Isolation: Mechanisms to guarantee deterministic degrees of isolation between virtual networks must be provided. It must be guaranteed that misbehaving VNs (either accidently or purposefully) do not affect the performance of other VNs sharing the same resources (e.g. memory space, storage, bandwidth, processing power), as well as non-VN traffic. Scalability: The number of VNs on a specific network administrative domain may grow to the order of thousands, or more. Any technical solution must be scalable to cope with any number of virtual networks. In particular, the isolation of different virtual networks should be guaranteed in a scalable way. Inter-domain interoperability: In the case of VNs spanning multiple infrastructure provider domains, seamless interoperability must be guaranteed. On-demand provisioning: It must be possible to create,modify and remove VNs dynamically at request, with a minimal impact on the underlying infrastructure. The characteristics of a VN may change dynamically at runtime and in many cases a VN will have a limited lifespan.
Dept. Of Computer Science,CUSAT.

Network Virtualization Network technology agnosticism: Network virtualization should not rely or depend on any specific network technology. In particular, multi-domain scenarios across dissimilar infrastructure domains should be possible. 3.2 Virtual Network ID, Virtual Link ID,virtual link aggregation and scalability A VN may span multiple network infrastructure domains. Inorder to provision and control VNs spanning multiple infrastructure domains, there must be a way to uniquely identify a specific VN on a global scale through a Virtual Network ID (VNID). This can be achieved by concatenating two pieces of information the identification of the organization responsible for the VN (in principle, the VNO), which is supposed to be globally unique, and the ID allocated by the VNO to the VN, which must be unique within that specific administrative domain. Additionally, in the data plane, identification of virtual links by means of a virtual link ID (VLID) is also required. The most straightforward solution would consist of simply mapping the VLID to whatever data link layer specific tag is in use. A number of available link virtualization techniques, such as ATM, Ethernet VLAN or MPLS, could perfectly be used for this purpose. For example, in the case of MPLS, a different LSP would be allocated to each virtual link. This means that the VLID would be used both as a virtual link identifier and a tag for data forwarding. This apparent simplicity does not come without a cost. On the one hand, for every new virtual link, a new VLID would have to be learned by all network nodes in the data path. Since the number of virtual links has a square law relationship to the number of virtual nodes in the network, this would represent a clear scalability concern. It is clear that large scale scenarios call for a more scalable approach. As a matter of fact, scalability is a major requirement for virtual networks. One of the reasons why VPNs, namely those based on the BGP/MPLS model, have been so successful lies in its good scalability properties, which basically result from the clear separation of edge and core functions. Only edge nodes deal with VPNs and support virtualization (or an illusion of it, to be more precise), whereas core nodes are only concerned with forwarding traffic trunks and have no awareness of VPNs whatsoever3. Thus, the growth of the number of VPNs does not have any impact on the network core.

Dept. Of Computer Science,CUSAT.

Network Virtualization For this reason, virtual link aggregation will certainly constitute an important network virtualization requirement in operator networks. A virtual link aggregate can be defined as a set of virtual links that follow a common path and are treated similarly between a pair of virtual nodes. In practice, virtual link aggregation can be performed by carrying at least two types of identifiers in the data plane one for endpoint VN identification and another for hop-by-hop forwarding, identifying the virtual link aggregate (nodes in the core need to be aware of the latter only). Virtual link aggregation simplifies the job of core substrate nodes (as they only have to deal with a limited number of virtual link aggregates, rather than a potentially high number of individual virtual links), enhances scalability (as the growth of number of VNs does not necessarily imply additional load in the network core nodes) and improves performance. However, there are a few issues about virtual link aggregation in a network virtualization environment to be noted, so the straight replication of the VPN scenario may not be entirely possible. Figure 3.2.1 illustrates link aggregation and its relationship with node virtualization in four different scenarios. Cases 1 and 2 correspond to the legacy VPN scenario. In both cases, only the edge nodes (a and e) support node virtualization in the data path.

Fig3.2.1 Vitual Link Aggregation Scenarios Virtual link aggregation, used in case 2, allows nodes b, c and d to handle a single virtual link aggregate and ignore individual virtual links, thus executing a much simpler job, compared to case 1 By contrast, in cases 3 and 4, all substrate nodes in the data path

Dept. Of Computer Science,CUSAT.

Network Virtualization support node virtualization. Contrary to the previous situation, the value of virtual link aggregation is in this case much smaller, if not null. In fact, to take advantage of the economies provided by link aggregation, the number of substrate nodes providing virtualization must be minimized. This is illustrated in cases 3 and 4 link aggregation in case 4 does not provide any advantage compared to case 3. Link aggregation is only effective if a significant number of substrate nodes along the data path perform transport functions only, and not virtualization. This suggests a trade-off between data plane efficiency offered by virtual link aggregation and flexibility offered by node virtualization in the network core. 3.3 QoS and isolation One of the limitations of legacy VPN approaches is that, because the matrix of incoming and outgoing traffic between every possible source-destination pair is impossible to predict, a guaranteed allocation of resources per VPN is usually not possible. On the contrary, VNs allow a fine grained allocation of resources, including bandwidth per virtual link on every segment of the substrate network. However, in practice, a fine grained per- VN resource control is not compatible with scalable aggregation based approaches, described before. Hence, there is a trade-off between scalability (which favors aggregation) and strict VN isolation (which requires control of resources per virtual link). Virtualization raises new QoS issues. More than QoS differentiation, virtualization requires QoS isolation. In all respects, including deterministic performance, a VN should replicate a network supported by physical resources. In a virtualizationbased environment, QoS can be handled at two levels: Inside a VN, any QoS mechanism and tool can in principle be defined and deployed by the VNO (for example, different classes of service for voice and data). Intra-VN QoS mechanisms must not be constrained by the characteristics of the substrate (including traffic engineering performed by the InP), or by the traffic running in other VNs sharing the same resources. At the substrate level, QoS isolation between virtual networks is a key requirement, as any QoS policy and mechanism inside the VN cannot be built

Dept. Of Computer Science,CUSAT.

10

Network Virtualization without adequate guarantees of performance. However, strict QoS isolation of virtual networks requires substrate nodes to apply a per-VN treatment to the traffic. As said before, this may not be possible, or desirable, in all cases, mainly for scalability reasons. In the cases where virtual link aggregation is used, all virtual links inside a given virtual link aggregate get a similar QoS treatment by the substrate nodes. Isolation between virtual links in a common aggregate must be then guaranteed by policing the traffic admitted into each virtual link at the source end point. The substrate nodes in the middle have to make sure that bandwidth is shared between aggregates in a fair way.

Dept. Of Computer Science,CUSAT.

11

Network Virtualization

4. BUILDING VIRTUAL NETWORKS

4.1 The VNP InP interface The VNP/InP interface is a key component of the network virtualization architecture. Through this interface, the VNP is able to request the establishment of a virtual network, the modification of the characteristics of a virtual network, or the removal of a virtual network. In turn, the InP acknowledges the VNP requests and notifies any relevant event (e.g. network error). The VNP is expected to build the virtual network topology and define resource capacity requirements, namely link bandwidth and node computational capability. As discussed before, other characteristics such as geographical location of the edge nodes will be needed in many cases. The information provided by the VNP to the InP must contain a model of the virtual network topology as a graph, with a set a virtual nodes and virtual links and the applicable constraints. Each virtual node and virtual link must be characterized by a number of parameters. A tentative list of parameters to characterize virtual networks, nodes, and links is shown in Table I. The VNP is expected to identify (one or multiple) candidate InPs that are able to provide the required physical resources. This selection presupposes a certain level of knowledge by the VNP about the network infrastructure run by each of the candidate InPs. This means that InPs are expected to announce the relevant information (including cost) in order for the VNP to make an informed decision about the resources to acquire to build a new VN. At the very least, the VNP has to be aware of some basic characteristics of the infrastructure: for example, the geographic reach of the infrastructure must be made public which means that the edge nodes of the InP network (or PoPs) must be externally known. However, the knowledge of the infrastructure internals by VNP is a much more contentious issue. In theory, a continuum of possible scenarios could be defined, ranging from no visibility (except PoPs) to full visibility, depending on the business and trust relationship between VNP and InP. However, in most practical cases, the internals of the network infrastructure are not exposed to the VNP. In most cases, the mapping of virtual nodes and links into the substrate nodes and links will be performed by the InPs as they are supposed to be able to manage resources
Dept. Of Computer Science,CUSAT.

12

Network Virtualization and engineer the network according to their own rules, independently of VNPs and VNOs. For example, virtual nodes and links may be physically relocated by the InP without the VNP and VNO even being aware. 4.2 VN admission control and mapping of virtual resources into physical resources Once the InP receives a VN setup request from a VNP, two actions must be performed: VN admission control: the InP verifies the availability of resources to fulfill the request to setup or modify a VN and decides whether or not it can be accepted. Virtual-to-physical resource mapping: the InP identifies the set of possible substrate nodes and links to host the requested virtual network and selects the optimal solution based on a specific criteria. Finding the optimal mapping between virtual resources and physical resources has some similarities with the traditional MPLS traffic engineering problem of selecting the best path for data traffic through a network domain, based on a set of constraints and performance objectives. In reality, this problem is considerably more complex because node virtualization brings a new dimension the computational resources of the network nodes. Several theoretical approaches have been proposed to handle this kind of problem . In practice, the mapping of virtual nodes into physical nodes is often constrained by physical location, in which case the selection of the physical node to associate with a specific virtual node is fixed, or limited to a small set of choices. This is the case of edge nodes, or PoPs, which for economical reasons are supposed to be physically close to customers or end-users. Typically, at least one virtual node should be located in each geographic area (e.g. city, region) where the service is to be deployed. On the contrary, for other types of virtual nodes, physical location is not relevant from the VNP point of view this usually is the case with core nodes (i.e. core VN-capable nodes and core transport nodes, following the terminology presented in Figure 4.2.1), without direct connection to end users.

Dept. Of Computer Science,CUSAT.

13

Network Virtualization

Fig 4.2.1 Mapping virtual resources into physical resources A simple example is shown in Figure 4.2.1. The virtual network, as requested by the VNP to the InP, is shown in the upper side of the figure and the bottom part represents the physical infrastructure topology, which accommodates the requested virtual network.. The proposed process of mapping virtual resources into physical resources can be divided in two sequential steps: 1. Allocation of edge virtual nodes, which are the points where traffic enters or exits the virtual network, typically constrained by physical location. In the example depicted in Figure 4.2.1, the mapping of virtual nodes A, B, C, F, G, H is supposed to be decided based on the geographic location of the respective service points of presence to a, b, e, l, m, p, respectively. 2. Allocation of core virtual nodes. In the example of Figure 5, this corresponds to nodes D and E, which could be mapped, in principle, to any of the substrate nodes f to k. This step is composed of two different but inter-related sub-problems: i. Selecting the substrate nodes to accommodate the requested virtual nodes; ii. Finding the best path to map the virtual links that interconnect the virtual nodes. Step 1 should be trivial in most cases, but step 2 may be quite complex. There are several possible approaches to handle subproblems i and ii above. A naive approach

Dept. Of Computer Science,CUSAT.

14

Network Virtualization would consist of handling the two problems in sequence i.e., first mapping the virtual nodes to physical nodes and, once this is complete, finding the best path for the virtual links. However, the optimal solutions for the two problems are not necessarily coincident. The selection of the physical nodes to support the requested virtual nodes depends on a number of metrics. The optimal solution is supposed to result from the proper combination of these metrics. Naturally, different factors may have different weight in the decision. For example, it is quite possible that network node resources constitute a more critical issue than link bandwidth availability. Ultimately, this should be decided based on the specific network characteristics and the policies defined by the network infrastructure operator. In the example shown in Figure 4.2.1, nodes substrate f and k have been selected out of six possible choices (f, g, h, i, j, k) to host virtual nodes D and E. Even if a specific substrate node is able to offer greater availability of resources, other metrics, such as available link bandwidth may impose a different choice. For any solution, a number of pre-defined constraints must be fulfilled, namely the maximum number of virtual nodes per physical node, the maximum occupancy of node computational resources, the maximum number of virtual links per physical link, the maximum percentage of bandwidth in a physical link that can be allocated to virtual links. VN admission control and the corresponding virtual-to-physical resource mapping are closely intertwined and cannot be separated into independent tasks. Both require an updated knowledge of the load and available resources in the network infrastructure and both are part of the same process. The selection of the network resources to be allocated in reply to a virtual network request has to be decided by the InPs, based on their updated knowledge of the physical resources. 4.3 Network reoptimization When a new virtual network request is received, the InP maps the virtual resources to physical resources taking into account the current state of the network, particularly the level of occupancy of the network resources at that moment. However, network state keeps changing as new VNs are setup and others are torn down, or as a

Dept. Of Computer Science,CUSAT.

15

Network Virtualization result of node or link failure conditions. This often leads to inefficient utilization of resources, in which some parts of the network infrastructure (either link resources or node resources) become excessively loaded while others are under-utilized. Therefore, the capability to reoptimize the allocation of virtual resources across the substrate network without traffic disruption should be a VN requirement. Reoptimization includes the process of finding the optimal path for a flow and, if necessary, rerouting it in a seamless way (i.e. not visible to the VNP or VNO). This process may be performed at request, on a periodic basis or as a result of updated network information. Two kinds of reoptimization can be performed in a network infrastructure supporting virtualization: rerouting of virtual links (keeping virtual nodes unchanged) and reallocation of virtual nodes to different substrate nodes. The first case corresponds to the traditional traffic engineering reoptimization, whereas the second case is a far more complex process, mainly because of the complexity of moving virtual nodes without a noticeable disruption of the service. Again, both kinds of reoptimization are supposed to be carried out by the InP, without any noticeable impact to the VNO.

Dept. Of Computer Science,CUSAT.

16

Network Virtualization

5. I/O VIRTUALIZATION
The recent resurgence in popularity of virtualization has led to its use in a growing number of contexts ,many of which require high-performance networking. Consider server consolidation, for example.The efficiency of network virtualization directly impacts the number of network servers that can effectively be consolidated onto a single physical machine. Unfortunately, modern network virtualization techniques incur significant overhead, which limits the achievable network performance. We need new network virtualization techniques to realize the full benefits of virtualization in network-intensive domains. To share a network interface among a set of virtual machines, the VMM (virtual machine monitor) must accomplish two key tasks. First, the VMM must provide shared access to the network interface. This means that the virtual machines outgoing network traffic must be multiplexed together before being sent over the network; similarly, incoming network traffic must be demultiplexed before being delivered to the appropriate virtual machines. Second, the VMM must protect the virtual machines from each other. This means that no virtual machine can be allowed to transfer data into or out of another virtual machines memory. Therefore, the challenge in network virtualization is to provide efficient,shared, and protected access to the network interface. I/O virtualization is by no means a new concept. The IBM System/360 allowed virtual machines to access I/O devices. In fact, many of the System/360s pioneering mechanisms are still in use today. The prevailing I/O virtualization architectures can be broadly classified into two categories: private assignment of each I/O device to a particular virtual machine or shared assignment of a particular I/O device to multiple virtual machines. 5.1 Private I/O Devices IBMs System/360 was the first widely available virtualization solution.1 The System/370 extended the virtualization support for processors and memory but continued

Dept. Of Computer Science,CUSAT.

17

Network Virtualization to use the same mechanisms as the System/360 for I/O virtualization.2 The initial I/O virtualization architectures developed for these systems did not permit shared access to physical I/O resources. Instead, physical I/O devices, such as terminals, were assigned exclusively to a particular virtual machine. Therefore, each virtual machine required its own physical I/O devices. These systems used channel programs to transfer data to/from I/O devices. The channel programs used programmed I/O to transfer data between memory and the I/O device. On a virtualized system, the channel programs were executed inside the VMM, allowing the VMM to ensure that a virtual machine could access only its own memory and devices. More recent virtualization systems have also relied on private device access, such as IBMs first release of the LPAR (logical partitioning) architecture for its Power4 processors.3 The Power4 architecture isolated devices at the PCI-slot level and assigned them to a particular virtual machine instance for management. Each virtual machine required a physically distinct disk controller for disk access and a physically distinct network interface for network access. Modern I/O devices use DMA (direct memory access) instead of programmed I/O, so it is not possible for the VMM to perform I/O data transfers. Instead, the Power4 LPAR architecture uses an IOMMU (I/O memory management unit) to restrict the memory that can be accessed by each device. To use an IOMMU in this manner, the VMM creates an I/O page table for each device, with memory mappings for the pages owned by the virtual machine to which that device is assigned. The IOMMU then consults the appropriate page table for every DMA operation. If a device tries to access memory without a valid mapping in its I/O page table, then the IOMMU will disallow the access. Private I/O access has obvious benefits and drawbacks.It yields highperformance I/O access, as each virtual machine can communicate directly with the physical devices that it owns. It is a costly solution, however, since it requires replicated physical devices for each virtual machine. These devices are likely to be underutilized, and for systems with large numbers of virtual machines it may be impossible to include a sufficient number of devices.

Dept. Of Computer Science,CUSAT.

18

Network Virtualization 5.2 Shared I/O Devices A good way to decrease the costs inherent in providing private I/O devices is to allow I/O devices to be shared among virtual machines. The first shared I/O virtualization solutions were part of the networking support for System/360 and System/370 between physically separated virtual machines.4 This networking architecture supported shared access to the network using a virtualized spool-file interface that was serviced by a special-purpose virtual machine, or I/O domain, dedicated to networking.The other virtual machines in the system could read from or write to virtualized spool files. The VMM would interpret these reads and writes based on whether the spool locations were physically located on the local machine or on a remote machine. If the data were on a remote machine, the VMM would transfer control to the I/O domain, which would then use its physical network interfaces to transfer data to/from a remote machine. The remote machine would use the same virtualized spool architecture and its own dedicated I/O domain to service requests This software I/O virtualization architecture is logically identical to most virtualization solutions today. Xen, VMware, and the Power5 virtualization architectures all share access to devices through virtualized software interfaces and rely on a dedicated software entity, such as the I/O domain, to perform physical device management.The software entity controlling the I/O device can be either a separate I/O domain or the VMMs hypervisor.If there is a separate I/O domain, I/O devices are effectively private to that domain, so memory accesses by the device should be restricted to the I/O domain. This can be enforced either by trusting the I/O domain or by using an IOMMU.

Dept. Of Computer Science,CUSAT.

19

Network Virtualization

6.NETWORK VIRTUALIZATION IN SOFTWARE

Since network interfaces are commonly shared in software,it would be instructive to look at a more concrete example of this I/O virtualization architecture. Xen is a popular open source VMM for the x86 architecture that shares I/O devices in software.8 Although differences exist among VMMs, Xens I/O virtualization architecture is representative of other systems that support shared I/Odevices in software. 6.1 Xen Network Virtualization Architecture Figure 6.1.1 shows the organization of the Xen VMM. Xen consists of two elements: the hypervisor and the driver domain. The hypervisor provides an abstractionlayer between the virtual machines and the actual hardware, enabling each virtual machine to execute as if it were running natively on the hardware.Shared I/O devices, however, are controlled by a special I/O domain,called the driver domain in Xen. It runs a modified version of Linux that uses native Linux device drivers to manage physical I/O devices.

Fig6.1.1 Xen Network Virtualization Architecture

Dept. Of Computer Science,CUSAT.

20

Network Virtualization To communicate with the driver domain, and thus the physical I/O devices, each virtual machine is given a virtual I/O device, controlled by a special device driver, called a front-end driver. To access a physical device the virtual machines front-end driver communicates with the corresponding back-end driver in the driver domain. The back-end network interface drivers are connected to the physical NIC (network interface card) by an Ethernet bridge within the driver domain. When a packet is transmitted by a virtual machine, it is first copied (or remapped) from the front-end driver in the virtual machine to the back-end driver in the driver domain.Within the driver domain, the packet is routed through the Ethernet bridge to the physical NICs device driver, which then enqueues the packet for transmission on the network interface as if it were generated normally by the operating system within the driver domain. When a packet is received, the network interface generates an interrupt that is captured by the hypervisor and routed to the NICs device driver in the driver domain as a virtual interrupt. The NICs device driver transfers the packet to the Ethernet bridge, which routes the packet to the appropriate back-end driver. The back-end driver then copies (or remaps) the packet to the front-end driver in the target virtual machine. Once the packet is transferred,the back-end driver requests that the hypervisor send an additional virtual interrupt to the target virtual machine notifying it of the new packet. Upon receiving the virtual interrupt, the front-end driver delivers the packet to the virtual machines network stack, as if it had come directly from a physical device. In Xen, the driver domain is responsible for protecting I/O accesses. This means that the driver domain must be trusted to direct the network interface only to use buffers that are owned by the driver domain. Future x86 systems will include an IOMMU that could be used by the hypervisor to enforce that restriction. Regardless, the driver domain must also be trusted to transfer only the appropriate network traffic to each virtual machine. The Xen network virtualization architecture incurs significant processing and communication overhead. For example, Linux is able to achieve only about one-third of the network throughput inside of Xen that it would be able to achieve running natively.9 This overhead is not exclusive to Xen; the basic operations that must be performed in software to enable shared I/O devices are common among VMM implementations.

Dept. Of Computer Science,CUSAT.

21

Network Virtualization An often-overlooked component of the overhead involved with shared I/O devices is domain scheduling within the VMM. Even if there is only a single virtual machine, sending or receiving a network packet involves two domains: the driver and the virtual machine. These domains must be scheduled, and in the correct order, before a network packet can be sent or received. This means that poor scheduling decisions can result in increased network latency. Part of the scheduling problem can be eliminated by managing shared I/O devices directly in the hypervisor. VMware ESX11 and early versions of Xen12 used this approach. This eliminates the need for scheduling an additional I/O domain in order to transfer a network packet. Promptly scheduling the virtual machines domain is still necessary, however, though it is not always possible, especially if multiple domains simultaneously have outstanding network traffic. Perhaps more importantly, this requires the device drivers for the network interfaces to be incorporated directly within the hypervisor. This negates one of the often-touted advantages of virtualization: removing the frequently buggy device drivers from the trusted code base of the machine. With the device drivers directly in the hypervisor, the complexity of the hypervisor is significantly increased, and its reliability is correspondingly decreased. Multiqueue Network Inter faces. 6.2 Multiqueue Networking Interfaces Typically, modern network interfaces include a transmit queue and a receive queue. The transmit queue holds pointers to outgoing network packets, and the receive queue holds pointers to empty buffers that will be used to store incoming network packets. The device driver communicates with the network interface almost exclusively through these queues. In recent years, network interfaces with multiple sets of these queues (sometimes referred to as multiqueue network interfaces) have emerged to improve networking performance in multicore systems. These devices support Microsofts Receive Side Scaling architecture and Linuxs Scalable I/O architecture. These architectures allow each core exclusive access to one of the sets of queues on the network interface. This enables each core to run the network interfaces device driver concurrently without the

Dept. Of Computer Science,CUSAT.

22

Network Virtualization need for additional synchronization. The use of multiqueue network interfaces in this manner increases the achievable parallelism within the network stack of the operating system, enabling more effective use of multiple cores for networking. Researchers at Hewlett-Packard Laboratories and Citrix have recently come up with a way to use these multiqueue network interfaces to improve the performance of network virtualization in Xen. They eliminate the Ethernet bridge within the driver domain by assigning network interface queues directly to back-end drivers in the driver domain. In this manner, the back-end drivers can communicate with the network interface concurrently without the need for synchronization. The network interface, rather than the Ethernet bridge within the driver domain, will then multiplex/demultiplex network traffic. To multiplex transmit network traffic, the network interface interleaves the network traffic of the virtual machines by servicing all of the transmit queues fairly. When the network interface receives network packets, the NIC uses each packets destination Ethernet address to identify the appropriate receive queue. The obvious benefit of using multiqueue network interfaces in this fashion is the elimination of the trafficmultiplexing overhead within the driver domain. A less obvious benefit is the elimination of copying between the driver domain and the virtual machines. Because the network traffic for each virtual machine will interact with the network interface only through a single set of queues, the driver domain can direct the network interface to transfer data directly to/from memory owned by that virtual machine. The virtual machine need only grant the driver domain the right to use its network buffers. The use of multiqueue network interfaces in this way should improve network virtualization performance by eliminating the multiplexing and copying overhead inherent in sharing I/O devices in software among virtual machines. This technique still has a few drawbacks, however. First, the driver domain must now protect each virtual machines network traffic by ensuring that the buffers enqueued to a particular queue on the network interface belong to the virtual machine assigned to that queue. Second, overhead and complexity are inherent in managing the buffers that virtual machines have granted to the driver domain to use for networking. Finally, the scheduling problems described earlier remain, as the driver domain must still be scheduled to communicate with the network interface by enqueuing/dequeuing buffers and receiving interrupts.

Dept. Of Computer Science,CUSAT.

23

Network Virtualization 6.3 CDNA Network Virtualization Architecture CDNA (concurrent direct network access) takes the use of multiqueue network interfaces one step further. In the CDNA network virtualization architecture, the hypervisor directly assigns each virtual machine its own set of queues in a multiqueue network interface.14 This eliminates the driver domain altogether from networking in Xen. Figure 6.3.1 shows the CDNA network virtualization architecture. As before, the network interface must support multiple sets of queues directly in hardware. Instead of assigning ownership of the entire network interface to the driver domain, however, the hypervisor treats each set of queues as if it were a physical network interface and assigns ownership of each set directly to a virtual machine. Notice the absence of the driver domain from the figure: each virtual machine can transmit and receive network traffic using its own private set of queues without any interaction with other virtual machines or the drive domain. The driver domain, however, is still present to perform control functions and allow access to other I/O devices.

Fig6.3.1 CDNA Network Virtualization Architecture

Dept. Of Computer Science,CUSAT.

24

Network Virtualization In the CDNA network virtualization architecture, the communication overhead between the driver domain and the virtual machines is eliminated entirely. The hypervisor must now provide protection across the sets of queues on the network interface and deliver interrupts directly to the virtual machines. Similar to the use of multiqueue network interfaces by the driver domain, CDNA eliminates the software-multiplexing overhead within the driver domain by multiplexing/ demultiplexing network traffic on the network interface. As before, each virtual machine is assigned a set of queues and an associated Ethernet address to enable the network interface to perform this task. Interrupts from the network interface, however, are no longer delivered to the appropriate back-end driver in the driver domain. Instead, they are delivered directly to the appropriate virtual machine. To do so, the hypervisor must understand that the network interface is owned by multiple virtual machines. Normally, the hypervisor assumes that each device is owned exclusively by a particular virtual machine. In the prototype implementation of CDNA, the network interface delivers a bit vector to the hypervisor and then generates a physical interrupt. Each bit in the bit vector corresponds to one set of queues on the network interface. The hypervisor then scans this bit vector and delivers virtual interrupts to the appropriate virtual machines. Similar behavior could be accomplished using message signaled interrupts, at the cost of an increased physical interrupt rate. Delivering interrupts directly to the virtual machines, without an intervening trip through the driver domain, reduces the virtual machines interrupt response time. The CDNA network virtualization architecture does improve the scheduling issue to some degree. For a virtual machine to transmit or receive network traffic, it does not need the driver domain to be scheduled. This improves overall responsiveness, which leads to improved network performance. The problem of scheduling multiple virtual machines with outstanding network traffic remains, however. Although the CDNA network virtualization architecture eliminates the overhead of the driver domain, it complicates memory protection. Since the trusted driver domain is no longer moderating access to the network interface, the virtual machines could potentially direct the network interface to access arbitrary memory locations. This problem is particularly acute in the x86 architecture, in which device drivers provide I/O

Dept. Of Computer Science,CUSAT.

25

Network Virtualization devices with physical addresses to memory. If a faulty or malicious virtual machine were to provide a physical address to the network interface that does not belong to its address space, it could co-opt the network interface into reading or writing memory that is not owned by that virtual machine. To guarantee memory protection in CDNA, the hypervisor must perform two tasks. First, it must validate all buffers before they are delivered to the network interface. The hypervisor validates a buffer only if it is owned by the virtual machine using it, ensuring that the virtual machine does not enqueue a buffer to the network interface that it does not own. If the network interface receives an unvalidated buffer, that buffer will not be used. Second, the hypervisor must ensure that the ownership of any buffer enqueued on the network interface does not change. In Xen, memory ownership is maintained by using reference counts to a page. In CDNA, a pages reference count is incremented when a buffer within that page is enqueued to the network interface, and it is decremented when the buffer is dequeued. This guarantees that the pages ownership does not change, as Xen will allow the ownership of a page to change only if its reference count is zero. While the CDNA prototype performs these protection tasks in software, they can be simplified with the use of an IOMMU. In a system with an IOMMU, the hypervisor would create an I/O page table for each virtual machine, with memory mappings for the pages that the virtual machine is allowed to use for I/O operations (potentially all of the virtual machines memory). The hypervisor would update these I/O page tables whenever memory ownership changed. This would obviate the need to validate each buffer individually and to track the ownership of network buffers. Instead, the IOMMU would consult the I/O page tables for every network transfer to ensure that each memory access made by the network interface was allowed. For this to work, however, the IOMMU must be able to differentiate memory accesses made by the network interface on behalf of the different virtual machines. The IOMMU needs this information to determine which I/O page table should be consulted for a particular access. Current PCI bus implementations offer no obvious method for a network interface to differentiate its memory accesses in this way, but the upcoming PCI I/O virtualization specification should include such a capability.

Dept. Of Computer Science,CUSAT.

26

Network Virtualization Figure6.3.2 shows the achievable aggregate bandwidth using both the Xen and CDNA network virtualization architectures as the number of virtual machines is varied. Only two network interfaces are used, limiting the bandwidth to 2 Gbps. As the figure shows, direct access to the network interface yields dramatic performance improvement. In fact, the figure understates the overall improvement, as there is idle time when using the CDNA network virtualization architecture. These idle processing resources could be used to achieve even higher network bandwidth if there were additional network interfaces in the system. In contrast, there is never idle time when using the Xen network virtualization architecture, so the figures show the peak network bandwidth of Xen.

Fig6.3.2 Performance of CDNA and Xen Network Virtualization Architectures

Dept. Of Computer Science,CUSAT.

27

Network Virtualization

7. CONCLUSION
For the past several decades, virtual machine monitors have shared I/O devices in software. Although such software I/O virtualization architectures are simple and flexible, they create a significant network performance barrier, which severely limits the value of virtualization for network-intensive domains. To realize the full potential of virtualization, we need more efficient network virtualization architectures. Given the unsolicited nature of network traffic, it is valuable to perform traffic multiplexing and demultiplexing directly on the network interface. This capability has already shown promise in improving the network efficiency of multicore systems. CDNA is an upcoming network virtualization architecture that further uses this capability to provide many of the performance benefits of using private network interfaces, without requiring a large number of underutilized network interfaces. This will dramatically increase the efficiency of virtualized systems and increase the number of network-intensive virtual machines that each physical machine can support.

Dept. Of Computer Science,CUSAT.

28

Network Virtualization

8. REFERENCES
1. Scott Rixner,Network Virtualization:Breaking the Performance Barrier , ACM Q ueue,Vol.6No.1,Pages 36 to 52,January/February 2008. 2. Tom Killalea,Meet the Virts,ACM Queue,Vol.6No.1,Pages14-18,January/February 2008. 3. Jorge Carapinha, Javier Jimnez ,Network virtualization: a view from the bottom , ACM,Pages73-80, August 2009. 4. http://networkvirtualization.com/Technologies/Network+Virtualization 5. http://www.cisco.mn/en/US/netsol/ns658/index.html

Dept. Of Computer Science,CUSAT.

29