CSC 660: Advanced OS

Virtual Machines

CSC 660: Advanced Operating Systems

Slide #1

Topics
1. 2. 3. 4. 5. 6. 7. 8. What is a VM? Process vs System VMs Virtualizing the Processor Virtualizing Memory Virtualizing I/O VM Performance Issues Intel VT-x Technology Paravirtualization
CSC 660: Advanced Operating Systems Slide #2

What is a VM?
A virtualized system that
Provides a consistent ABI to guest programs. Runs on a host system (software + hardware.) Controls resources available to guest programs. May provide different resources than hardware
Different Type (ex: JVML in Java VM) Different Quantity (ex: more/fewer CPUs, disks, etc.)

May be of two major types

Process: provides VM to a single process. System: emulates an entire machine w/ guest OS.
CSC 660: Advanced Operating Systems Slide #3

System Models

Non-virtual Machine

Virtual Machine

CSC 660: Advanced Operating Systems

Slide #4

Why use Virtual Machines?

Portability
Run software on a different OS. Run software on a different CPU.

Aggregation
Modern machines are fast and underused. Put multiple servers in VMs on one real machine.

Development
Complex software environments. Processor testing and simulation.

Debugging
Can analyze every aspect of hardware behavior.

Security
VMs provide greater isolation of software than regular OS.
CSC 660: Advanced Operating Systems Slide #5

Types of VMs

CSC 660: Advanced Operating Systems

Slide #6

Process VMs
Multitasking
Each process in a multitasking OS. VM = System call interface + ISA + VirtMem

Emulators
Allow a process to run on a different OS/ISA. Types:
Interpreter Dynamic binary translator

High Level Language VMs

ex: Pascal, JVM, CLR
CSC 660: Advanced Operating Systems Slide #7

HLL VMs
HLL Program Compiler Front End Intermediate Code Compiler Back End Object Code Loader Memory Image
CSC 660: Advanced Operating Systems

HLL Program Compiler Byte Code Dist VM Loader Virtual Memory Image

Dist

VM Host Instructions
Slide #8

System VMs
Virtual Machine Monitor (VMM)
Provides illusion of multiple isolated machines. Manages allocation of and access to hardware resources for multiple guest OSes. Layer between hardware and guest OS.

VMM tasks
State management Resource control
CSC 660: Advanced Operating Systems Slide #9

System VMs
Guest Apps Guest Apps Applications Guest OS OS Guest OS VMM

VMM
Hardware Hardware

Host OS
Hardware

a. Traditional OS

b. Native VMM

c. User-mode Hosted VMM

Slide #10

CSC 660: Advanced Operating Systems

VMM Modes
Requirement: guest OS may not change hardware resources to impact other VMs or the VMM. Guest OS runs in user mode. VMM runs in supervisor mode.
Tracks virtual mode of VM. User programs run in virtual user mode. OS runs in virtual supervisor mode.

Exceptions & interrupts invoke VMM.

VMM can handle directly or produce a virtual exception for guest OS.
CSC 660: Advanced Operating Systems Slide #11

System VM Execution
1. 2. 3. 4. 5. 6. 7. Timer Interrupt in running VM. Context switch to VMM. VMM saves state of running VM. VMM determines next VM to execute. VMM sets timer interrupt. VMM restores state of next VM. VMM sets PC to timer interrupt handler of next VM. 8. Next VM active.
CSC 660: Advanced Operating Systems Slide #12

IBM VM/370
Mainframe VMM OS.
First VM environment in System/360 1965. Control program was a native VMM. Each user had VM running single-user CMS. Principles still used in z/VM on IBM zSeries.

CSC 660: Advanced Operating Systems

Slide #13

VMWare
x86-based VM. VMWare Workstation
Hosted VMM runs on Linux or Windows. Any x86 OS can be used as guest OS.

VMWare ESX Server

Native VMM runs directly on x86 hardware. VMotion allows VM migration.

CSC 660: Advanced Operating Systems

Slide #14

Virtualizing the Processor

Emulation
VMM examines each guest instruction and emulates on virtualized resources the effects that instruction would have had on real resources. Only method when host/guest ISA differs. May be necessary with identical host/guest ISA.

Direct Native Execution

Host ISA must be identical to guest ISA. Most instructions executed directly on CPU. Some sensitive instructions must be emulated.
CSC 660: Advanced Operating Systems Slide #15

Privileged Instructions
Instrs that trap if executed in user mode.
I/O Memory Management

Can only be executed in supervisor mode.

CSC 660: Advanced Operating Systems

Slide #16

Handling Privileged Instructions

Instruction Trap

Dispatcher Instructions to change machine resources. Allocator Interpreter Routine #1

Interpreter Routine #N

CSC 660: Advanced Operating Systems

Slide #17

Handling Privileged Instructions

1. 2. 3. 4. 5. 6. 7. Instruction Trap invokes VMM Dispatcher. Dispatcher calls Instruction Routine. Changes mode to supervisor. Emulates instruction. Computes return target. Restores mode to user. Jumps to target.
CSC 660: Advanced Operating Systems Slide #18

Sensitive Instructions
Some instructions cannot be executed directly on the CPU because they would interfere with the VMM or other VMs.
Control-sensitive Instructions
Attempt to modify resource configuration.

Behavior-sensitive Instructions
Behavior depends on resource configuration, including user/kernel mode setting.

CSC 660: Advanced Operating Systems

Slide #19

Non-Privileged Sensitive Instructions

Example: POPF
Pops flag registers from stack. Includes interrupt-enable flag. User mode, POPF modifies all but interrupt flag. Kernel mode, POPF modifies all flags.

CSC 660: Advanced Operating Systems

Slide #20

Efficient VMs
If all sensitive instructions are privileged, the processor is considered to be virtualizable. All sensitive instructions produce a trap. If non-privileged sensitive instructions exist, then the VMM must examine all instructions before execution to trap these instructions.
Emulation Dynamic Binary Translation (Patching)
CSC 660: Advanced Operating Systems Slide #21

Memory Virtualization
Virtual Memory in a VM
Each guest OS maintains own set of page tables. Guest OS translates virtual memory locations to real memory locations (physical memory of VM.) Guest OS has swap space on virtual disk.

VMM
Translates real memory to physical memory using MMU. Doubles number of page faults. VMM has swap space on physical disk.

CSC 660: Advanced Operating Systems

Slide #22

Shadow Page Tables

VMM maintains shadow page tables
Map virtual->physical instead of real->physical. VMM validates guest page table updates. Replicates guest changes in shadow page table.

Virtualize page table pointer register.

VMM manages real page table pointer. Updates page table ptr when switching VMs.

CSC 660: Advanced Operating Systems

Slide #23

Shadow Page Tables

guest reads Guest Page Table

guest writes
Accessed & dirty bits Updates

Guest OS

Shadow Page Table

VMM MMU
CSC 660: Advanced Operating Systems

Hardware
Slide #24

I/O Virtualization
VMM must intercept all guest I/O ops.
PC: privileged IN and OUT instructions. I/O operation may consist of many INs/OUTs.

Problem: huge array of diverse hardware

Native VMM needs driver for each device. Hosted VMM uses host drivers w/ perf penalty.

CSC 660: Advanced Operating Systems

Slide #25

Virtualizing Devices
Dedicated Devices
VM has sole control of device.

Partitioned Devices
VM has dedicated slice of device, treats as full. VMM translates virtual full dev parameters to parameters for underlying physical device.

Shared Devices
VMM can multiplex devices. Each VM may have own virtual device state.

Nonexistent Devices
Virtual software devices with no physical counterpart.
CSC 660: Advanced Operating Systems Slide #26

Virtualizing a Network Card

CSC 660: Advanced Operating Systems

Slide #27

VM Performance
Why is VM slower than physical hardware?
Emulation: Sensitive instructions must be emulated. Interrupt Handling: VMM must handle interrupts, even if eventually passed to guest. Context Switches: VMM must save VM state when controlled transferred to VMM. Bookkeeping: VMM has to do work to simulate behavior of real machine, such as keeping track of time for VMs. Memory: Memory accesses may require access to both shadow and local page tables.

CSC 660: Advanced Operating Systems

Slide #28

VT-x Technology
New CPU modes: VMX root/non-root
VMM runs in VMX root. Guest VMs run in VMX non-root. Each mode has rings 0..3.

Virtual Machine Control Structure (VMCS)

Guest Area, Host Area.

Transitions
VM Entry: root to non-root transition.
Load processor state from VMCS guest area.

VM Exit: non-root to root transition.

Save state to VMCS guest area, load state from host area.
CSC 660: Advanced Operating Systems Slide #29

VT-x Technology
Instructions
Some sensitive instructions operate on non-root VMX state; others produce a VM exit. VMM controls which instructions VM exit.

Interrupts
External interrupts cause VM exits. VMM controls which exceptions VM exit.

CSC 660: Advanced Operating Systems

Slide #30

Paravirtualization: Xen
Provide VM abstraction similar to hardware.
Modifies guest OS to use Xen/x86 architecture.

Memory
Guest has read access to hardware page tables. Updates batched and validated by Xen VMM.

CPU
Guest OS installs direct system call handler. Sensitive instructions replaced with Xen calls.

I/O
Event mechanism replaces hardware interrupts.
CSC 660: Advanced Operating Systems Slide #31

Xen 1.2 Architecture

CSC 660: Advanced Operating Systems

Slide #32

Xen VMM
VMM resides in top 64MB. Protected by segmentation, not page tbl for perf.
4GB 3GB Xen Kernel S

S
ring 3 ring 1 ring 0
Slide #33

User

0GB

CSC 660: Advanced Operating Systems

Xen System Performance

1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0

SPEC INT2000 (score)

Linux build time (s)

OSDB-OLTP (tup/s)

SPEC WEB99 (score)

Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)
CSC 660: Advanced Operating Systems Slide #34

References
1. 2. 3. 4. Paul Barham et. al., Xen and the Art of Virtualization, 19th ACM Symposium on Operating Systems Principles, Oct. 19-22 2003. Ian Pratt, Xen 3.0 and the Art of Virtualization, Ottawa Linux Symposium 2005. John Scott Robin and Cynthia E. Irvine, Analysis of the Intel Pentiums Ability to Support a Secure Virtual Machine Monitor, Proceedings of the 9th USENIX Security Symposium, Aug 14-17 2000. Mendel Rosenblum and Tal Garfinkel, Virtual Machine Monitors: Current Technology and Future Trends, IEEE Computer, May 2005. James E. Smith and Ravi Nair, Virtual Machines, Elsevier, 2005. Abraham Silberschatz, Peter Baer Galvin, and Greg Gagne, Operating System Concepts, 6th edition, Wiley, 2003. Jeremy Sugerman, et. al., Virtualizing I/O Devices on VMware Workstations Hosted Virtual Machine Monitor, Proceedings of the 2001 USENIX Annual Technical Conference, 2001. Rich Uhlig et. al., Intel Virtualization Technology, IEEE Computer, May 2005.

5. 6.
7.

8.

CSC 660: Advanced Operating Systems

Slide #35