Oracle VM

The Underground Oracle VM Manual http://itnewscast.
com/book/export/html/844
The Underground Oracle VM Manual

underground.jpg
The only Oracle Virtualization and Oracle VM Manual. New chapters will be added and posted to ITNewsCast regularly.
The Underground Oracle VM Manual
Oracle VM for x86 Deployment Guide
Show a printer friendly version of this book with its sub pages
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved.
Author: Roddy Rodstein, CISSP
Change Log
Revision Change Description Updated By Date
1.0 First Release of the Roddy 05/2009

Underground Oracle VM Rodstein
Manual
2.0 Second Release of the Roddy 09/2009

Manual
3.0 Third Release of the Roddy 08/2010

Manual
4.0 Forth Release of the Roddy 12/2010

Manual
1 Complete rewrite of Roddy 03/08/11

Chapter 1 Rodstein
1 Oracle VM Centralized Roddy 03/29/11

Logging Draft Rodstein

Chapter 2 Rodstein
Limits of Liability and Disclaimer of Warranty

This publication contains information protected by copyright. This publication may not be duplicated in any way without the express
written consent of the publisher, except in the form of brief excerpts or quotations for the purpose of review. The information
contained herein is for the personal use of the reader and may not be incorporated in any commercial programs, other books,
databases, or any kind of software without the written consent of the publisher. Making copies of this book or any portion, for any
purpose other than your own, is a violation of United States copyright laws.
Warning and Disclaimer

Every effort has been made to make this publication as complete and as accurate as possible, but no warranty or fitness is implied.
The information provided is on an "as is" basis. The authors and the publisher shall have neither liability nor responsibility to any
person or entity with respect to any loss or damages arising from the information contained in this publication.
The information found in this publication was gathered from many different sources in the computing world. It is provided for
informational purposes only. Use common sense in applying these concepts and tips. Screen shots may vary from environment to
environment. Please verify correctness and applicability in a test environment first and then deploy to your production
environment(s).
Trademarks
Trademarked names appear throughout this publication. Rather than listing the names and entities that own the trademarks or
include a trademark symbol with each mention of the trademark name, the publisher states that he is using the name for editorial
purposes only and to the benefit of the trademark owner, with no intention of infringing upon that trademark.
About the Author

Roddy Rodstein (CISSP, CEH) has over 10 years of professional experience in the IT industry. In his current role at Oracle Roddy is a
member of the Virtualization and Linux team. Before joining Oracle, Roddy spent six years at Citrix supporting Citrix's core product,
XenApp. Earlier in his career Roddy successfully established, owned, and operated an IT consulting business that specialized in server
based computing and remote access solutions. His professional achievements also extend to writing and self-publishing industry
reference guides currently available on Amazon, Securing Microsoft Terminal Services (ISBN: 061514330X) and Citrix CCA
MetaFrame 1.8 for Windows Exam Cram (ISBN: 1576109453).
Audience
This book is intended to assist solution architects, sales engineers, field engineers and consultants in planning, designing, deploying
and supporting Oracle VM for x86 in internal and external clouds. This book assumes that the reader has an architectural
understanding of cloud computing, Oracle technologies, storage and network system, and related software.
1 of 243 18.04.2011 16:35

The Underground Oracle VM Manual http://itnewscast.com/book/export/html/844
Objectives
This book is intended to articulate the design considerations and validation efforts required to design, deploy and support Oracle VM
for x86 in an internal or external cloud.
Chapters at a Glance
Chapter 1: Oracle VM Introduction
Chapter 2: Oracle VM for x86 Reference Design and Architectural Introduction
Chapter 3: Hard and Soft Partitioning Oracle Technologies with Oracle VM
Chapter 4: Oracle VM Server Sizing, Installation and Updates
Chapter 5: Oracle VM Manager Sizing, Installation, Updates and the Oracle VM Manager Command Line Interface
...Oracle VM 2.1.5 to 2.2 Upgrade
Chapter 6: Oracle VM 2.2 SAN, iSCSI and NFS Back-end Storage Configurations
Chapter 7: Oracle VM Networking from A to 802.1Q
Chapter 8: Virtualizing Oracle Database 10g/11g
Chapter 9: Virtualizing Oracle Enterprise Manager 10g
Oracle VM Centralized Logging
The Virtualization Policy Project
Watch this space

Oracle OpenWorld 2011 Call for Papers - Submit Today!
Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7)
Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files
New VDI Training Course now available!
HIMSS 2011 and New Press Release
Oracle Desktop Virtualization at HIMSS 2011
Oracle VM VirtualBox 4.0.4 Released!
Two Virtualization Webinars This Week
We're Hiring! - Server and Desktop Virtualization Product Management
Chapter 1: Oracle VM Introduction

An introduction to Oracle VM
Table of Contents
Change Log
The Evolution of Oracle VM for x86
Oracle VM for x86 Introduction
...Oracle VM for x86 Virtualization Modes
...Oracle VM for x86 Packaging and Pricing
Accelerating Application Deployments with Oracle VM for x86
...Virtual Machine Cloning
...Oracle VM Templates
…...Oracle VM Templates Packaging and Pricing
...Oracle VM Template Builder
…...Oracle VM Template Builder Packaging and Pricing
...Oracle Virtual Assembly Builder
…...Oracle Virtual Assembly Builder Packaging and Pricing
...Oracle JRockit Virtual Edition
…...Oracle JRockit Virtual Edition Packaging and Pricing
The Oracle VM Management Pack Plug-in
Oracle's x86 Virtualization Support Policy
Change Log
Revision Change Description Updated By Date

Chapter 1 Rodstein
The Evolution of Oracle VM for x86

At Oracle OpenWorld 2007, Oracle announced its entry into the x86 server virtualization market with the first release of Oracle VM
for x86, version 2.1. Oracle VM for x86 was introduced as a low cost, high performance, open source server-virtualization solution for
2 of 243 18.04.2011 16:35

Oracle and non-Oracle workloads, backed by Oracle's world-class support organization. With the release of Oracle VM for x86, Oracle
had a flagship x86 server virtualization solution and a clear x86 server virtualization strategy.
Note: The first release of Oracle VM for x86 was version 2.1. An Oracle colleague explained to me that Larry Ellison avoids using 1.0
for Oracle product releases to help drive early adoption.
In August 2008, Oracle announced the release of Oracle VM templates. Oracle VM templates are self-contained preconfigured virtual
machines with key Oracle technologies, which can be downloaded from the Oracle Linux eDelivery portal and quickly deployed into
production using Oracle VM for x86. Oracle VM templates completely eliminate the operating system and application installation
process, reducing risk and dramatically shortening Oracle application deployment timelines.
In August 2009, Oracle announced the release of Oracle VM Template Builder. Oracle VM Template Builder is an application-
packaging studio that generates preconfigured, reusable Oracle VM templates with Oracle and non-Oracle applications on Oracle
Linux JeOS (just enough operating system). Oracle Linux JeOS is a customizable small-footprint x86 and x86-64 Oracle Linux
operating system.
On January 27, 2010 Oracle completed the acquisition of Sun Microsystems, which added Sun's software and hardware assets to
Oracle's product portfolio. Shorty after the Sun acquisition, Oracle rebranded numerous Sun virtualization technologies with the
Oracle VM product name and created the Oracle VM product family with the following three separate product lines: Oracle VM for
x86, Oracle VM for SPARC (formally LDOMs), and Oracle VM VirtualBox. Each product line has add-on products that extend the
functionally of the line. Then, Oracle announced Oracle Virtual Assembly Builder and JRockit Virtual Edition. Oracle Virtual Assembly
Builder and JRockit Virtual Edition extended Oracle VM templating into the Oracle Fusion Middleware product portfolio.
With the addition of Oracle VM templates, Oracle VM Template Builder, Oracle Virtual Assembly Builder, and JRockit Virtual Edition,
Oracle VM for x86 was transformed from a x86 server-virtualization solution into Oracle's next-generation application delivery
platform.
The next sections will examine Oracle VM for x86, Oracle VM templates, Oracle VM Template Builder, Oracle Virtual Assembly
Builder, JRockit Virtual Edition and the Oracle VM Management Pack.
Oracle VM for x86 Introduction

Oracle VM for x86 is Oracle's x86 cloud virtualization and next generation application delivery platform. Oracle VM for x86 ships
preinstalled on the Sun x86 server and blade product line and is also fully supported on third-party x86-64 servers. Oracle VM for x86
is the preferred virtualization and application delivery platform for internal and/or external clouds using x86 hardware running Oracle
Linux, Solaris x86, Red Hat Enterprise Linux, or Windows virtual machines. With Oracle VM for x86, mission-critical applications can
be migrated from physical to virtual machines with unparalleled density, performance, and ease of management.
Operating system and application installations can be very time consuming. Oracle VM for x86 offers next-generation operating
system and application packaging and deployment features that eliminate the traditional operating system and application installation
process. Oracle VM for x86 enables operating system and application configurations to be packaged in preconfigured, reusable
virtual-machine templates. Virtual machine templates can consist of a single virtual machine or a group of virtual machines, called an
assembly. Virtual machine templates can be rapidly redeployed an unlimited number of times as copies of an original virtual machine
or virtual machine assembly, or as a unique new virtual machine or virtual machine assembly.
The Oracle VM for x86 is a type 1 hypervisor that installs directly on x86-64 hardware, enabling multiple concurrently running virtual
machines to share a single piece of hardware. Each virtual machine has its own operating system, kernel, virtual CPUs, RAM, network
interfaces, storage, and applications. Oracle VM for x86 increases server utilization by breaking the traditional one-workload-per-box
approach to server provisioning while making operating systems and enterprise applications easier to deploy, manage, and support by
eliminating the operating system and application installation process.
Oracle VM for x86 consists of an x86 server component and a manager component used to manage one or more clustered servers.
The server component is based on the open source Xen.org hypervisor, and is called Oracle VM server. The Xen hypervisor is a thin
layer of code that installs directly on hardware that enables multiple guest operating systems to run concurrently on a single piece of
hardware. Xen has emerged as the industry open standard for x86 virtualization, with broad adoption from ISVs such as Oracle, IBM,
HP, Dell, and Citrix, and cloud providers such as Amazon AWS, Savvis Cloud Services, and Rackspace Cloud. Oracle makes subtle
changes to the original Xen.org code that create a unique Xen distribution, which Oracle maintains and redistributes as Oracle VM
server. Oracle VM server was designed to support the most demanding high I/O workloads like Oracle Database products and Oracle
Exalogic, which other hypervisors and Xen distributions are unable to support.
The manager component is a traditional Oracle application, named Oracle VM Manager, that installs on Oracle Linux and Red Hat
Enterprise Linux. Oracle VM Manager is used to manage one or more clustered Oracle VM servers, virtual machines, and virtual
machine resources. Oracle VM Manager is a traditional Oracle application consisting of an Oracle database, an Oracle application
server, and a J2EE application with an OS-/browser-neutral Oracle Application Development Framework (ADF) administrative portal.
All of the Oracle VM Manager components can be installed in an all-in-one configuration on an Oracle Linux virtual machine or in a
multiple node HA configuration. Oracle VM Manager is distributed from the Oracle Linux eDelivery portal as an ISO file and as a
preconfigured, production-ready Oracle VM template.
Figure 1 shows the Oracle VM Manager administrative portal.
3 of 243 18.04.2011 16:35

Oracle VM for x86 Virtualization Modes

Oracle VM for x86 supports two unique virtualization modes, paravirtualization mode (PV mode) and hardware virtualization mode
(HVM mode). Oracle VM servers can support both paravirtualization mode and hardware virtualization mode simultaneously on a
single x86-64 server using either an Intel or an AMD virtualization technology. Intel and AMD virtualization is a requirement only for
hardware virtualized mode, not for paravirtualization mode. Intel and AMD virtualization technologies are enabled and managed
using the system BIOS.
Paravirtualization mode requires the virtual machine operating system to run a Xen kernel and Xen network and I/O drivers. Xen
paravirtualized kernels ship along with the default Linux kernel with Oracle Linux and Red Hat Enterprise Linux operating systems.
Paravirtualized virtual machines are hypervisor aware and run without the additional overhead of hardware emulation.
Paravirtualization offers much less overhead with timers, interrupts, I/O traffic, and context switches, allowing superior scalability
under heavy loads when compared to hardware virtualization mode.
Unlike paravirtualization mode, which requires the virtual machine to run a Xen kernel, hardware virtualization mode supports native
unmodified operating systems. Virtual machines that run under hardware virtualization mode are called hardware virtualized
machines (HVM). Hardware virtualized machines are unaware that they have been virtualized and think they are on physical
hardware. To provide acceptable performance, hardware virtualized machines need to use paravirtualized network and I/O drivers.
From Oracle Linux and Red Hat Enterprise Linux 4.7 onwards, the stock kernels provide paravirtualized network and I/O drivers for
hardware virtualized guests. From Solaris 10 10/09 onwards, the stock kernels provide paravirtualized network and I/O drivers for
hardware virtualized machines. Windows does not have native paravirtualization support, although Windows virtual machines can run
as hardware virtualized machines using Oracle's paravirtualized network and I/O drivers. Oracle has released paravirtualized network
and I/O drivers for the Windows operating system that can be freely downloaded from the Oracle Linux eDelivery portal.
In hardware virtualization mode, the hypervisor, the Intel or AMD virtualization technologies, and dom0 with QEMU work in concert
to emulate hardware for hardware virtualized machines. Hardware virtualization mode has a higher overhead than paravirtualization
mode, due in part to the overhead of emulating hardware for hardware virtualized machines.
Table 1 shows the guest operating system support matrix for Oracle VM for x86. 2.2.x
Guest Hardware Hardware

Paravirtualized Paravirtualized
Operating Virtualized Virtualized
32-bit 64-bit
System 32-bit 64-bit
*Oracle Linux
* * * *
6.x
Oracle Linux
* * * *
5.x
4 of 243 18.04.2011 16:35

Oracle Linux
* * * *
4.x
*Red Hat
Enterprise * * * *
Linux 6.x
Red Hat
Enterprise * * * *
Linux 5.x
Red Hat
Enterprise * * * *
Linux 4.x
Red Hat
Enterprise * *
Linux 3.x
Microsoft
* *
Windows 2000
Microsoft
* *
Windows 2003
Microsoft
Windows XP * *
Pro
Microsoft
* *
Windows Vista
Microsoft
* *
Windows 7
Microsoft
Windows 2008 * *
SP1
Microsoft
Windows 2008 *
R2
Oracle Solaris
* *
10 10/09+
Oracle Solaris
* *
11 Express
* Oracle VM 2.2.x does not have native ext4 support, therefore Oracle Linux 6 and Red Hat Enterprise Linux 6 virtual machines must
use ext3 for the /boot partition to be able to boot.
Oracle VM for x86 Packaging and Pricing

Oracle VM for x86 is not a licensed Oracle technology product. Support for Oracle VM for x86 is available as an add-on component of
Oracle’s enterprise support package. Support for Oracle VM for x86 is available with Sun x86 hardware as an add-on Premier Support
for Systems package. Oracle VM for x86 support for third-party hardware is sold as an add-on support component of Oracle’s
enterprise support package.
Premier Support for Systems costs 12% of the net Sun system purchase price and includes comprehensive support for the system
hardware and firmware, as well as operating system support for Solaris x86, Solaris 11 Express, Oracle Linux, and Oracle VM for x86.
Along with hardware and firmware support, Premier Support for Systems includes operating system support for one or more virtual
instances Oracle Linux, Solaris 10 x86, and Solaris 11 Express x86 running on Oracle VM for x86.
Support for Oracle VM for x86 for third-party hardware is sold in two packages: Oracle VM Premier Limited support for servers with
up to two sockets and Oracle VM Premier support for servers with more than two sockets. Support for Oracle VM for x86 for
third-party hardware only includes support for Oracle VM for x86. Operating System support for Oracle Linux, Red Hat Enterprise
Linux, Solaris x86, Solaris 11 Express, and Windows virtual machines running on Oracle VM for x86 must be purchased separately.
The next section will examine how to accelerate application deployments using the Oracle VM for x86 product line and add-on
products.
Accelerating Application Deployments with Oracle VM for x86

As mentioned in the Oracle VM for x86 introduction, the Oracle VM for x86 product line includes comprehensive virtual machine and
application packaging and deployment options that accelerate application deployments by eliminating time-consuming operating
5 of 243 18.04.2011 16:35

system and application installations. Oracle offers five unique options for accelerating application deployments using the Oracle VM
for x86 product line:
List 1: Five options for accelerating application deployments using Oracle VM for x86.
1. Virtual Machine Cloning

2. Oracle VM Templates
3. Oracle VM Template Builder
4. Oracle Virtual Assembly Builder
5. Oracle JRockit Virtual Edition
The next sections will examine virtual machine cloning, Oracle VM Templates, Oracle VM Template Builder, Oracle Virtual Assembly
Builder, and will conclude with Oracle JRockit Virtual Edition.
Virtual Machine Cloning

Operating system and application installations can be very time consuming. Virtual machine cloning using Oracle VM Manager can
help accelerate application deployments by eliminating the entire operating system installation and configuration process.
Applications that can tolerate host name and IP changes can be included within clones and can be rapidly redeployed an unlimited
number of times as copies of the original virtual machine.
Virtual machine cloning accelerates application development and testing by reducing the time required to deploy a clean operating
system environment for application testing. For example, using Oracle VM Manager, a virtual machine can be saved as a clone or
saved as a template and then the clone or the template can be rapidly redeployed an unlimited number of times as copies of the
original virtual machine or as unique new virtual machines.
Note: The minimum requirement to create and Oracle VM templates is an Oracle VM server managed by Oracle VM Manager.
Virtual machine cloning is an Oracle VM Manager image management feature. The Oracle VM Manager clone and save-as-template
features offer sparse and nonsparse virtual machine image provisioning. The term “sparse” is commonly referred to as “thin
provisioning,” in contrast to nonsparse, which is referred to as “thick provisioning”.
A thin-provisioned clone will not write the parent virtual machine's zeroed blocks to disk, whereas a thick clone will copy all of the
parent virtual machine's blocks to disk. Thin-provisioned disks grow proportionally to the number of writes to the disk by the virtual
machine, so that large portions of the unused disk do not consume space.
As of this writing, both thin and thick clone provisioning are off-line operations, meaning that the parent virtual machine must be
powered off to create a clone. The advantage of selecting thin provisioning for clones is that the storage is allocated only when
needed, which reduces the time it takes to create clones in addition to saving disk space. The disadvantage of using thin provisioning
is that the file system free-space reports may be misleading. For example, since thin-provisioned disks are allocated only when
needed, the file system free space reports may not be accurate since large portions of unused disk, that is, the zeroed blocks, have not
yet been written to disk.
Tip: Some applications do not support copying sparse files and may copy the entire uncompressed size of the file including the sparse
sections.
For more information on virtual machine cloning please refer to Chapter 5. For more information about Oracle VM Storage, please
refer to Chapter 6.
Oracle VM Templates
Operating system and Oracle application installations can be very time consuming. The Oracle VM for x86 Template program was
created to accelerate Oracle application deployments by offering self-contained, preconfigured virtual machines with key Oracle
technologies. Oracle VM templates can be downloaded from the Oracle Linux eDelivery portal and rapidly deployed an unlimited
number of times using Oracle VM for x86. The minimum requirement to run Oracle VM Template Builder and Oracle VM templates is
an x86-64 server with Oracle VM for x86, Oracle VM Manager is optional although highly recommended.
List 2: The steps to download and deploy an Oracle VM template.

Select and Download the desired Oracle VM template from the Oracle Linux eDelivery portal.
1. Copy the Oracle VM template zip file to your Oracle VM server's /OVS/running_pool directory.
2. Unzip and untar the files in the /OVS/running_pool directory.
3. Import the Oracle VM Template using Oracle VM Manager.
4. Start the Oracle VM template using Oracle VM Manager.
5. Access the Oracle VM template's console and select the initial boot-time options.
An Oracle VM template can consist of one or more virtual machines containing a preconfigured Oracle operating system and Oracle
technology products. Using Oracle VM templates eliminates the operating systems and Oracle application installation and
configuration process, which dramatically shortens application deployment timelines. There is no other vendor in the x86
virtualization market that can offer pre-packaging of production ready enterprise applications.
6 of 243 18.04.2011 16:35

List 3: Oracle’s Oracle VM templates collection as of this writing.
Virtualization and Management
Oracle VM Manager 2.2 and 2.1.5

Oracle VM Template Builder 2.1
Oracle VM Server 2.2.1 and 2.1.5 SDK
Sun Ray Software 5
Oracle Secure Global Desktop
Oracle Enterprise Manager Grid Control 10g R5 (10.2.0.5)
Applications
E-Business Suite 12.1.3

E-Business Suite 12.X Sparse Middle Tier
JD Edwards EnterpriseOne 9.0 Update 1 with ESUs and JD Edwards EnterpriseOne Tools 8.98 Update 3
JD Edwards EnterpriseOne 9.0 Update 1 and JD Edwards EnterpriseOneTools 8.98 Update 2
PeopleSoft ELM 9.1 Bundle #2 with PeopleTools 8.50.09
PeopleSoft FSCM 9.1 Bundle #4 (includes Maintenance Pack 2) with PeopleTools 8.50.10
PeopleSoft CRM 9.1 Bundle #2 with PeopleTools 8.50.09
PeopleSoft Portal Solutions 9.1 and PeopleTools 8.50.09
PeopleSoft HCM 9.1 and PeopleTools 8.50.02
Siebel CRM SIA 8.1.1
Siebel CRM SIA 8.0
Middleware
Oracle WebLogic Server on JRockit Virtual Edition 11g R1 (10.3.2)

Oracle WebLogic Server 10g Release 3
Oracle Business Intelligence Enterprise Edition 10.1.3.4
Oracle Application Server 10g Release 3 WebCenter
Oracle Identity Management 10g Release 2
Oracle Fusion Middleware Service Oriented Architecture (SOA) 10.1.3.4 and 10.1.3.3
Database & Real Application Clusters (RAC)
Oracle Real Application Clusters (RAC) 11g Release 2

Oracle Database 11g and Oracle Database 10g Release 2
Operating System
Oracle Linux 4 and 5

Oracle Solaris 10
Third-party Software
Zeus Traffic Manager
Oracle VM Templates Packaging and Pricing

Oracle VM templates can be downloaded from the Oracle Linux eDelivery portal.
An Oracle VM template license includes a free download and free trial use of the Oracle technologies with the option to purchase a
product license. Oracle VM templates do not have time limits or feature limitations. Oracle VM templates can be quickly transitioned
from evaluations into test, development. or production by purchasing the Oracle technology product license.
Oracle frequently updates Oracle VM templates and adds new Oracle VM templates to the Oracle Linux eDelivery portal. The best
way to stay up to date about Oracle VM templates is to visit ITNewsCast's Oracle VM section.
For information on how to get and deploy Oracle VM templates, please refer to Chapter 8 and Chapter 9.
Oracle VM Template Builder

Oracle VM Template Builder is a free add-on product for Oracle VM for x86 that packages Oracle and non-Oracle applications on
7 of 243 18.04.2011 16:35

Oracle Linux JeOS (just enough operating system) as a preconfigured, reusable Oracle VM template. Oracle Linux JeOS is a
prepackaged, customizable small-footprint x86 and x86-64 Oracle Linux operating system.
Deploying preconfigured Oracle VM templates eliminates the entire operating system and application installation and configuration
process. Oracle VM Template Builder dramatically shortens application deployments by packaging unique application configurations
within preconfigured Oracle VM templates that can be rapidly deployed one or more times using Oracle VM for x86. The minimum
requirement to run Oracle VM Template Builder and Oracle VM templates is an x86-64 server with Oracle VM for x86, Oracle VM
Manager is optional although highly recommended.
Note: Oracle VM Template Builder is the application Oracle uses to create Oracle VM templates.
Oracle VM Template Builder templates are designed to be configured at the initial boot time to ensure that each template has unique
operating system and application configurations. The initial boot time configuration selections can be default values and/or actions
based on the user’s input.
Note: Oracle Linux JeOS is shipped with only English language support. Additional language packages for JeOS are available from
Oracle Linux eDelivery portal.
Oracle VM templates can be built and used by anyone including Oracle, ISVs, Oracle Partners, VARs, and end users. JeOS Oracle VM
templates can be redistributed without an agreement from Oracle. Redistributing templates with applications may or may not be
allowed. It is necessary to consult with the ISV to determine if their application can be legally redistributed within an Oracle Linux
JeOS template.
Packaging software applications with Oracle VM Template Builder requires a thorough understanding of the software application's
behavior. This particularly applies to the Oracle VM JeOS template reconfiguration and cleanup scripts used in the Oracle VM JeOS
template.
Creating Oracle VM JeOS templates is a multistep process. The number of steps to create an Oracle VM template depends on the type
and complexity of the application software added to the Oracle VM JeOS template.
Oracle VM Builder Packaging and Pricing

Oracle VM Template Builder is distributed as an RPM package and as a Oracle VM template from the Linux eDelivery portal. Oracle
Linux JeOS is distributed with Oracle VM Template Builder and as RPMs from the Linux eDelivery portal.
Oracle VM Template Builder and Oracle's JeOS are open source, free to download, free to use, and free to redistribute.
Oracle Virtual Assembly Builder

Oracle Virtual Assembly Builder is an add-on product for Oracle VM for x86 that can profile and package reference physical or virtual
multitier Oracle Fusion Middleware environments as prepackaged Oracle VM templates, which are called assemblies. Assemblies
contain preconfigured JeOS virtual machines with configurable initial boot-time metadata that allow each assembly to have unique
operating system and application configurations. The initial boot-time configuration selections can be default values and/or actions
based on the user’s input.
Note: Introspection is Oracle's terminology for application profiling.
Assemblies are created and managed using the Oracle Virtual Assembly Builder Studio, which is a standalone administrative
application. Oracle Virtual Assembly Builder Studio integrates with Oracle VM for x86 using the Oracle VM Manager API. The
minimum requirements for Oracle Virtual Assembly Builder are a) Oracle Virtual Assembly Builder Studio, b) Oracle VM for x86
server pool, c) Oracle VM Manager, and d) a physical and/or virtual reference Oracle Fusion Middleware environment to introspect.
The current release of Oracle Virtual Assembly Builder is 11g Release 1 (11.1.1). Oracle Virtual Assembly Builder 11g Release 1
works exclusively with Oracle VM for x86 2.x. Oracle describes the this release as a developer-centric release, as opposed to an
enterprise release, due to the lack of enterprise security features, such as root access to files and directories and the absence of
role-based access. The next release of Oracle Virtual Assembly Builder should be an enterprise release with the enterprise security
features, broader Oracle Fusions Middleware application support, and Oracle VM 3.0 integration.
Oracle Virtual Assembly Builder Packaging and Pricing

Oracle Virtual Assembly Builder is distributed from the Oracle Technology Network (OTN). Oracle Virtual Assembly Builder is
supported on 32-bit version of Oracle Linux and Red Hat Enterprise Linux 5.
Oracle Virtual Assembly Builder is not a standalone Oracle technology product that can be purchased a la carte. The right to use
Oracle Virtual Assembly Builder is bundled with WebLogic Enterprise Edition. The only way to purchase Oracle Virtual Assembly
Builder is by purchasing WebLogic Enterprise Edition licenses.
Note: I intend to add a chapter about the next enterprise release of Oracle Virtual Assembly Builder. Please stay tuned to
ITNewsCast.com and the underground Oracle VM Manual for the new Oracle Virtual Assembly Builder chapter.
8 of 243 18.04.2011 16:35

Oracle JRockit Virtual Edition

Oracle JRockit Virtual Edition is an add-on product for Oracle VM for x86 that is packaged as a virtual appliance that runs exclusively
on Oracle VM for x86. Oracle JRockit Virtual Edition appliances contain a full featured implementation of Oracle WebLogic Server 11g
that runs directly on the hypervisor with the help of a thin OS layer with networking, hardware device interaction, I/O, and process
scheduling. Oracle JRockit Virtual Edition is a single-user, single-process system that is designed to run any JDK 6 compatible
application with full support for domains and clusters. With Oracle JRockit Virtual Edition, mission critical Java applications can be
quickly deployed or migrated from physical or third-party virtual environments to Oracle JRockit Virtual Edition virtual appliances
with unparalleled density, performance, and ease of management.
Figure 2 shows an Oracle JRockit Virtual Edition appliance, highlighting the JRockit Virtual Edition OS layer.
The Oracle JRockit Virtual Edition OS layer contains a purpose-built kernel that contains only the essential services to run Oracle
WebLogic Server 11g with JDK 6 compatible applications. Oracle WebLogic Server 11g on Oracle JRockit Virtual Edition offers up to
30% better performance for Java applications compared to WebLogic deployments using a traditional operating system, and has a
footprint up to five hundred times smaller than that of WebLogic deployments using a traditional operating system. Without a
traditional operating system, JRockit Virtual Edition eliminates the operating system installation, management, security, and storage
requirements as well as operating system license and support costs.
To illustrate just how thin the Oracle JRockit Virtual Edition OS layer is, Table 2 compares a Linux operating system to JRockit Virtual
Edition.
Numbers Linux JRockit Virtual

Edition
Administration +-500 1
Utilities
Administrative +-3000 10
Commands
Command Parameters +-50000 100
Disk Image Size +-10 GB 1 GB
Oracle JRockit Virtual Edition uses the same administrative infrastructure, tools, and WLST scripts as a traditional WebLogic
deployment. Each Oracle JRockit Virtual Edition appliance ships with an administrative server equipped with WebLogic Server
Administration Console. Oracle JRockit Virtual Edition appliances support both off-line and on-line management of applications
running in WebLogic instances.
Oracle JRockit Virtual Edition is intended to be a deployment platform. Oracle recommends that Java application development be done
on traditional WebLogic platforms. Once the application is ready to be deployed into production, it can be moved from the WebLogic
development system to an Oracle JRockit Virtual Edition appliance.
Oracle JRockit Virtual Edition Packaging and Pricing

Oracle JRockit Virtual Edition is distributed as a virtual appliance from the Oracle eDelivery portal.
Oracle JRockit Virtual Edition is not a standalone Oracle technology product that can be purchased a la cart. The right to use Oracle
JRockit Virtual Edition is bundled with WebLogic Enterprise Edition. The only way to purchase Oracle JRockit Virtual Edition is by
purchasing WebLogic Enterprise Edition licenses.
The Oracle VM Management Pack Plug-in

The Oracle VM Management Pack Plug-in adds Oracle VM Manager functionality to Oracle's system management solution, Oracle
Enterprise Manager (OEM). The Oracle VM Management Pack Plug-in is a complete port of Oracle VM Manager to Oracle Enterprise
Manager 10g and 11g. Along with the Oracle VM Manager feature set, the Oracle VM Management Pack Plug-in offers advanced
monitoring, provisioning, patch management, and configuration management. Oracle VM servers can only be managed from one or
9 of 243 18.04.2011 16:35

the other of the two management options, Oracle VM Manager or the Oracle VM Management Pack Plug-in, not both.
Note: The Oracle VM Management Pack is licensed software; Oracle VM Server and Oracle VM Manager are not licensed software.
Figure 3 shows the Oracle Enterprise Manager administrative console.
It's difficult to compare a full-featured system management solution like Oracle Enterprise Manager with a single-product
management solution like Oracle VM Manager, although for completeness Table 3 contrasts the high-level features of the Oracle
Enterprise Manager with those of the Oracle VM Management Pack Plug-in and Oracle VM Manager.
Capability Oracle VM OEM /

Manager Oracle VM
Manager
Plug-in
Oracle VM server, Oracle VM *

pool, virtual machine, and
virtual machine resources
management, quality
monitoring and provisioning.
*Oracle Linux, Red Hat *

Enterprise Linux, Solaris and
Oracle VM server provisioning
and patch management.
Database and application *

administration, quality
management and provisioning.
Configuration Management, *
i.e. tracking of configuration
drifts.
**Parse Oracle VM server and *

virtual machine operating
system configuration files to
track changes/differences and
send alerts/notification on
when changes are made.
Scriptable command Line * *

Interface.
* Oracle Enterprise Manager does not support Oracle VM server patch management.
** The Application Change Console.
The Application Change Console is a separate standalone application that is licensed with the Oracle VM Management Pack Plug-in.
The Application Change Console (ACC) is able to parse configuration files to track changes and differences and send alerts and
notifications when changes are made. The Application Change Console has prebuilt parsers for Operating System configuration files.
10 of 243 18.04.2011 16:35

After installing the Application Change Console, you can point the Application Change Console to the Oracle VM servers and Oracle
VM guests using hostnames. The Application Change Console will connect to a host via SSH to gather and track configurations. No
additional agents are required. All of the Application Change Console components can run on Oracle Linux on Oracle VM.
Oracle's x86 Virtualization Support Policy

Since the first release of Oracle VM for x86, which was announced at Oracle Open World in November 2007, Oracle has provided
certified support for Oracle technology products on Oracle VM for x86. Oracle VM for x86 is the only x86 virtualization solution that
Oracle tests and certifies and is the only one supported by Oracle for Oracle technology products. Oracle does not test, certify, or
support Oracle technology products on any third-party x86 virtualization solutions. For example, Oracle does not certify or test Oracle
technology products on VMware ESX, Microsoft Hyper-V, Citrix Xen Server, Red Hat with Xen, or KVM or on any Xen or KVM-enabled
Linux system. Oracle maintains an up-to-date list of Oracle certified software on Oracle VM via Metalink Doc ID: 464754.1.
Oracle’s support policy, regarding Oracle technologies and virtualization, can be referenced via MetaLink documents 794016.1 and
249212.1. In short, MetaLink documents 794016.1 and 249212.1 explain that service requests (SRs) involving uncertified
virtualization solutions will receive best effort for known issues on physical systems. Oracle may request that the customer reproduce
the problem on native hardware if the third-party virtualization software cannot be ruled out as the root cause of the problem.
Major ISVs, for example, Microsoft and IBM, have virtualization support policies similar to Oracle’s. Microsoft and IBM may request
that customers reproduce problems on native hardware if the third-party virtualization software cannot be ruled out as the root cause
of a problem.
Microsoft: "As part of the investigation, Microsoft may require the issue to be reproduced by the customer independently from the
non-Microsoft hardware virtualization software."
IBM: “Will IBM correct all defects for IBM software products that are running in a supported virtualization environment?
A: Not necessarily. As with other operating environments, IBM does not warrant that all code defects will be corrected. IBM will issue
defect correction information, a restriction, or a bypass to IBM products if the defect is applicable to a supported virtualized
environment or to a native physical machine environment (that is, without the virtualization software).”
Watch this space
Chapter 2: Oracle VM for x86 Reference Design and

Architectural Introduction
The chapter starts with an architectural review of Oracle VM for x86 and ends with an introduction to the Oracle VM for x86
reference design. The goal of this chapter is to articulate the architectural and design considerations for Oracle VM for x86 in order
to provide the technical foundation for using the Oracle VM for x86 reference design.
Note: This chapter replaces the original “Chapter 2: Oracle VM Architectural Review” that is still available as an Oracle VM for x86
architectural reference.
Table of Contents
Change Log
Oracle VM Server and Xen Architectural Review
...Oracle VM for x86 Server Specifications
Oracle VM Manager Architectural Review
Oracle VM Agent
Oracle VM for x86 Networking
Oracle VM for x86 Server Pools, HA and Live Migration
Oracle VM for x86 Storage
Oracle VM for x86 Intra-component Communication & Firewall Requirements
1.0 – The Oracle VM for x86 System Reference Design Introduction
...1.1 – The Oracle VM for x86 System Reference Design Implementation Overview
...1.2 – The Oracle VM for x86 System Reference Design Document Overview
2.0 – The Oracle VM for x86 System Reference Design Architectural Overview
...2.1 - The Oracle VM for x86 System Reference Design Support Infrastructure
Part 1 - The Oracle VM for x86 System Reference Design
3.0 - Cloud Infrastructure Architecture
11 of 243 18.04.2011 16:35

...3.1 - Oracle VM for x86 Hardware Architecture

...3.2 - Oracle VM for x86 Server Pool Configurations
...3.3 - Oracle VM for x86 Network Topology
...3.4 - Oracle VM for x86 Network Security Architecture
...3.5 - Oracle VM for x86 Administration and Monitoring
...3.6 - Oracle VM for x86 Virtual Machine Operating Systems
...3.7 - Oracle VM for x86 Application Integration
...3.8 - Oracle VM for x86 Support Service
Change Log
Revision Change Updated Date

Description By
1 New chapter Roddy 04/03/11

Rodstein
1 3.4 – Oracle VM Roddy 04/11/11

for x86 Network Rodstein
Security
Architecture -
Added "SSH
login banners"
1 3.5 - Oracle VM Roddy 04/11/11

for x86 Rodstein
Administration
and Monitoring -
Added "Rootkit
prevention and
monitoring"
Oracle VM Server and Xen Architectural Review

Oracle VM for x86 consists of an x86 server component and a manager component that is used to manage one or more clustered
servers. The server component is a type 1 hypervisor based on the Xen.org open source hypervisor, which is called the Oracle VM
server. Xen is considered a type 1 hypervisor because Xen installs directly on hardware, whereas a type 2 hypervisor, such as Oracle
VM VirtualBox, is a software applications installed on top of a desktop operating system. The Xen hypervisor is responsible for
monitoring virtual machine operating systems and for scheduling virtual machine operating system requests to the physical server's
CPU and memory. Xen is not responsible for and has no knowledge of networking, external storage, video, or common I/O functions.
The Xen hypervisor works in concert with a control domain, which, in Xen terminology, is called dom0 or domain 0. Xen and dom0 are
automatically started when an Oracle VM server boots. Dom0’s role includes providing an interface to Xen, hardware detection,
device support, domU management, and presenting domUs with networking, storage, video, and I/O interfaces. Xen and dom0 enable
multiple concurrently running virtual machines to share a single piece of hardware. In Xen terminology, unprivileged domains are
called domU or domain U; the industry term for domU is virtual machine. Dom0 and domUs are all virtual machines.
Oracle makes subtle changes to the Xen.org code that create a unique Xen distribution, which is redistributed as Oracle VM server.
Oracle has developed a Xen/dom0 configuration for Oracle VM that supports the most demanding high I/O workloads that other Linux
distributions with Xen do not use. For example, each Linux distribution ships with Xen as an operating system virtualization feature.
When Xen is enabled on a Linux system, the entire Linux operating system, including all of the user space applications, is turned into
dom0. Xen-enabled Linux systems do not have an optimized Xen or dom0 configuration. In contrast to Xen/Linux, Oracle VM server is
a deliberately built virtualization platform, specifically designed for high I/O workloads such as Oracle's Database and Oracle Fusion
Middleware workloads.
Oracle VM server supports two unique virtualization modes, paravirtualization mode (PV mode) and hardware virtualization mode
(HVM mode). Oracle VM servers can support both paravirtualization mode and hardware virtualization mode simultaneously on a
single x86-64 server that has either Intel or AMD virtualization technologies. Intel and AMD virtualization is a requirement only for
hardware virtualization mode, not for paravirtualization mode. Intel and AMD virtualization technologies are enabled and managed
using the system BIOS.
Paravirtualization mode requires the virtual machine operating system to run a Xen kernel and Xen network and I/O drivers. Xen
paravirtualized guest kernels are available for the Oracle Linux and Red Hat Enterprise Linux operating systems. Paravirtualized
virtual machines are hypervisor aware and run without the additional overhead of hardware emulation. Paravirtualization requires
much less overhead for timers, interrupts, I/O traffic, and context switches, allowing superior scalability under heavy loads, when
compared to hardware virtualization mode.
Unlike paravirtualization mode, which requires the virtual machine to run a Xen kernel, hardware virtualization mode supports
unmodified operating systems. Virtual machines that run under hardware virtualization mode are called “hardware virtualized
machines” (HVM). Hardware virtualized machines are unaware that they have been virtualized and think they are on physical
hardware. To provide acceptable performance, hardware virtualized machines should use paravirtualized network and I/O drivers.
12 of 243 18.04.2011 16:35

From Oracle Linux and Red Hat Enterprise Linux 4.7 onwards, the stock kernels have provided paravirtualized network and I/O
drivers for hardware virtualized guests. From Solaris 10 10/09 onwards, the stock kernels have provided paravirtualized network and
I/O drivers for hardware virtualized machines. Windows does not have native paravirtualization support, although Windows virtual
machines can run as hardware virtualized machines using Oracle's paravirtualized network and I/O drivers. Oracle has released
paravirtualized network and I/O drivers for the Windows operating system that can be freely downloaded from the Oracle Linux
eDelivery portal.
Paravirtualization and hardware virtualization modes use very different techniques to provide resources to virtual machines. For
example, hardware virtualization mode uses Intel or AMD virtualization technologies for memory management and to emulate the
boot environment for hardware virtualized machines. Hardware virtualization mode also uses QEMU in dom0 for device emulation for
hardware virtualized machines. Paravirtualization mode leverages the guest operating system's Xen kernel for the boot process using
the pygrub bootloader, Xen for memory management, and dom0 without QEMU for device support.
With paravirtualization mode, dom0 multiplexes native Linux devices for paravirtualized domUs. dom0 runs the back-end drivers, and
domU runs the paravirtualized front-end drivers in a back-end dom0, front-end domU driver model. For networking and I/O, dom0
runs a network back-end driver and a block back-end driver to support network and I/O requests for paravirtualized domUs. The
network back-end driver communicates through dom0 to the local hardware device to process all domU networking requests. The
block back-end driver communicates through dom0 to the local storage to process all domU storage requests.
Dom0 runs a dedicated quemu-dm process for each hardware virtualized machine. Hardware virtualized machines do not use the
same dom0 back-end and domU front-end drivers as paravirtualized domUs. Hardware virtualized machines use emulated drivers that
are created and managed by the quemu-dm processes in dom0. Hardware virtualization mode has a higher overhead than
paravirtualization mode, due in part to the CPU overhead of emulating hardware for hardware virtualized machines.
Figure 1 illustrates the Xen architecture explained above, highlighting the hypervisor, dom0, QEMU, two paravirtualized domUs, one
hardware virtualized domU, and the direct and virtual I/O paths.
Tip: The only way to determine which virtualization mode will provide the best performance for your environment is to benchmark
the same workload using the same operating system in paravirtualization mode and in hardware virtualization mode. If you do not
have the time or expertise to conduct the benchmarks, consider only using paravirtualization mode for your virtual machines.
To better understand the capabilities of Oracle VM server, List 1 highlights the technical specifications of Oracle VM server.
Oracle VM for x86 Server Specifications

Supported Platforms
x86 or x86_64 processors with PAE (Physical Address Extension) support
Minimum Processor Class
I686 class
Minimum Memory
1GB
Maximum Memory
1TB
CPUs Supporting Paravirtualization Mode
Intel Pentium-PRO or newer

AMD Athlon/Duron or newer
13 of 243 18.04.2011 16:35

At least a Pentium IV or Athlon CPU recommended
CPUs Supporting Hardware Virtualization Mode
Some Intel Pentium D, Core, Core2 and Xeon models (/proc/cpuinfo should list "vmx" among the flags)
Some AMD Athlon and Opteron models (/proc/cpuinfo should show "svm" among the flags)
Maximum Number of CPUs or Threads
128 CPUs or 128 threads
Boot Options
Local disk, SAN, iSCSI, NFS, flash
Oracle VM for x86 Virtual Machine Specifications

Maximum Number of Virtual CPUs per Virtual Machine
32 virtual CPUs
Maximum Amount of Memory per Virtual Machine
x86 (32-bit): 63GB

x86_64 (64-bit): 510GB
Maximum Number of Virtual Disks per Virtual Machine
Paravirtualized Mode: 20 hda, 26 sda, 5 xvd

Hardware Virtualization Mode: 4 hda, 7 sda, 5 xvd
*Each Oracle VM Server can support a maximum of 128 virtual disks
Maximum Number of Virtual NICs per Virtual Machine
Paravirtualized Mode: 31
Hardware Virtualization Mode: 8
List 1 shows that an Oracle VM server supports a total of a 1TB of RAM and a total of 128 CPU threads. An Oracle VM server with a
1TB of RAM and 128 CPU threads could allocate the majority of the 1TB of RAM and more than 128 CPU threads to virtual machines.
Oracle VM server supports CPU oversubscription, which means that an Oracle VM server with 128 CPU threads can overallocate the
total number of CPU threads to virtual machines. Oracle VM server does not support memory oversubscription, which means that an
Oracle VM server with 1TB of RAM cannot overallocate RAM. By default, each Oracle VM server reserves 512MB of memory for
dom0. The average memory overhead for each running guest on a dom0 is approximately 20MB plus 1% of the guest’s memory size.
The remaining physical memory can be allocated to guests.
Avoid oversubscribing CPU-bound workloads such as the Oracle Database. CPU oversubscription with CPU-bound workloads
negatively effects performance and availability. CPU oversubscription for non-CPU-bound workloads, such as Oracle Fusion
Middleware products, is recommended. It is common to oversubscribe CPU cores 3-to-1 with non-CPU-bound workloads. For example,
each CPU core could allocate 3 virtual CPUs for non-CPU-bound workloads without a performance penalty.
The maximum amount of RAM and CPU cores that an Oracle VM server with a 1TB or RAM and 128 CPU cores could allocate to a
single virtual machine is 32 virtual CPUs with 510GB of RAM. The minimum amount of RAM and CPU cores that an Oracle VM server
with a 1TB or RAM and 128 CPU threads could allocate to a single virtual machine is 1 virtual CPUs with as little as 256MB of RAM.
Note: A virtual machine cannot aggregate CPU and RAM resources from more than one server. That is, virtual machines consume
resources only from the host where the virtual machine was started.
Oracle VM for x86 consists of an x86 server component and a manager component used to manage one or more clustered servers.
Oracle VM server enables multiple concurrently running virtual machines to share a single piece of x86 hardware. The manager
component is a traditional Oracle application, named Oracle VM Manager. Oracle VM Manager facilitates centralized management of
one or more clustered Oracle VM servers, virtual machines, and virtual machine resources.
The next section provides an architectural review of Oracle VM Manager.
Oracle VM Manager Architectural Review

Oracle VM Manager is the default “no cost management component” for Oracle VM for x86. Oracle VM Manager is a traditional
Oracle application that installs on Oracle Linux and Red Hat Enterprise Linux. It is used to manage one or more clustered Oracle VM
servers, virtual machines, and virtual machine resources. Oracle VM Manager consists of an Oracle database, an Oracle application
server, and a J2EE application with an OS and browser neutral Oracle Application Development Framework (ADF) administrative
portal. Oracle VM Manager also has a command line interface that allows Oracle VM Manager administrative tasks to be performed
from the command line or to be executed as scripts.
Oracle VM Manager is distributed from the Oracle Linux eDelivery portal as an ISO file and as a preconfigured, production-ready
14 of 243 18.04.2011 16:35

Oracle VM template. Oracle VM Manager can be installed in an all-in-one configuration using the default Oracle 10g Express
Database or in a more traditional two tier architecture with an OC4J web tier and a 10 or 11g database tier. The Oracle VM Manager
template is an example of an all-in-one installation with the Oracle 10g Express Database, an OC4J application server, and the Oracle
VM Manager application all installed on the same operating system.
Table 1 lists the packaged applications in the Oracle VM Manager ISO file.
Application Capability
Oracle Database The Oracle Database 10g Express Edition is

10g Express packaged in the Oracle VM ISO.
Edition
Oracle VM The Oracle VM Manager package contains

Manager package the Oracle VM Manager web application.
The Oracle VM Manager web application
runs on Oracle Application Server 10g
Release 3 (10.1.3) and is deployed into an
OC4J container.
Oracle Oracle Containers for J2EE (OC4J) is the

Containers for J2EE runtime environment for the Oracle
J2EE (OC4J) Application Server. Oracle Application
Standalone 10.1.3 Development Framework (ADF) is a Java EE
packaged with framework.
Application
Development
Framework (ADF)
10.1.3.3
XML-RPC 3.0 The XML-RPC 3.0 package is installed

during an Oracle VM Manager installation.
XML-RPC is used for Oracle VM Manager to
Oracle VM agent communication.
The total number of Oracle VM servers that Oracle VM Manager can manage is limited by the type of database repository used by
Oracle VM Manager. The default Oracle Database 10g Express Edition has a 4GB limit for on-disk storage, which is a bottleneck in
supporting Oracle VM environments with more than 50 Oracle VM servers. Using Oracle Standard Edition or Oracle Enterprise
Edition eliminates the 4GB limit for on-disk storage. For example, if your Oracle VM Manager database repository is not using Oracle
Database 10g Express Edition but an Oracle Standard or Enterprise Edition database on a reasonably sized server with a gigE
networking, Oracle VM Manager could easily scale up to 1000+ Oracle VM servers with thousands of virtual machines.
Tip: The Oracle VM Manager application runs much faster using an Oracle Standard or Enterprise Edition database.
The Oracle VM Manager application is a great candidate for virtualization on Oracle VM and Oracle VM VirtualBox. Virtualizing
Oracle VM Manager saves on hardware costs and improves application flexibility while reducing data center space. The Oracle VM
Manager template can be used to quickly deploy a production all-in-one Oracle VM Manager installation as a virtual machine, without
the need to dedicate a server for Oracle VM Manager. For a distributed Oracle VM Manager installation, the web tier and the
database tier can both be virtualized on Oracle VM using Oracle Linux virtual machines.
Note: Oracle VM Manager is not supported and should not be installed in an Oracle VM server's dom0.
Oracle VM Agent
Figure 2 illustrates the Xen architecture with the addition of the Oracle VM Manager. Oracle VM Manager is running on a virtual
machine with an all-in-one Oracle VM Manager installation, managing a server pool with one Oracle VM server hosting three virtual
machines.
15 of 243 18.04.2011 16:35

Oracle VM Manager dispatches administrative commands made in the Oracle VM Manager portal to the Oracle VM agent in dom0.
The Oracle VM agent executes the Oracle VM Manager commands in dom0 using python scripts located in the /opt/ovs-agent-latest/
directory.
Oracle VM Manager facilitates centralized management of server pools and their resources using an agent-based architecture. When
an Oracle VM server is added to a server pool, one or more Oracle VM agent roles are assigned to the Oracle VM server. There are a
total of three Oracle VM agent roles; 1) the Server Pool Master, 2) the Utility Server and 3) the Virtual Machine Server. When an
Oracle VM server is added to a new server pool, it can be assigned one, two, or all three of the agent roles.
List 2 explains each of the three Oracle VM agent roles.
Server Pool Master
The server pool master is the principal server pool role within a server pool. The server pool master is the only agent role that
communicates with Oracle VM Manager. The server pool master dispatches commands received from Oracle VM Manager to
other servers within a server pool. There can be only one server pool master in a server pool at any instant. The server pool
"Virtual IP" feature in Oracle VM Manager 2.2 and above will detect the loss of the server pool master and automatically failover
the pool master server role to the first pool member that can lock the cluster.
Tip: To eliminate a single point of failure, always enable the Virtual IP feature. The Virtual IP feature requires a unique IP
address. If you have enabled the Virtual IP feature and do not know which server is the server pool master, ssh to the Virtual IP
to validate which host is the server pool master.
Utility Server
The utility server role is responsible for I/O-intensive operations such as virtual machine creation and removal and server pool
creation and removal, as well for as copying and moving guest files. The server pool master dispatches operations to utility
server agents. There can be one or more utility server agents in a server pool. When there are multiple utility server agents in a
pool, the server pool master will select the least loaded utility server to conduct a task.
Tip: For production Oracle VM systems, use dedicated utility servers to isolate the impact of I/O intensive operations to utility
servers. For example, colocating the utility server agent with the virtual machine server agent will affect guest performance
during utility server operations.
Virtual Machine Server
Servers with the virtual machine server agent role are responsible for allocating CPU, memory, and disk resources to the virtual
machines in a server pool. There can be one or more virtual machine servers in a server pool.
Oracle VM Manager dispatches commands using XML RPC to each server pool master agent, which, in turn, dispatches commands to
other pool members over a dedicated management interface using XML RPC. Oracle VM agent architecture is exceptionally
bandwidth efficient, since the intracomponent traffic is limited between a) Oracle VM Manager and each server pool master agent and
b) each server pool master agent and its server pool members. For example, an Oracle VM environment with 20 server pools, each
server pool having 20 servers, would have a total of 20 communication channels between Oracle VM Manager and each server pool
master. If Oracle employed a direct Oracle VM Manager to Oracle VM agent relationships, the same Oracle VM environment with 20
server pools, each server pool having 20 servers, would have a total of 400 communication channels.
Oracle VM for x86 Networking

The default Oracle VM server networking configuration routes all dom0 and virtual machine traffic through a Xen bridge. A Xen
16 of 243 18.04.2011 16:35

bridge operates at layer 2 of the OSI model, effectively acting as a layer 2 (L2) switch passing packets to the egress port; it relies on
the TCP protocol for rate control and packet loss. Oracle VM's default network configuration pairs each network interface (NIC) with
a Xen bridge. An Oracle VM server with two NICs will have two Xen bridges, eth0/xenbr0 and eth1/xenbr1. The first Xen bridge,
eth0/xenbr0, is configured with an IP address on xenbr0 and is used for Oracle VM management traffic. The second Xen bridge will
not have an IP address assigned; it effectively acts as a layer 2 switch for guest traffic. The default Oracle VM server networking
configuration can be used as-is or modified to meet your business requirements, for example, to use 802.3AD NIC bonding with
802.1Q.
Figure 3 shows the default Oracle VM Xen bridge configuration: an Oracle VM server with two NICs wired into a single switch,
hosting three guests.
Tip: In an HA-enabled pool, the loss of network connectivity for the Oracle VM management interface causes an HA event. When an
HA event occurs, an Oracle VM server is fenced from the pool and reboots, then all HA-enabled guests are restarted on a live Oracle
VM pool member.
As of this writing, Oracle VM network configuration management is not included in Oracle VM Manager; it must therefore be
performed by hand in dom0. Both 802.3AD NIC bonding and 802.1Q VLANning are supported by Oracle VM for x86, although
bonding and VLANning must also be configured by hand in dom0.
Figure 4 shows an example 802.3AD NIC bonding and Xen bridges with 802.1Q VLANning.
For more information on Oracle VM networking please refer to Chapter 7: Oracle VM Networking from A to 802.1Q
Oracle VM Manager administers Oracle VM servers, virtual machines, and virtual machine resources in server pools. A server pool is
a management boundary that contains one or more clustered Oracle VM servers with virtual machines and virtual machine resources.
The next section will examine Oracle VM server pools.
Oracle VM for x86 Server Pools, HA, and Live Migration

Oracle VM Manager uses the concept of a server pool to group together and manage one or more clustered Oracle VM servers. An
Oracle VM server pool defines the management boundaries and the feature set of Oracle VM servers, virtual machines, and virtual
machine resources. Once a server pool is created, resources such as servers, virtual machines, operating system installation ISO files,
Oracle VM templates, administrative users, and groups can be configured and managed within the context of the server pool. For
example, an Oracle VM environment with multiple server pools could be managed from one single Oracle VM Manager instance,
although each server pool's resources, such as servers, virtual machines, operating system installation ISO files, Oracle VM
17 of 243 18.04.2011 16:35

templates, administrative users, and groups are isolated to their respected server pool.
An Oracle VM server with local storage is limited to a server pool with only one server, without high availability (HA) or Live
Migration functionally. To add additional Oracle VM Servers to a server pool, and to enable HA and Live Migration, at least one
shared SAN, iSCSI, or NFS storage repository is required. Shared SAN, iSCSI, or NFS storage allows virtual machines to start, run,
and migrate to any Oracle VM Server within a server pool. Without shared storage, virtual machines can only start and run on one
Oracle VM server without HA and Live Migration functionality.
Figure 5 shows an Oracle VM environment with four Oracle VM servers managed in two server pools.
Oracle VM for x86 Storage

Oracle VM uses two different types of storage repositories for server pools. The first type of storage repository, called a root
repository, is used to host a server pool's Oracle Cluster File System v2 (OCFS2) cluster configurations, HA configurations for HA
enabled pools and, optionally, virtual machine configuration files and images. There can only be one OCFS2 or NFS root repository
per server pool. The other type of storage repository, called an extended repository, is used exclusively for server pools with more
than one server, to host virtual machine configuration files and images. There can be one or more extended OCFS2 and/or NFS
repositories in a server pool with more than one server. When a server pool needs more storage capacity, additional extended OCFS2
and/or NFS repositories are added to a server pool to increase capacity.
Note: OCFS2 is not integrated or supported with any volume manager (LVM) solutions to manage the back-end block storage. Fibre
Channel and iSCSI OCFS2 partitions must be provisioned at static sizes, that is, partition sizes cannot change once a partition is
formatted with OCFS2.
As of this writing, there are no administrative features in Oracle VM for x86 for managing storage repositories. Oracle VM storage
repository configurations are made in dom0 and storage management is done using the native storage-array management tools.
Oracle VM Manager is responsible for pool creation, not for storage repository management. For example, storage repository
provisioning, storage repository snapshotting, storage repository replication, and storage repository monitoring, as well as storage
repository backup and restoration, are preformed using the storage-array administrative functionality.
Tip: A best practice with server pools that have more than one server is to dedicate the root repository to host only OCFS2 and HA
configurations, without virtual machine configuration files and images. A dedicated root repository without virtual machine
configuration files and images reduces the risk of cluster data corruption from virtual machine operations.
For more information on Oracle VM storage please refer to Chapter 6: Oracle VM 2.2 SAN, iSCSI and NFS Back-end Storage
Configurations
Figure 6 shows an Oracle VM server pool with three servers using a dedicated root repository hosting the cluster and HA
configurations and two extended repositories hosting virtual machine configuration files and images.
18 of 243 18.04.2011 16:35

The Oracle VM for x86 HA feature detects server failures within a server pool and responds by restarting the virtual machines from a
failed server on a live server in the server pool. Oracle VM leverages a lightly modified OCFS2 cluster stack to monitor the status of
each Oracle VM server, using a network heartbeat over the management interface and a storage heartbeat on the root storage
repository. If any node in a server pool fails to update/respond to its network or storage heartbeat, the node is fenced from the pool
and promptly reboots, then all HA-enabled guests are restarted on a live server in the pool.
Figure 7 shows a server pool with three servers. One of the three servers has failed and the virtual machines from the failed server
have been restarted on the live servers.
Oracle VM HA adds negligible traffic to the Oracle VM server management network: 1 packet every 2 seconds for the HA heartbeat
traffic plus distributed lock manager (DLM) traffic when locks are taken during guest start and stops. The HA heartbeat traffic is
latency sensitive, and the allowable latency for heartbeat traffic is 60 seconds. For example, network heartbeat latency is 30 seconds,
but after 30 seconds the cluster will attempt to establish a disk heartbeat. If the cluster establishes a disk heartbeat, it will then try to
reconnect via the network. The second disk heartbeat timeout latency is set to 30 seconds, so it takes a full 60 seconds plus the
inability to see the root storage repository to trigger an HA event.
Note: In an HA-enabled pool, the loss of network connectivity for the Oracle VM management interface causes an HA event. When an
VM pool member.
The Oracle VM for x86 Live Migration feature moves running virtual machines between server pool members across a LAN without
loss of availability. Live Migration has two primary use cases. The first use case is to eliminate planned downtime by migrating
running guests from one server pool member to another during planned maintenance events. The second use case is to migrate
running guests from an overutilized pool member to a pool member with available resources.
There are three requirements for Live Migration. The first requirement is a shared SAN, iSCSI, or NFS storage repository to host the
virtual machine configuration files and images. The second requirement is that the source and target pool member servers CPUs be
identical. The final requirement is that the target server pool member must have sufficient memory to accommodate the memory
requirements of the virtual machine or machines that will be migrated.
Figure 8 shows a server pool with virtual machines Live Migrating between server pool members.
19 of 243 18.04.2011 16:35

Oracle VM uses an iterative precopy method to migrate running guests over a TCP/SSL connection between two pool members. A Live
Migration event starts when the source server sends a migration request to the target server, which contains the guest resource
requirements. If the target accepts the migration request, the source starts the iterative precopy phase. The iterative precopy phase
starts by iteratively copying the guest’s memory pages from the source to the target server over the management network. If a
memory page changes during the precopy phase, it is marked dirty and resent. Once the majority of the pages are copied, the
stop-and-copy phase begins. The stop-and-copy phase starts by pausing the guest while the remaining dirty pages are copied to the
target, which usually takes 60 to 300 milliseconds. Once the pages are copied to the target, the guest is started on target server.
Note: Oracle VM server does not support memory oversubscription, which means that an Oracle VM server cannot accept a Live
Migration request unless the server has available RAM for the virtual machines. Oracle VM server supports CPU oversubscription,
which means that an Oracle VM server can accept a Live Migration request and overallocate the total number of CPU threads to
virtual machines, if necessary.
Oracle VM for x86 Intracomponent Communication & Firewall Requirements

This section will examine Oracle VM for x86's intracomponent communication and firewall requirements. The goal of this section is to
illustrate the communication ports and system passwords required to help plan, build, and support an Oracle VM for x86
environment.
List 3 highlights Oracle VM communication ports and system passwords used with Oracle VM Manager.
Oracle VM Manager:
TCP 22/ssh is the default service used to access the Oracle VM Manager CLI.
Oracle VM Manager communicates with the Server Pool Master agent using TCP/8899.
The default HTTPS port for the Oracle VM Manager portal is 4443.
The default HTTP port for Oracle VM Manager portal is 8888.
The default HTTP port for the Oracle Database 10g Express Edition portal is 8080.
The default HTTP port for the Oracle Application Server 10g portal is 8888.
The default Database listening port for the Oracle Database 10g Express Edition repository is 1521.
Oracle Database 10g Express Edition accounts are SYS and SYSTEM.
Virtual machine VNC console access from the Oracle VM Manager portal uses ports 5900 through 5999.
Virtual machine VNC console access requires a VNC password without a user name. Virtual machine VNC console passwords are
assigned using Oracle VM Manager and are saved in clear text in each virtual machine's vm.cfg file.
Oracle VM Server:
The Oracle VM Agent listens on TCP 8899 and requires a password. The agent password is selected during the Oracle VM server
installation and can be configured from dom0 by typing “service ovs-agent configure”.
The xend-relocation-server service listens for Live Migration requests on TCP 8002. The xend-relocation-server is managed by
xend and configured using the /etc/xen/xend-config.sxp file. TCP 8002 must be open to all server pool members.
The cluster heartbeat is active on TCP 7777 and must be open to all server pool members.
SSH is enabled by default on TCP 22.
VNC console access to virtual machines uses TCP 5900 through 5999.
The rpcbind process (portmapper) listens on TCP/UDP 111.
Figure 9 illustrates Oracle VM for x86 intra-component communication and firewall requirements with Oracle VM Manager.
20 of 243 18.04.2011 16:35

Next, we will examine the intracomponent communication and firewall requirements for an Oracle VM for x86 environment using the
Oracle Enterprise Manager Oracle VM Management Pack Plug-in.
List 4 highlights communication ports used with the Oracle Enterprise Manager Oracle VM Management Pack Plug-in.
The Oracle Enterprise Manager Oracle VM Management Pack Plug-in:
Oracle Enterprise Manager Oracle VM Management Pack Plug-in communicates with the Server Pool Master agent using
TCP/8899.
The default HTTP port for the Enterprise Manager Grid Control portal is 4889.
The default database listening port for the Oracle Database repository is 1521.
Virtual machine VNC console access from the Oracle VM Manager portal uses ports 5900 through 5999.
Virtual machine VNC console access requires a VNC password without a user name. Virtual machine VNC console passwords are
assigned using Oracle VM Manager and are saved in clear text in each virtual machine's vm.cfg file.
Oracle VM Server:
The Oracle VM Agent listens on TCP 8899 and requires a password. The agent password is selected during the Oracle VM server
installation and can be configured from dom0 by typing “service ovs-agent configure”.
The xend-relocation-server service listens for Live Migration requests on TCP 8002. The xend-relocation-server is managed by
xend and configured using the /etc/xen/xend-config.sxp file. TCP 8002 must be open to all server pool members.
The cluster heartbeat is active on TCP 7777 and must be open to all server pool members.
SSH is enabled by default on TCP 22.
VNC console access to virtual machines uses TCP 5900 through 5999.
The rpcbind process (portmapper) listens on TCP/UDP 111.
Oracle VM Guests:
The Oracle Management Agent to Oracle Management Service communication is HTTPS on 4889.
Figure 10 illustrates Oracle VM for x86 intra-component communication and firewall requirements with the Oracle VM Management
Pack.
21 of 243 18.04.2011 16:35

1.0 – The Oracle VM for x86 System Reference Design Introduction

This section presents the Oracle VM for x86 reference design. The Oracle VM for x86 reference designs encompass the software,
hardware, storage, and network components required to deploy a scalable, secure, and supportable Oracle VM for x86 solution in an
internal or external cloud.
The Oracle VM for x86 reference design is a field-tested best-practice standard, designed with simplicity, reproducibility, usability,
scalability, supportability and security. The Oracle VM for x86 reference designs represent a complete Oracle VM for x86 standard
that can be leveraged as a vanilla solution or modified to more accurately reflect organization-specific needs. The Oracle VM for x86
reference design includes the following categories and solutions:
Cloud Infrastructure
Administration and Monitoring
Oracle VM Server Pools
Oracle VM Server Pool Storage Design
Oracle VM Server Pool Network Design
Oracle VM Templates
Virtual Machine Operating Systems
Note: A detailed explanation of each category and solution in the Oracle VM for x86 reference design is presented in the architectural
overview section.
1.1 – The Oracle VM for x86 System Reference Design Implementation Overview
The Oracle VM for x86 reference design provides a well defined starting point for each implementation. It also serves as a baseline
upon which all solution additions, revisions, and tools will be based. As such, there is an increasing value to Oracle VM for x86
reference design users in keeping implementations as close to the reference design as possible.
Prior to implementing Oracle VM for x86 based upon the Oracle VM for x86 reference design, it’s important that an infrastructure
assessment (IA) and gap analysis (GA) be performed. During the IA/GA, the architecture of the solution will match the customer’s
business needs while maintaining the integrity of the Oracle VM for x86 reference design. Implementation and support will follow the
analysis phase after careful consideration has been given to any specific design modifications that deviate from the Oracle VM for x86
reference design.
1.2 – The Oracle VM for x86 System Reference Design Document Overview
This document outlines the decision points necessary for implementing the Oracle VM for x86 reference design. For decisions that
rely on preexisting factors or specific organizational needs, the appropriate best practice will be discovered in the infrastructure
assessment (IA) and gap analysis (GA). The best practices should be analyzed carefully and decisions should be made based on
organizational needs, existing architecture, and budget resource availability.
2.0 – The Oracle VM for x86 System Reference Design Architectural Overview
22 of 243 18.04.2011 16:35

The Oracle VM for x86 reference design is designed to be scalable and resilient for ease of implementation, high availability, and ease
of maintenance for internal and external clouds. The complete solution is made up of six architectural components that work together
to provide flexibility and options with respect to server consolidation, application delivery, monitoring, authentication and security
requirements, user access scenarios, and component migration/modification. The design breaks down into the following six
components:
Cloud Infrastructure. The Oracle VM for x86 reference design offers a cloud virtualization and management layer that allows
multiple Oracle Linux, Red Hat Linux, Solaris, and Windows virtual machines to run on a shared x86-64 hardware environment
allowing x86-64 bit server consolidation, and rapid application provisioning.
Administration and Monitoring: The Oracle VM for x86 reference design provides complete transparency from a cloud
infrastructure and an application perspective to any Oracle VM server, virtual machine, or application running in the cloud using
Oracle Enterprise Manager with Oracle VM Manager. Oracle Enterprise Manager allows organizations to consistently and
quantitatively measure performance across the organization for all of the hosted applications in the cloud.
Oracle VM Server Pools. The Oracle VM for x86 reference design may have one or more Oracle VM server pools to provide
isolation, defense in depth, the principle of least privilege, compartmentalization of information, and security domains, and to
accommodate different applications and their performance, authentication, and security requirements. This design builds on the
ability to test new applications alongside production applications in isolated Oracle VM server pools, without sacrificing the
integrity of the production environment.
Oracle VM Templates. The Oracle VM for x86 reference design will support an organization’s entire Oracle application
portfolio using Oracle VM templates. An Oracle VM template can consist of one or more virtual machines containing a
preconfigured operating system with an application. Using Oracle VM templates eliminates the operating system and application
installation and configuration process, allowing applications to be deployed from a library of Oracle VM templates. Oracle VM
templates used in conjunction with Oracle VM server pools accommodate different applications and their performance,
authentication, and security requirements. This design builds on the ability to test new applications alongside production
applications in isolated Oracle VM server pools, without sacrificing the integrity of the production environment.
Virtual Machine Operating Systems. The Oracle VM for x86 reference design standardizes on a small number of virtual
machine operating systems, operating system versions, and virtualization modes to streamline operations and to increase the
level of file duplication on production and archival virtual machine data. This design reduces complexity and increases
operational efficiency by limiting the number of supported operating systems.
Oracle VM Server Pool Storage Design. The Oracle VM for x86 reference design standardizes one OCFS2 or NFS 4GB root
repository that is used to host each server pool's Oracle Cluster File System v2 (OCFS2) cluster and HA configurations, and a
maximum of 10 extended repositories used exclusively to host virtual machine configuration files and images. When a server pool
needs more storage capacity, additional extended OCFS2 and/or NFS repositories are added to a server pool to increase capacity.
Oracle VM Server Pool Network Design. The Oracle VM for x86 reference design standardizes one of two Oracle VM server
pool network designs that both support defense in depth, the principle of least privilege, compartmentalization of information,
and security domains, and accommodate different applications and their performance, authentication, and security requirements.
1. A single two NIC 802.3AD bond with Xen bridges using 802.1Q-Tagged VLANs to isolate traffic between the Oracle VM
Manager management network and the virtual machine networks.
Figure 11
2. 4 NICs with two 802.3AD bonds. One two NIC bond is dedicated to Oracle VM management traffic with a fixed IP and the
second two NIC bond will use Xen bridges with 802.1Q-Tagged VLANs to isolate virtual machine networks.
Figure 12
23 of 243 18.04.2011 16:35

Figure 13 shows a high-level overview of the Oracle VM for x86 reference design components.
The Oracle VM for x86 reference design isolates Oracle VM server pools into the following four security domains:
Controlled: A controlled security domain is used to restrict access between security domains. A controlled security domain could
contain groups of users with their network equipment or a demilitarized zone (DMZ).
Uncontrolled: An uncontrolled security domain refers to any network not in control of an organization, such as the Internet.
Restricted: A restricted security domain can represent an organization’s production, test and development networks. Access is
restricted to authorized personnel, and there is no direct access from the Internet.
Secured: A secured security domain is a network that is only accessible to a small group of highly trusted users, such as
administrators and auditors.
Note: The classification of security domains is very similar to data classifications. FIPS PUB 199 is the Standards for Security
Categorization of Federal Information and Information Systems. FIPS PUB 199 can be used to determine the security category of
systems and within which security domain systems should reside.
2.1 - The Oracle VM for x86 System Reference Design Support Infrastructure
Support is an integral part of the Oracle VM for x86 reference design and includes a combination of an Oracle support agreement and
on- and off-site support from the implementing party. Administrators will have several options for support, including live assistance,
phone support, and forums.
Part 1 - The Oracle VM for x86 System Reference Design
3.0 Cloud Infrastructure Architecture

This section provides a decision matrix for the Oracle VM for x86 reference design. Implementers of the Oracle VM for x86 reference
design can use the decision matrix as quick reference guide to identify settings and configuration decisions to be implemented in the
environment. Decisions highlighted in yellow may rely on preexisting environment factors or differ depending on organizational
needs. These decisions should be carefully analyzed during a gap analysis phase.
3.1 - Oracle VM for x86 Hardware Architecture
24 of 243 18.04.2011 16:35

Decision Decision Justification

Point
Server Servers will be NA

Vendor procured from a
reliable hardware
vendor.
Processors Two socket The Maximum Number of CPUs or

multiple-core threads an Oracle VM server can
processors for support is 128. Oracle VM server
standard workloads maps a virtual CPU to a hardware
and four socket thread on a CPU core in a CPU
multiple-core socket. A CPU core or a
processors for large hyperthread is considered a
CPU-bound workloads. physical CPU by the Xen hypervisor.
For example, a server with an Intel

Xeon processor 5600-series CPU
with hyperthreading can have up to
six cores and twelve threads per
socket. A two socket server with an
Intel Xeon processor 5600-series
CPU could allocate twenty four
virtual CPUs without
oversubscribing the physical CPUs.
CPU-bound workloads should not
be on servers that have
oversubscribed CPUs.
Two socket multiple-core

processors are ideal for standard
non-CPU-bound workloads. For
example, workloads with up to 8
virtual CPUs.
Four socket multiple-core

processors are ideal for large
CPU-bound workloads. For
example, workloads with 16 up to
32 virtual CPUs.
RAM Servers will be Oracle VM server does not support

ordered with the memory oversubscription, which
maximum amount of means that an Oracle VM server
physical memory. cannot accept a Live Migration or
HA request unless the server has
available RAM for the virtual
machines. Having available RAM on
each server provides flexibility in
terms of adding new virtual
machines to the server pool, and to
allow Live Migration and HA within
a server pool.
Internal Unless the Oracle VM An Oracle VM server installation

storage server is booting from “without” local virtual machine
SAN, redundant storage requires 1GB of storage
internal hard drives or and 1GB of swap space. Oracle VM
a 2GB flash drive is server can be installed on a 2GB
required. Virtual flash drive or on local redundant
machine images will hard drives.
be stored on shared
SAN, iSCSI, or NFS
repositories.
NICs A minimum of two and For network-interface height

up to four NICs per availability 802.3AD bonds will be
server. used for each pair of network
interfaces up to a maximum of four
network interface cards with two
bonds.
25 of 243 18.04.2011 16:35

Build Process PXE boot Oracle VM server will be installed

using an automated PXE boot
configuration to ensure that each
server has a consistent installation
configuration.
3.2 - Oracle VM for x86 Server Pool Configurations

Point
Oracle VM for Use multiple Oracle Single point of administration for

x86 server VM server pools multiple Oracle VM server pools
pool design managed from Oracle accommodating organization-
and VM Manager. specific needs, i.e., defense in
management depth, the principle of least
privilege, compartmentalization of
information, security domains, and
to accommodate different
applications and their
performance, authentication, and
security requirements. This design
builds on the ability to test new
applications alongside production
applications in isolated Oracle VM
server pools, without sacrificing
the integrity of the production
environment.
If more than one location exists,

Oracle VM server pools may be
dispersed to different locations.
Version of The latest version of The latest version of Oracle VM

Oracle VM Oracle VM server and server and Oracle VM Manager
server and Oracle VM Manager will be used to ensure that system
Oracle VM will be used. has the latest platform and
Manager security updates and patches.
Oracle VM Server Pool Master Server Pool Master

agent role Each Oracle VM To eliminate a single point of
configurations server pool will failure, each Oracle VM server
enable the Virtual IP pool will enable the Virtual IP
feature. feature.
Utility Server Utility Server

Each Oracle VM Each Oracle VM server pool will
server pool will have have at least one dedicated utility
at least one dedicated server to isolate the impact of I/O
utility server with intensive operations on the Utility
only the Utility Server Servers. Utility Servers will only
role enabled. have Utility Server role enabled.
Virtual Machine Virtual Machine Server

Server Virtual Machine Servers will only
Virtual Machine have the Virtual Machine Server
Servers will have only role enabled to dedicate system
the Virtual Machine resources to running virtual
Server role enabled. machines, eliminating the effect of
Utility Server operations.
Oracle VM Each Oracle VM Each Oracle VM server pool uses

Server Pool server pool will have one OCFS2 or NFS 4GB root
Storage one OCFS2 or NFS repository that is used to host
Configurations 4GB root repository each server pool's OCFS2 cluster
that is used to host and HA configurations to isolate
each server pool's and protect the cluster and HA
Oracle Cluster File configurations from virtual
System v2 (OCFS2) machine configuration operations.
cluster and HA Up to 10 extended repositories can
26 of 243 18.04.2011 16:35

configurations and up be used exclusively to host virtual

to a maximum of 10 machine configuration files and
extended repositories images. The Oracle VM agent can
used exclusively to reliably support up to 10 extended
host virtual machine repositories.
configuration files and
images. OCFS2 is not integrated or
supported with any volume
manager (LVM) solutions to
manage the back-end block
storage. Fibre Channel and iSCSI
OCFS2 partitions must be
provisioned at static sizes, i.e.,
partition sizes cannot change once
a partition is formatted with
OCFS2. When a server pool needs
more storage capacity, up to ten
extended OCFS2 and/or NFS
repositories can be added to a
server pool to increase capacity.
Oracle VM Each Oracle VM pool For network interface link

Server Pool member will use one aggregation and height availability
Network of the two following 802.3AD bonds will be used for
Configurations. network each pair of network interfaces up
configurations: to a maximum of four network
1) A single two NIC interface cards with two bonds.
802.3AD bond with
Xen bridges using 802.1Q-Tagged VLANs are used to
802.1Q-Tagged configure isolated virtual networks
VLANs to isolate using "smart" or managed
traffic between the Ethernet switches, rather than
Oracle VM relying on cables wired into
management network physical switches and routers. For
and the virtual example, with VLANs a LAN can
machine networks. be segmented into isolated
2) 4 NICs with two security domains that offer
802.3AD bonds. One defense in depth, the principle of
two NIC bond is least privilege, and
dedicated to Oracle compartmentalization of
VM management information, and can
traffic with a fixed IP accommodate different
and the second two applications and their
NIC bond will use Xen performance, authentication, and
bridges with 802.1Q- security requirements, without
Tagged VLANs to being restricted by physical
isolate virtual connections.
machine networks.
Patch Oracle VM servers A key component of patch

Management will be configured to management is acquiring and
use local custom yum vetting patches for production
repositories. systems. Patches must be
researched to identify which
All patches will be patches, security fixes, and
regression tested in application updates are applicable
the lab environment to your environment. Newly
before they are released patches, security
deployed on updates, and application updates
production systems. will be tested before being
High-priority patches, deployed in to production using
security fixes, and time stamped local custom
application upgrades repositories.
updates will be
applied as needed in Local yum repositories will be
accordance with maintained for patch testing and
<Company Name>’s production using a point-in-time
Change Management static channel for each supported
Policy. operating system to ensure all like
operating systems are patched in a
Noncritical fixes will consistent manner across the
be applied on a organization.
Quarterly basis in
accordance with Pre- and post-production audits
27 of 243 18.04.2011 16:35

<Company Name>’s will be conducted in accordance

Change Management with <Company Name>’s Change
Policy. Management Policy to validate
configuration and patch
All production compliance.
systems will undergo
security audits in
accordance with
<Company Name>’s
Change Management
Policy to validate
configuration and
patch compliance.
3.3 - Oracle VM for x86 Network Topology

Point
NIC Each Oracle VM server For network interface link

Configuration will use 802.3AD aggregation and height availability
bonds for each pair of 802.3AD bonds will be used for
network interfaces up each pair of network interfaces up
to a maximum of four to a maximum of four network
network interface interface cards with two bonds.
cards with two bonds.
Switch Port 802.1Q-Tagged VLANs 802.1Q-Tagged VLANs are used to

Configuration configure isolated virtual networks
using "smart" or managed Ethernet
switches, rather than relying on
cables wired into physical switches
and routers. For example, with
802.1Q-Tagged VLANs a LAN can
be segmented into isolated security
domains that offer defense in
depth, the principle of least
privilege, and
compartmentalization of
information, and can accommodate
different applications and their
performance, authentication, and
security requirements, without
being restricted by physical
connections.
IP Addresses Static Using static reserved IP addresses

eliminates issues associated with
duplicated or reassigned IP
addresses.
Location of Each production Each Oracle VM server pool will

Oracle VM Oracle VM server pool restrict Oracle VM management
server pool have an isolated traffic to an isolated network
management management network subnet to protect the server pool
network subnet. from broadcast storms that could
saturate the management network
and cause pool members to loose
network connectivity and fence
from the pool.
Location of Oracle VM Manager Oracle VM Manager will be the

Oracle VM will be located on a single point of administration for all
Manager isolated network Oracle VM server pools and should
segment. be located on a secure isolated
network segment to protect access
to the Oracle VM Manager portal
and to better control and monitor
communication to the server pool
masters.
28 of 243 18.04.2011 16:35

Virtual Inbound TCP 5900 To facilitate virtual machine

Machine through 5999 traffic console access, inbound TCP 5900
Console will be allowed to each through 5999 traffic must be
Access Oracle VM server pool allowed to each Oracle VM server
network subnet. pool network subnet from the
system administrators network
subnet.
Port The default ports will Using the default ports creates
Numbers be used for all Oracle consistency and simplifies
VM server and Oracle configuration.
VM Manager
components.
NFS Storage If NFS shared storage NFS shared storage repositories

Repositories repositories are used, will be restricted to the Oracle VM
the storage network Management network subnet to
will be restricted to isolate and secure NFS storage
the Oracle VM traffic.
Management network
subnet.
3.4 – Oracle VM for x86 Network Security Architecture

Point
Oracle VM The iptables service will Host firewalls, for example

Server and be enabled on each iptables, are a fundamental part
Oracle VM Oracle VM server using of information security that
Manager the default protect hosts from attacks and
iptables policy and ruleset in intrusions.
configurations /etc/sysconfig/iptables.
Root ssh Root ssh access will be By default, Oracle VM servers

access disabled on all Oracle permit ssh access using the root
VM servers and sudo super user account. One of the
will be configured for most important security measure
root access. that can be taken on an Oracle
VM server is to prevent
To disable root ssh unauthorized access to the root
access, edit the default user account by disabling root ssh
/etc/ssh/sshd_config file access.
and uncomment the Systems administrators will
the access the Oracle VM servers
“#PermitRootLogin with user privileges and use sudo
yes” line and change for root access. All sudo user
the yes to no; that is, access will be tracked and logged
“PermitRootLogin in the /var/log/secure file.
no”. Next, restart the
sshd service by typing
“service sshd restart” to
enable the change.
To enable sudo access

for a users, from dom0
type “viedit” and add
the account names
under the following
lines:
## Allow root to run
any commands
anywhere
root ALL=(ALL) ALL
newuser ALL=(ALL)
ALL
The above example

provides root access to
29 of 243 18.04.2011 16:35

all commands for the

newuser account. More
restrictive access to
commands can be
configured for each
user. Consult the
sudoers man page for
details.
SSH login Pre and post SSH login To be able to successfully

banners banners will be prosecute individuals who
configured on each improperly use a computer, the
Oracle VM server. computer must have a warning
banner displayed at all access
Pre-login banner: points.
Edit the /etc/ssh
/sshd_config and add SSH login banners presents a
the following directive: definitive warning or disclaimer
Banner /etc/banner.net to all users that wish to access
your systems using SSH. SSH
Next, create the login banners should clarify
/etc/banner.net file and which types of activities are
add your login banner, illegal as well as advise legitimate
i.e. users of their obligations relating
to the acceptable use of the
vi /etc/banner.net system.
This system is
restricted to authorized
access only. All
activities on this system
are recorded and
logged. Unauthorized
access will be fully
investigated and
reported to the
appropriate law
enforcement agencies.
:wq!
Once the file has been

created and the banner
text is added and saved,
restart the sshd by
typing:
# service sshd restart
Post login banner:

Edit /etc/motd and add
your login banner text,
i.e.
vi /etc/motd
This system is
restricted to authorized
access only. All
activities on this system
are recorded and
logged. Unauthorized
access will be fully
investigated and
reported to the
appropriate law
enforcement agencies.
:wq!
Once the file has been

edited and saved,
restart the sshd by
typing:
# service sshd restart
Iptables failed Iptables failed Failed connect logging is a

connection connection logging will fundamental part of information
logging be enabled on each security that allows detection of
30 of 243 18.04.2011 16:35

Oracle VM server. attacks and intrusions.
The following two lines

will be added prior to
the last REJECT line in
the /etc/sysconfig
/iptables file:
-A RH-Firewall-1-INPUT
-m limit --limit
15/minute -j LOG
--log-prefix "FW Drop:"
-A RH-Firewall-1-INPUT
-j REJECT --reject-with
icmp-hostprohibited
Central log A central log host will Centralized logging for user
host be used to log all user logins and iptables connection
logins and iptables failures simplifies security
connection failures. management for the detection of
attacks and intrusions.
Oracle VM Oracle VM Manager The Oracle VM Manager

Manager in a will not be placed in a application was not designed to
DMZ DMZ. be an Internet facing application.
If remote access is a requirement
for Oracle VM Manager VPN
access will be used.
Oracle VM Oracle VM server Oracle VM servers in a DMZ are

Servers in a hosting Internet facing restricted from inbound and
DMZ virtual machines will be outbound Internet connectivity to
placed in a DMZ minimize the risk of attack from
without connectivity to the Internet.
the Internet or internal
network segments, Oracle VM servers in a DMZ are
except TCP/8899 from restricted from access to internal
Oracle VM Manager. network segments except
TCP/8899 from Oracle VM
Manager.
If virtual machine console access

is required, inbound TCP/5900
through 5999 will be configured
to each Oracle VM server in the
DMZ for virtual machine console
access.
3.5 - Oracle VM for x86 Administration and Monitoring

Point
Oracle VM for Oracle VM for x86 Oracle VM Manager with the

x86 server server pool Oracle VM Manager Command
pool administration will be Line Interface has the broadest
administration done using Oracle VM administration feature set
and Manager with the available for Oracle VM for x86.
monitoring Oracle VM Manager For example, the Oracle
Command Line Enterprise Manager Oracle VM
Interface. Management Pack Plug-in does
not a have comparable Command
Oracle VM for x86 Line Interface to automate
server pools will be administrative functions.
monitored using an
SNMP based solution. Oracle VM Manager does include
monitoring. An SNMP based
solution for monitoring Oracle VM
servers is the best option since
there are no additional software
requirements.
31 of 243 18.04.2011 16:35

Centralized A central log host will When things go wrong with an

logging for be configured to Oracle VM server pool, being able
Oracle VM capture the Oracle VM to quickly determine the “root
server agent server agent logs and cause” of an issue can eliminate or
logs and the the Oracle VM reduce down time. The most
Oracle VM Manager OC4J logs. effective way to identify problems
Manager with an Oracle VM server pool is
oc4j.log. to analyze the Oracle VM Manager
OC4J log and Oracle VM server'
agent log files.
Rootkit Each Oracle VM server Wikipedia describes a rootkit as” A

prevention will have the rootkit is software that enables
and chkrootkit RPM continued privileged access to a
monitoring installed. computer while actively hiding its
presence from administrators by
subverting standard operating
system functionality or other
applications.”
Monitoring systems for rootkits is

fundamental part of information
security that allows the detection
of rootkits to prevent attacks and
intrusions.
3.6 - Oracle VM for x86 Virtual Machine Operating Systems

Point
Virtual A small number of Standardizing on a small number of

Machine virtual machine virtual machine operating systems
Operating operating systems will streamlines operations and
Systems be used. increases the level of file
duplication on production and
archival virtual machine data. This
design reduces complexity and
increases operational efficiency by
limiting the number of supported
operating systems.
Virtual In accordance with Each application has an operating

Machine <Company Name>’s system support matrix that lists the
Operating Application Software supported operating systems, patch
System Policy and Application levels, and software prerequisites.
Versioning Software Standards,
applications will In accordance with <Company
determine the Name>’s Application Software
operating system type Policy and Application Software
and version. Standards, applications will
determine the operating system
type and version.
Virtual All new virtual machine A virtual machine template is a

Machine operating systems will self-contained, preconfigured
Operating be deployed using a virtual machine with an operating
System virtual machine system and optionally an
Deployments template in accordance application installed in accordance
with <Company with <Company Name>’s Server
Name>’s Server Policy, Policy, Server Security Policy, and
and Server Security Operating System Installation
Policy. Guidelines.
Each time a new virtual machine is

deployed using a virtual machine
template, <Company Name>’s
standards are applied to each new
virtual machine.
32 of 243 18.04.2011 16:35

Patch Linux virtual machines A key component of patch

Management will be configured to management is acquiring and
use local custom yum vetting patches for production
repositories. systems. Patches must be
researched to identify which
All patches will be patches, security fixes, and
regression tested in the application updates are applicable
lab environment before to your environment. Newly
they are deployed on released patches, security updates,
production systems. and application updates will be
High-priority patches, tested before being deployed in to
security fixes, and production using time stamped
application upgrades local custom repositories.
updates will be applied
as needed in Local yum repositories will be
accordance with maintained for patch testing and
<Company Name>’s production using a point-in-time
Change Management static channel for each supported
Policy. operating system to ensure all like
operating systems are patched in a
Noncritical fixes will be consistent manner across the
applied on a Quarterly organization.
basis in accordance
with <Company Pre- and post-production audits will
Name>’s Change be conducted in accordance with
Management Policy. <Company Name>’s Change
Management Policy to validate
All production systems configuration and patch
will undergo security compliance.
audits in accordance
with <Company
Name>’s Change
Management Policy to
validate configuration
and patch compliance.
3.7 - Oracle VM for x86 Application Integration

Point
Application Applications must be Applications that are incompatible

Support supported by the with and unsupported by Oracle
independent software VM for x86 cannot be supported by
vendor (ISV) on the <Company Name> on Oracle VM
latest version of Oracle for x86. Only applications with ISV
VM for x86 to be support for the latest version of
included in the Oracle Oracle VM for x86 can be deployed
VM for x86 and supported by <Company
environment. Name>, the ISV, and Oracle on
Oracle VM for x86.
Application Applications will be Applications will be analyzed for

Requirements analyzed for requirements and dependencies
and requirements and and tested to ensure compliance
Dependencies dependencies and with ISV specifications.
tested in accordance
with <Company
Name>'s Software
Installation Standards.
Application Applications should be Application installations that are

Installation, installed and packaged packaged in Oracle VM templates
Packaging, using Oracle VM or deployed using Oracle
and Template Builder or Enterprise Manager have a
Distribution Oracle Enterprise consistent installation
Manager. configuration.
33 of 243 18.04.2011 16:35

Applications that are

installed and packaged
using Oracle VM
Template Builder will
be deployed as Oracle
VM templates.
Applications that are

installed using Oracle
Enterprise Manager
will be installed on
Oracle VM templates.
Application Applications will be Applications that have reached the

sunsetting sunsetted in end of their life cycle and are no
accordance with longer supported by a vendor will
<Company Name>'s be given a sunset date. The sunset
Hardware and date is when the product is
Software Sunset Policy scheduled to be removed from
production.
Sunsetting applications that have

reached the end of their life cycle
results in better customer service
and reduced costs.
Patch All patches will be A key component of patch

Management regression tested in management is acquiring and
the lab environment vetting patches for production
before they are systems. Patches must be
deployed on researched to identify which
production systems in patches, security fixes, and
accordance with application updates are applicable
<Company Name>’s to your environment. Newly
Change Management released patches, security updates,
Policy. and application updates will be
tested before being deployed in to
Noncritical fixes will production using time stamped
be applied on a local custom repositories.
Quarterly basis in
accordance with Pre- and post-production audits
<Company Name>’s will be conducted in accordance
Change Management with <Company Name>’s Change
Policy. Management Policy to validate
configuration and patch
All production systems compliance.
will undergo security
audits in accordance
with <Company
Name>’s Change
Management Policy to
validate configuration
and patch compliance.
3.8 - Oracle VM for x86 Support Services

Point
Oracle Oracle Support Support is an integral part of every

Support Agreement for Oracle successful IT project. An Oracle
Agreement VM and Oracle Linux support agreement is necessary to
will be active and up to be able to receive RPM patches and
date. updates for Oracle VM and Oracle
Linux and to create and manage
service requests.
34 of 243 18.04.2011 16:35

On-site and On-site and off-site On-site and off-site support from
Off-site support from the the implementing party for problem
support implementing party will resolution, system maintenance,
be used for site reviews, upgrades, and
maintenance, site security audits augments the
reviews, upgrades, and Oracle support agreement and
security audits. internal IT operations staff.
Watch this space

Chapter 3: Hard and Soft Partitioning Oracle Technologies

with Oracle VM
cube.jpg
This chapter clarifies how Oracle VM can be used with hard and soft partitioning to manage your Oracle technology license costs.
Last update 12-07-2010
This chapter will review hard and soft partitioning Oracle technologies with Oracle VM. The goal of this chapter is to clarify how
Oracle VM can be used with hard and soft partitioning to help manage your Oracle enterprise technology license costs. The chapter
starts with a brief introduction to Oracle licensing. Next, we will review Oracle technology named user plus licensing followed with
processor licensing with hard and soft partitioning using Oracle VM. The chapter concludes with hard partitioning examples and
virtual CPU binding testing techniques.
Note: While Oracle recognizes hard and soft partitioning with Oracle VM, this does not imply that this applies when using other
vendor's virtualization technologies. Please refer to the SIG or your Oracle representative if you have questions about the licensing
impact of other vendor's virtualization approaches.
Table of Contents
Oracle Technology Licensing
Oracle Technology Licensing with Oracle VM
…Named User Plus Licensing
…Named User Plus Licensing versus Processor Licensing
…Processor Licensing
Hard and Soft Partitioning with Oracle VM
…Soft Partitioning Examples
…Hard partitioning Examples
…Oracle VM Manager Manual Placement Policy Configuration
Hard Partitioning an Oracle VM Guest
…CPU Pinning Examples
…CPU Pinning with xm
Resources
Oracle Technology Licensing

Oracle segments its product portfolio into two categories, technology and applications. The Oracle technology and applications
license models are very different. The only similarity between the technology and applications licensing models is the ability to
execute an unlimited license agreement (ULA). Technology products have three forms of licensing, 1) processor 2) named user plus
(NUP) and 3) unlimited license agreement (ULA). Applications licensing also have three forms of licensing, 1) component pricing, 2)
custom applications suite pricing, and 3) enterprise pricing aka an unlimited license agreement (ULA).
List 1 shows Oracle’s technology offering.
Database
Enterprise Managers
Application and System Management
Application Server
Fusion Middleware
Business Intelligence
35 of 243 18.04.2011 16:35

Identity Management
Tools
Enterprise 2.0
Collaboration
Data Warehousing Products
Integration products
List 2 shows Oracle’s application offering.
Oracle Analytic and Business Intelligence (BI) Applications

Oracle Customer Relationship Management (CRM)
Oracle Financial Management
Oracle Governance, Risk, and Compliance (GRC) Management
Oracle Human Capital Management (HCM)
Oracle Procurement
Oracle Project Management
Oracle Supply Chain Management (SCM)
The only way to determine the most beneficial licensing model for your Oracle software investment is to evaluate your organization’s
Oracle software requirements, along with your hardware, operating system and virtualization configurations. Most organizations
initially engage their Oracle sales representatives as a first step, in order to help evaluate and quote license options. Customers
typically use the initial licensing evaluation and quotes as a starting point to help determine which licensing model and configuration
provides the best value.
An important Oracle technology licensing consideration is your organization’s hardware, operating system and virtualization
configurations. Oracle recognizes a wide variety of hardware, operating system and virtualization configurations that directly affect
the CPU count used to calculate Oracle technology processor licenses. For example, Oracle recognizes various hard and soft
partitioning configurations for Oracle VM, as well as the big 3 UNIX platforms, that directly affect how to count Oracle technology
CPU licenses.
From an Oracle technology licensing perspective, hard partitioning allows customers to license a subset of a server’s CPUs.
Conversely, soft partitioning counts the total number of a server’s CPUs.
Note: Oracle VM is not a licensed technology product.
Oracle Technology Licensing with Oracle VM

Of the three Oracle technology licensing options, 1) processor 2) named user plus (NUP) and 3) unlimited license agreement (ULA),
Oracle VM can help manage and reduce processor licensing costs with Oracle enterprise edition technology products. Oracle VM
helps manage enterprise edition technology processor licensing, by using hard and soft partitioning.
Hard partitioning with Oracle VM allows a customer to license a subset of an Oracle VM server’s CPUs. Soft partitioning is used to
take advantage of Live Migration, which is not supported with hard partitioning. Along with Live Migration, soft partitioning provides
the ability to manage the number of licensed Oracle technology product CPUs within an Oracle VM pool.
Hard and soft partitioning with Oracle VM provide the ability to manage the number of licensed Oracle technology CPUs. Named user
plus licensing and unlimited license agreements are not CPU regulated, which preclude using Oracle VM as a license management
option.
The SIG states that Oracle technology standard edition products are limited to 2 or 4 sockets, i.e. installed on a server with no more
than 2 or 4 physical CPUs. Standard edition products can run on an Oracle VM as long as the Oracle VM server meets the standard
edition’s CPU requirements. Most contemporary virtualization servers are equipped with 2 or 4 sockets, which may preclude hosting
standard edition products on Oracle VM.
Understanding how and where to use hard and soft partitioning with Oracle VM can help organizations to better manage their Oracle
Enterprise technology licensing costs for development and production environments.
Named User Plus Licensing

Smaller deployment with less than 50 users regularly select named user plus licensing. For smaller environments, named user plus
licensing can be more cost effective when compared with processor licensing. Along with named user plus licensing, customers
regularly select standard editions products over enterprise editions products to further reduce costs.
Named User Plus Licensing versus Processor Licensing

Oracle VM with processor licensing could provide an alternative to named user plus licensing by leveraging hard partitioning to
manage the number of licensed CPUs. The ability to manage the number of licensed CPUs can help control licensing costs, which may
provide a cost advantage over named user plus licensing. You would need to run the numbers to determine if hard partitioning could
provide a cost advantage over named user plus licensing.
Processor Licensing
Processor licensing is when an Oracle customer pays per processor (CPU), to run an Oracle technology product. Larger deployments,
with 50 or more users, typically use processor based licensing. Oracle recognizes each CPU core as a separate CPU and each CPU
type with a different processor factor. The processor factor determines the CPU count. The CPU count determines the number of
CPUs required to license the Oracle technology product.
Note: Be sure to refer to the latest Oracle Processor Core Factor Table to find out the processor core factor for your hardware.
36 of 243 18.04.2011 16:35

Table 1 lists the processor factors.

Oracle Processor Licensing Processor Factor
UltraSparc T1 0.25
AMD/Intel 0.50
All other Multi-core Servers 0.75
Single Core Servers 1.00

To better understand how to calculate a processor factor, List 1 shows the processor factor for a single quad core Intel, AMD, Sun
Sparc and an IBM Power CPU.
Intel or AMD CPU

1 quad core CPU requires 2 processor licenses (4 cores multiplied by a factor of .50 equal 2 processor licenses).
Sun SPARC64 VI CPU (* different models of Sun CPUs may have different core factors)
1 quad core SPARC64 VI CPU requires 3 processor licenses (4 cores multiplied by a factor of .75 equal 3 processor license).
IBM Power6 CPU (* different models of IBM CPUs may have different core factors)
1 quad core Power6 CPU requires 4 processor licenses (4 cores multiplied by a factor of 1.0 equal 4 processor licenses)
Two and four core CPUs are now end of life. New Intel x86 servers’ ship with six or eight core CPUs. AMD plans to ship their 12 core
CPUs in the first half of 2010. Both Sun Sparc and IBM Power servers now ship with eight core CPUs. As the chip vendors add more
cores to CPUs, Oracle technology licensing costs can increase.
To better understand the impact to Oracle processor licensing with multi-core CPUs, let’s review List 2.
List 2 shows the processor factor for a single eight core Intel, AMD, Sun Sparc, and IBM Power CPU.
Intel or AMD CPU

1 eight core CPU requires 2 processor licenses (8 cores multiplied by a factor of .50 equal 4 processor licenses).
Sun SPARC64 VI CPU (* different models of Sun CPUs may have different core factors)
1 eight core SPARC64 VI CPU requires 6 processor licenses (8 cores multiplied by a factor of .75 equal 6 processor license).
IBM Power6 CPU (* different models of IBM CPUs may have different core factors)
1 eight core Power6 CPU requires 8 processor licenses (8 cores multiplied by a factor of 1.0 equal 8 processor licenses)
As illustrated in the above examples, a single eight core CPU doubled the Oracle technology license CPU count, when compared to
the single four cores CPU in List 1. Oracle customers using processor licensing will have to carefully consider the licensing impact of
a hardware refresh due to the additional CPU cores.
List 3 highlights various options to help manage the Oracle technology license CPU count, with multi-core CPUs.
One of the options is to move from processor licensing to an unlimited license agreement. Customers with an unlimited license
agreement (ULA) have no CPU restrictions with Oracle technology products.
Another option would be to use hard partitioning with processor licensing to control the number of licensed CPUs. Hard
partitioning allows a customer to license a subset of a server’s CPUs. However, Live Migration is not supported with hard
partitioning.
Customers can also use soft partitioning with processor licensing and Oracle VM to limit the number of licensed CPUs within a
server pool. Soft partitioning supports Live Migration.
Hard and Soft Partitioning with Oracle VM

Oracle recognizes both hard and soft partitioning for Oracle technologies with Oracle VM. Hard and soft partitioning with Oracle VM
can be used with processor licensing and enterprise edition products to manage the number of licensed CPUs, for development and
production environments.
Note: Standard edition products can run on an Oracle VM as long as the Oracle VM server meets the standard edition’s CPU
requirements. Please refer to the relevant licensing documentation for the Standard Edition product in question to verify if
the Standard Edition product can be hosted on your server platform with Oracle VM.
The difference between hard and soft partitioning is how Oracle recognizes the Oracle technology CPU license count, and the
supported virtualization feature set. For example, Live Migration is not supported with Oracle VM when used with hard partitioning.
Conversely, soft partitioning can be used within an Oracle VM pool to take advantage of Live Migration, along with the ability to
manage the Oracle technology CPU count.
From an Oracle technology licensing perspective, hard partitioning allows customers to license a subset of a server’s CPUs.
Conversely, soft partitioning counts the total number of a server’s CPUs. Soft partitioning with Live Migration requires each Oracle
VM server, running a guest with an Oracle technology product to be licensed. We can limit the number of soft partitioned pool
members, where a guest can run, by configuring an Oracle VM Manager manual placement policy.
Table 1 provides an overview of hard and soft partitioning.

Overview Requirements
Hard Hard partitioning 1. All hard partitioned guests

Partitioning allows a must pin the virtual CPUs to the
37 of 243 18.04.2011 16:35

customer to Oracle VM server’s physical CPU

license a subset cores in the guest’s vm.cfg file.
of a server’s 2. All hard partitioned guests
CPUs. must have an Oracle VM Manager
manual placement policy to
confine the guests to the pinned
Oracle VM server(s).
3. All hard partitioned guests
“cannot” use Live Migration.
Soft Soft partitioning 1. Each Oracle VM server

Partitioning requires the sum running a guest with an Oracle
of an Oracle VM technology product must be
server’s CPU licensed. We can use a manual
cores to be placement policy to license a
licensed. subset of pool member servers.
For example, in a 10 server pool,
you could license 2 of the 10 pool
members.
List 4 shows three hard and soft partitioning examples with Oracle technology licensing.
1. A single Intel server with 16 CPU cores, with Linux installed running 11G has a processor factor of 8 CPUs. The Linux server can
run one 11G instance.
2. A single Intel server with 16 CPU cores, with Oracle VM installed using soft partitioning has a processor factor of 8 CPUs. The
Oracle VM server could run more than 16 single CPU guests each with 11G.
3. A single Intel server with 16 CPU cores with Oracle VM installed using hard partitioning. The Oracle VM server is capable of
running more that 16 single CPU guests, although only one of the guests is running 11G with 2 virtual CPUs. In this example,
using hard partitioning, we could license a subset of the 8 CPUs. For example, we could hard partition 1 of the 8 CPUs.
A single Oracle VM server with 2 Intel eight core CPUs (16 cores), could run 16 one CPU guests, without oversubscribing the servers’
CPUs. Oracle VM supports both CPU and memory oversubscription, which allows a single Oracle VM server to oversubscribe CPU and
memory resources to guests. For example, an oversubscribed host with 2 Intel eight core CPUs (16 cores), could provision more than
16 cores to guests.
Figure 1 shows three hosts. The first host has two eight core CPUs with Linux installed running 11G. The second host has two eight
core CPUs using soft partitioning with Oracle VM installed, hosting 8 guests running 11G. The third host has two eight core CPUs
using hard partitioning with Oracle VM installed, hosting 8 guests. Only one of the guests is running 11G.
As shown in Figure 1, the Linux server requires eight Oracle technology CPU licenses and is hosting one 11G application. Servers that
host one application are commonly referred to as application silos. The traditional one application per server deployment
methodology, shown in Figure 1, inevitably leads to over-provisioning and underutilization of hardware. Studies show that most
servers run at 5-15% of their total capacity. For example, most servers spend the majority of their life idle, consuming electricity and
taking up valuable data center space. Underutilized servers can be consolidated using Oracle VM with hard or soft partitioning to
provide better license and resource utilization when compared to application silos.
The soft partitioning example in Figure 1 shows how an Oracle VM server with eight processor licenses can host multiple isolated
guests, running 11G, for the same CPU cost as the application silo. Oracle VM with soft partitioning provides superior license and
resource utilization when compared to an application silo. Oracle VM supports CPU and memory oversubscription, which allows you
to run even more workloads per server when compared to an application silo.
The hard partitioning example in Figure 1 shows how a shared infrastructure can be used to support Oracle technology products
along with the ability to license a subset of the server’s CPUs. For example, we can license one of the eight CPUs. Hard partitioning
with Oracle VM can be used with processor licensing and enterprise edition products to manage the number of licensed CPUs for
38 of 243 18.04.2011 16:35

development and production environments.
Soft Partitioning Examples

Soft partitioning with Oracle VM can be used with processor licensing for both development and production environments. The use
case for Oracle VM and soft partitioning with development environments is to consolidate application silos to a shared Oracle VM
infrastructure. Migrating from application silos to a shared infrastructure can help reduce the total number of licensed CPUs, reduce
electricity consumption, consolidate underutilized server, and free up data center space.
The use case for Oracle VM and soft partitioning with production environments, is the ability to use Live Migration, along with the
ability to manage the number of licensed CPUs. For example, it is not necessary to license the sum of all Oracle VM pool member’s
CPU cores when using Live Migration with soft partitioning. We can configure an Oracle VM Manager manual placement policy to
control which pool members a guest can run on. Using a placement policy with soft partitioning allows us to license a subset of an
Oracle VM pool’s CPU cores. A manual placement policy confines a guest to run on the pool members listed in the manual placement
policy.
Figure 2 shows an Oracle VM server pool with six Oracle VM servers. Each Oracle VM server has two eight core CPUs. The Oracle
VM server pool has a total of 96 cores, and a processor factor of 48 CPUs. In Figure 2, there is a total of 8 guests in the pool running
11G, with the ability to run on all 6 Oracle VM pool members. The example shown in Figure 2 would require 48 Oracle technology
processor licenses.
Figure 3 shows the same server pool as in Figure 2, with a total of 96 cores and a processor factor of 48 CPUs. In Figure 3, there is a
total of 8 guests in the pool running 11G, with the ability to run on two Oracle VM pool members. The scenario shown in Figure 3
requires 16 Oracle technology processor licenses.
39 of 243 18.04.2011 16:35

Figure 4 shows the same server pool as in Figure 2, with 96 cores, and a processor factor of 48 CPUs. In Figure 4, there is a total of
96 guests in the pool, each running 11G, with the ability to run on any of the Oracle VM pool members. The scenario shown in Figure
4 would require 48 Oracle technology processor licenses.
We can limit the number of Oracle VM pool members that a guest can run on, by configuring a manual placement policy in Oracle VM
Manager. A manual placement policy allows you to limit which Oracle VM pool members a guest is allowed to run on. Once a manual
placement policy is configured, HA events and Live Migration will be limited to the Oracle VM pool members listed in the manual
placement policy.
Tip: An auto placement policy will start a guest on the least busy pool member and does not limit a guest’s ability to HA or Live
Migrate to any pool members.
Hard Partitioning Examples

Hard partitioning, also referred to as sub-capacity licensing, allows Oracle customers to license a subset of a server’s CPUs. Hard
partitioning is a two step process. The first step is to create a manual placement policy. The manual placement policy will confine the
guest to the pinned Oracle VM server. The second step, is to edit the hard partitioned guest’s vm.cfg file, to pin the guest’s virtual
CPU to the Oracle VM server’s physical CPU cores.
40 of 243 18.04.2011 16:35

Oracle’s hard partitioning policy states that a hard partitioned guest’s virtual CPUs mapping must be hardcoded in the guest’s vm.cfg
file. To confine the hard partitioned guest to the mapped host, a manual placement policy must be configured. Oracle restricts the use
of Live Migration with hard partitioning.
Our first hard partitioning example in Figure 5 shows an Oracle VM server with two eight core Intel CPUs, with one hard partitioned
guest running 11G. The guest is pinned to two of the Oracle VM server’s cores (2 cores = 1 CPU). From an Oracle technology
licensing perspective, the server has a processor factor of 8. Using hard partitioning, the pinned guest would require only 1 CPU
license. The additional 7 CPUs could be used to license other Oracle technologies or be shared to run other workloads on the same
Oracle VM server.
Another example with the server in Figure 5 would be to hard partition two guests, each guest running 11G. Each of the two guests is
pinned to 1 of the Oracle VM server’s cores (2 cores = 1 CPU). The additional 7 CPUs could be used to license other Oracle
technologies or be shared to run other workloads on the same Oracle VM server.
Figure 6 shows an Oracle VM server with two eight core Intel CPUs, with two hard partitioned guests running 11G. Each guest is
pinned to 1 of the Oracle VM server’s cores, 2 cores = 1 CPU.
Hard partitioning an Oracle VM guest is a two step process. The first step is to create a manual placement policy for the hard
partitioned guest using Oracle VM Manager. The manual placement policy will confine the guest to the pinned Oracle VM server. The
second step is to edit the hard partitioned guest’s vm.cfg file to pin the guest’s virtual CPU to the Oracle VM server’s physical CPU
cores.
Oracle VM Manager Manual Placement Policy Configuration

The ability to limit a guest to an Oracle VM server is accomplished by configuring an Oracle VM Manager manual placement policy. A
manual placement policy allows you to configure which Oracle VM pool members a guest is allowed to run on. Once a manual
placement policy is configured, HA events and Live Migration will be limited to the pool members listed in the manual placement
41 of 243 18.04.2011 16:35

policy. A manual placement policy is a guest property that can be configured during or after guest creation.
In the next section, we will walk through the configuration of a manual placement policy. Please note that guests must be powered off
to configure a placement policy.
The first step is to access Oracle VM Manager and power off the guest. Next, click on the guest’s Virtual Machine Name as shown
in Figure 7 to access the guest’s properties.
From the General Information page, click the Policies link to access the Policies properties page, as shown in Figure 8.
From the Policies page click the Placement Policy tab, as shown in Figure 9.
From the Placement Policy page click the Manual button to access the Prefer Server page, as shown in Figure 10.
42 of 243 18.04.2011 16:35

From the Preferred Server page, select the desired Oracle VM server(s) from the preferred server list. When you select an Oracle
VM server from the preferred server list, the manual placement policy will limit the guest to the selected server(s). Once you have
selected the preferred server, click the Confirm button, as shown in Figure 11.
After clicking the Confirm button, the page refreshes and displays the Placement Policy page. The new manual placement policy
will be displayed as shown Figure 12.
We have successfully configured a manual placement policy for an Oracle VM hard partitioned guest.
The next and final step to hard partition an Oracle VM guest is to pin the guest’s virtual CPUs to the Oracle VM server’s CPU cores.
Each hard partitioned guest should be pinned to the Oracle VM server that is listed in the guest’s manual placement policy.
Hard Partitioning an Oracle VM Guest

This section will start with a brief review of the credit scheduler. Next, we walk through the procedure to hard partitioning an Oracle
VM guest by adding the “cpus=” directive in a guest’s vm.cfg file. We conclude the section with CPU pinning examples, using the xm
and virsh commands.
Oracle VM’s default CPU scheduler is the credit scheduler. The credit scheduler uses a credit/debit system to fairly share CPU
resources between guests. Credits are assigned to each running guest, along with the fraction of CPU resources. The credit scheduler
continually increments/decrements credits from running guests, which is how the credit scheduler balances resources. In many ways,
the credit scheduler is like the Linux scheduler. The Linux scheduler is used as the default CPU scheduler with the KVM hypervisor.
Both schedulers can preempt processes as needed while trying to ensure proportional fair share allocations.
The default behavior of the credit scheduler is to bind each virtual CPU to a separate physical core. For example, when you create a
guest with two virtual CPUs, the credit scheduler will map the two virtual CPUs to two physical cores. So when pinning virtual CPUs,
we should follow the credit scheduler’s default behavior of mapping virtual CPUs to a server’s individual CPU cores.
43 of 243 18.04.2011 16:35

Unless you have pinned a guest’s virtual CPUs, virtual CPUs will occasionally bind to different physical cores. Virtual CPUs bind to
different physical cores, due to the credit scheduler’s use of the credit/debit system, which dynamically re-balances CPU resources.
For example, if you where to periodically check an unpinned guest’s CPU mapping, you would see a different CPU mapping
throughout the day.
There are two methods to pin virtual CPUs. We can use the xm command to pin a guests’s virtual CPUs or we can hardcode the CPU
mapping in a guest’s vm.cfg file. The difference between pinning CPUs with xm and hard coding the CPU mapping in a guest’s vm.cfg
file is the persistence of the CPU mapping. CPUs that are pinned with xm are not persistent between reboots. Hard coding the CPU
mapping in a guest’s vm.cfg file is persistent between reboots. To comply with Oracle’s hard partitioning policy, we must hardcode the
CPU mapping in a guest’s vm.cfg file.
Please note that hard partitioning could cause guest performance issues. For example, if you pin a guest’s virtual CPU to a specific
subset of named CPUs without considering how the lower-level I/O interrupts are being assigned, you can end-up hurting
performance. I/O interrupts are typically mapped to a specific CPU. If that CPU is not the same as the pinned CPU, the interrupts have
to be "re-directed" to the CPU you pinned, which could cause the performance of the guest to decrease. If a hard partitioned guest is
experiencing performance issues, the CPU pinning would be an area to investigate.
Next, we will review how to hard partition an Oracle VM guest. After the hard partitioning example, we will review pinning an Oracle
VM guest using the xm command. Unfortunately, all CPU cores are not equal, so you may need to test various virtual CPU mappings
using the xm and virsh commands.
CPU Pinning Examples

In the following example, we will hard partition a guest running Oracle Database 10g enterprise edition with two virtual CPUs. Two
virtual Intel or AMD CPUs equal one Oracle technology CPU. The guest will be pinned to an Oracle VM server with two four core
CPUs. The Oracle VM server has a processor factor of four Oracle technology CPUs. Using hard partitioning, we will license only one
of the Oracle VM server’s four licensable CPUs.
To comply with Oracle’s hard partitioning policy, we must hardcode a guest’s virtual CPU mapping, by adding the “cpus=” directive in
the guest’s vm.cfg file. By adding the “cpus=” directive in the guest’s vm.cfg file, we pin the guest’s virtual CPUs to the Oracle VM
server’s cores.
Let’s review two different “cpus=” configurations, to help explain how to pin a guest’s virtual CPUs to an Oracle VM server’s CPU
cores.
In the first vm.cfg example, we add a new line in the vm.cfg file, cpus = '0-3'.The cpus = '0-3' entry pins the guest’s virtual CPUs to
the Oracle VM server’s CPU cores 0, 1, 2, and 3.
Please note the vcpus = 4 entry, one line above the cpus = '0-3' entry. The vcpus = 4 entry defines the number of virtual CPUs. The
vcpus = directive can be edited to select the desired number of virtual CPUs.
#vi /OVS/running_pool/v52x6410g1/vm.cfg
bootloader = '/usr/bin/pygrub'
disk = ['file:/OVS/running_pool/v52x6410g1/System.img,xvda,w',
'file:/OVS/running_pool/v52x6410g1/oracle10g_x86_64_asm.img,xvdb,w',
]
memory = '2048'
name = 'v52x6410g1'
on_crash = 'restart'
on_reboot = 'restart'
uuid = 'd428ba07-31b9-5667-2085-8753a0342425'
vcpus = 4
cpus = '0-3'
vfb = ['type=vnc,vncunused=1,vnclisten=0.0.0.0']
vif = ['bridge=xenbr0,mac=00:16:3E:20:18:19,type=netfront']
vif_other_config = []
The above example vm.cfg file shows a hard partitioned guest with 4 virtual CPUs. The guest’s 4 virtual CPUs are pinned to the
Oracle VM server’s CPU cores 0, 1, 2, and 3. The same guest could also be pinned using cpus = '0' in the vm.cfg file. Using cpus =
'0' would pin all 4 virtual CPUs to the same physical core, number 0, on the Oracle VM server. The same guest could also be pinned
using cpus = '0,1' in the vm.cfg file. Using cpus = '0,1’ would pin 2 virtual CPUs to core number 0 and 2 virtual CPUs to core
number 1.
We can also use regular expression inversion. For example, we could use cpus=’^0-1’, which means any core but 0 and 1.
In the second vm.cfg example, we add a new line in the vm.cfg file, cpus = '0,1'.The cpus = '0,1' entry pins the guest’s virtual CPUs
to the Oracle VM server’s CPU cores 0 and 1.
Please note the vcpus = 2 entry above the cpus = '0,1' entry. The vcpus = 2 entrydefines the number of virtual CPUs. The vcpus =
directive can be edited to select the desired number of virtual CPUs.
]
44 of 243 18.04.2011 16:35

memory = '2048'
name = 'v52x6410g1'
uuid = 'd428ba07-31b9-5667-2085-8753a0342425'
vcpus = 2
cpus = '0,1'
The above example vm.cfg file shows a hard partitioned guest with 2 virtual CPUs pinned to the Oracle VM server’s CPU cores 0 and
1.
We can also use regular expression inversion. For example, we could use cpus=’^0-1’, which means any core but 0 and 1.
Note: We must reboot the virtual machine to enforce any new hard partitioning configurations.
To be able to hard partition a guest, we need to know the number of CPUs and the number of cores on the pinned Oracle VM server.
There are a number of commands to list the CPU details of an Oracle VM server. From dom0 we could type “xm info” or “virsh
nodeifo” to list the CPU and core details as shown in the next example.
# virsh nodeinfo
libvir: Remote error : No such file or directory
libvir: warning : Failed to find the network: Is the daemon running ?
CPU model: i686
CPU(s): 8
CPU frequency: 2992 MHz
CPU socket(s): 2
Core(s) per socket: 4
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 16775168 kB
The “virsh nodeinfo” example shows that the Oracle VM server has two four core CPUs (sockets) with a total of eight cores. The
example Oracle VM server has an Oracle technology license processor factor of 4 CPUs.
To list the CPU cores, we can type “grep -i processor /proc/cpuinfo”, as shown in the next example.
# grep -i processor /proc/cpuinfo
processor :0
processor :1
processor :2
processor :3
processor :4
processor :5
processor :6
processor :7
The “grep -i processor /proc/cpuinfo” example lists the number of all eight CPU cores. If you would like to list all of the CPU details
type “cat /proc/cpuinfo”.
Once we have the Oracle VM server’s CPU and core details, we can pin the guest’s virtual CPUs to any of the physical cores. We will
follow the default behavior of the credit scheduler and bind each virtual CPU to a separate physical core.
Before we pin the guest, let’s review the guest’s vm.cfg file. Please note the vcpus = 2 directive, which indicates the number of
virtual CPUs for the guest.
]
memory = '2048'
name = 'v52x6410g1'
uuid = 'd428ba07-31b9-5667-2085-8753a0342425'
vcpus = 2
Next, we will pin the two virtual CPUs to core 7 and 3 on the Oracle VM server. We will add the cpus =’7,3’ directive to pin the
guest’s two virtual CPUs to core 7 and 3 on the Oracle VM server.
45 of 243 18.04.2011 16:35

]
memory = '2048'
name = 'v52x6410g1'
uuid = 'd428ba07-31b9-5667-2085-8753a0342425'
vcpus = 2
cpus =’7,3’
We must reboot the virtual machine to enforce the new hard partitioning configuration.
Note: If a hard partitioned guest uses Live Migration, or has the CPU properties edited with the xm command, the hard coded CPU
mapping in the vm.cfg file will be lost. If the CPU mappings get removed by Live Migration, or xm, you will need to re-pin the virtual
CPUs in the guest’s vm.cfg file.
Once we reboot the guest we can validate our new hard partition configuration by accessing dom0 as root and type xm vcpu-list
[domain], as shown in the next example.
# xm vcpu-list v52x6410g1
Name ID VCPU CPU State Time(s) CPU Affinity
v52x6410g1 19 0 7 -b- 12.8 3,7
v52x6410g1 19 1 3 -b- 5.6 3,7
The “xm vcpu-list v52x6410g1” command validates that our hard partition configuration is enforced. By adding the cpus =”7,3”
directive, we pinned the guest’s two virtual CPUs, one virtual CPU is pinned to core 7 and one virtual CPU is pinned to core 3.
The above hard partition example showed how to license a subset of an Oracle VM servers’ CPUs. The example Oracle VM server has
a processor factor of four CPUs. Using hard partitioning, we licensed only one of the Oracle VM server’s four licensable CPUs.
CPU Pinning with xm

We can also manage the number of virtual CPUs for a running guest using the “xm vcpu-set” command. Using the “xm vcpu-set”
command allows us to test and troubleshoot virtual CPU mappings. Please note that using the “xm vcpu-set” command to pin virtual
CPU is not recognized by Oracle for hard partitioning.
Note: If you have hard coded the CPU mapping in a guest’s vm.cfg file and use the xm command to change the CPU properties, the
hard coded CPU mapping will be lost.
The “xm vcpu-set” command allow us to select any number of virtual CPUs up to number of virtual CPUs listed in the vcpus = n
directive in the vm.cfg file. For example, if a guest has four virtual CPUs, (vcpus = 4) we could use the “xm vcpu-set” command to
reconfigure a guest to use 1, 2 or all four of the virtual CPUs.
To view a guest’s virtual CPU statistics, from dom0 as root, type xm vcpu-list [domain], as shown in the next example. If you type “xm
vcpu-list”, it will list all of the running guest’s virtual CPU statistics.
v52x6410g1 18 0 2 -b- 351.4 2
v52x6410g1 18 1 6 -b- 220.7 6
We can also use the virsh command to list a guest’s virtual CPU details by typing virsh vcpuinfo [domain], as shown in the next
example.
# virsh vcpuinfo v52x6410g1
libvir: Remote error : No such file or directory
libvir: warning : Failed to find the network: Is the daemon running ?
VCPU: 0
CPU: 2
State: blocked
CPU time: 236.2s
CPU Affinity: ---y---y
VCPU: 1
CPU: 6
State: blocked
CPU time: 178.5s
CPU Affinity: ---y---y
In the above example, the guest has two virtual CPUs, 0 and 1. Virtual CPU 0 is in the "blocked" state on the physical core number 2.
Virtual CPU 1 is in the "blocked" state on the physical core number 6. Both virtual CPUs are in the blocked state, which means the
guest is waiting on I/O or has gone to sleep. There is a total of six virtual CPU states, r for running, b for blocked, p for paused, s for
shutdown, c for crashed and finally, d for dying.
The next example shows how to change the virtual CPU count from two virtual CPUs to one virtual CPU using the “xm vcpu-set”
46 of 243 18.04.2011 16:35

command.
# xm vcpu-set v52x6410g1 1
v52x6410g1 18 0 2 -b- 359.5 2
v52x6410g1 18 1 - --p 227.0 6
As shown in the above example, typing “xm vcpu-set v52x6410g1 1” paused one of the two virtual CPUs. A paused virtual CPU is not
eligible for scheduling by the credit scheduler. The paused virtual CPU will remain paused until resumed, for example, by typing “xm
vcpu-set v52x6410g1 2”, as shown in the next example.
# xm vcpu-set v52x6410g1 2
v52x6410g1 19 0 2 -b- 266.4 2
v52x6410g1 19 1 6 -b- 190.5 6
Next, we will pin the guest’s virtual CPUs to the Oracle VM server’s physical cores using the “xm vcpu-pin <domain> <vcpu>
<pcpu>” command. In the next example, we will pin the guest’s virtual CPU 0 to core 1, and virtual CPU 1 to core 4.
# xm vcpu-pin v52x6410g1 0 1
# xm vcpu-pin v52x6410g1 1 4
v52x6410g1 19 0 1 -b- 268.4 4
v52x6410g1 19 1 4 -b- 190.5 6
As shown in the above example, typing “xm vcpu-pin v52x6410g1 0 1” followed by typing “xm vcpu-pin v52x6410g1 1 4” pinned the
guest’s virtual CPU 0 to core 1 and virtual CPU 1 to core 4.
Resources:
The Oracle Software Investment Guide (SIG)

Hard Partitioning with Oracle VM
Red Hat Knowledgebase: How do I determine if my x86-compatible Intel system is multi-processor, multi-core or supports
hyperthreading?
Watch this space

Chapter 4: Oracle VM Server Sizing, Installation and

Updates
Chapter 4 covers the following topics:
Introduction to the installation options for Oracle VM server

Oracle VM Server sizing and resource requirements
Walkthroughs of two Oracle VM server CD-ROM installations
Configuring a boot server for an Oracle VM server PXE/kickstart installation
How to create the PXE boot and kickstart files for an automated Oracle VM server installation
Overview of how to update an Oracle VM server from the Unbreakable Linux Network (ULN)
Table of Contents
Oracle VM Server Installation Options
Oracle VM Media Pack Download
Oracle VM Server Sizing and Resource Requirements
…High-Availability and Disaster-Recovery Requirements
…Separation of Duties
…Oracle VM Server CPU Requirements
…Oracle VM Server Memory Requirements
…Oracle VM Server Storage Requirements
…Oracle VM Server Network Requirements
47 of 243 18.04.2011 16:35

…Oracle VM Server Boot From SAN Requirements

CD-ROM Oracle VM Server Pool Member Installation
…Oracle VM Server Boot From SAN
…Installing Oracle VM Server From Other Sources
…Post Installation Checklist
CD-ROM Oracle VM Server Installation with Local Storage
…Installing Oracle VM Server From Other Sources
…Post Installation Checklist
Oracle VM Server PXE/Kickstart Installation
...HTTP Services Configuration
......How to Make the Oracle VM Server Installation Tree Available for a PXE/Kickstart Installation
...DHCP Service Configurations
...TFTP Service Configurations
How to Create the PXE boot Files
Kickstart File Examples
...How to Generate an Encrypted Password with grub-md5-crypt
...How to Stage a Kickstart File for an Oracle VM Server PXE/kickstart Installation
...PXE Boot Client Configuration
...PXE/Kickstart Oracle VM Server Installation and Troubleshooting Tips
How to Register and Update an Oracle VM Server from the Oracle Unbreakable Linux Network
...The Oracle Unbreakable Linux Support Program
...Oracle VM Server Update Options
...up2date Program Proxy Configurations
...up2date Program Kernel Configurations
...Import Oracle’s GPG Key
...The Oracle Unbreakable Linux Network Registration Process
…Update an Oracle VM Server Using the up2date Program and the Oracle Unbreakable Linux Network
Oracle VM Server Installation Options

Oracle VM server can be installed either from a CD-ROM or over the network using a pre-boot execution environment (PXE). To install
Oracle VM server using the CD-ROM or PXE boot method, you will need to visit the Oracle eDelivery/Linux site to download the
Oracle VM Media Pack. The Oracle VM Media Pack is delivered as a zip file that contains the Oracle VM server ISO file. The Oracle
VM server ISO file can be burned as a bootable disk and used to install Oracle VM server from a CD-ROM drive.
Note: Oracle VM Manager is not supported and should not be installed in Oracle VM server's dom0.
The zip file from the Oracle eDelivery/Linux site contains a single Oracle VM server ISO file; this file is used for both x86 and x86-64
hardware. The Oracle VM server install routine automatically detects the hardware platform and installs the appropriate 32-bit or
64-bit Xen hypervisor. Regardless of the hardware platform, 32-bit or 64-bit, dom0 is 32-bit.
A PXE boot installation requires several additional steps; for example, a boot server and a kickstart file to automate the Oracle VM
server installation must be created. The boot server allows a bare-metal system to automatically receive an IP address via DHCP, load
a kernel via TFTP, and then boot without an installed operating system. Once the bare-metal server boots, you can install Oracle VM
server from the installation media or use a kickstart file to automate the Oracle VM server installation.
The default behavior of the Oracle VM server installer is to install Oracle VM server on the Oracle VM server's local disk. To enable
the boot from SAN option, type “boot: linux mpath [enter]” at the installation boot prompt. The installation boot prompt is visable
on the first Oracle VM Server installation screen. Typing “boot: linux mpath [enter]” tells the installer to use the device-mapper-
multipath drivers.
Oracle VM server can be installed directly from a bootable CDROM, as well as from the Oracle VM server media files that have been
staged on a) an Oracle VM server's local hard drive b) on an NFS share c) on an FTP server and/or d) on a web server. The Oracle VM
madia files are also refered to as the Oracle VM server media installation tree. To install Oracle VM server from the Oracle VM media
files, from the installation boot prompt type "boot: linux askmethod [enter]. Typing "linux askmethod [enter] from the installation
boot prompt will enable the Install Method installation screen. From the Install Method screen you can select to install Oracle VM
server from a) Local CDROM b) Hard Disk c) NFS image d) FTP or e) HTTP.
Tip: Installing Oracle VM server using a bootable CDROM with Lights out Management (LOM) solutions may generate file copy
installation errors. If you experience file copy errors, stage the Oracle VM server media files on the a) Hard Disk b) NFS image c) FTP
or d) HTTP and from the installation boot prompt type "boot: linux askmethod [enter] to enable the Install Method installation
screen.
Oracle VM Media Pack Download

To download the Oracle VM Media Pack, point your browser to the Oracle eDelivery/Linux site and fill out the registration/export
regulations form to gain access to the Oracle VM Media Pack. Once the registration/export regulation form is complete, you be
redirected to the Media Pack Search page. From the Media Pack Search page, select Oracle VM from the Select a Product
Pack dropdown menu, then select x86 32-bit or x86 64-bit from the Platform dropdown menu. Note that selecting x86 32-bit or
x86 64-bit will take you to the same download page; there is only one media pack for both x86 32-bit and x86 64-bit platforms. Click
the Go button to be taken to the Oracle VM Media Pack download page.
Figure 1 shows the result from the Media Pack Search for Oracle VM.
48 of 243 18.04.2011 16:35

Select the desired Oracle VM Media Pack, then press the Continue button or click the Oracle VM <version> Media Pack
hyperlink to go to the download page. On the Oracle VM <version> Media Pack download page, click the Download button to
download the Oracle VM Server <version> media pack.
Figure 2 shows the Oracle VM Media Pack download page.
The Oracle VM Server media is delivered as a zip file. The zip file name corresponds to the Part Number listed on the download
page. Once the zip file is downloaded, use your favorite zip utility to unzip the Oracle VM ISO file. Next, burn the ISO file to a
bootable CD or DVD that can be used to install Oracle VM server using a CD-ROM.
Oracle VM Server Sizing and Resource Requirements

This section starts with an introduction to the resource requirements for Oracle VM server. The section concludes with a discussion of
Oracle VM server CPU, memory, storage, and networking considerations.
Oracle VM server runs on x86 32-bit and x86 64-bit platforms with Intel or AMD chips. The minimum resource requirement for your
Oracle VM hardware depends on the resource requirements of the guests that will run on your Oracle VM servers. For example,
Oracle recommends a dual core CPU or multiple CPUs with at least 1GB or 2GB of RAM. Oracle’s minimum resource recommendation
for Oracle VM is a great starting point for running a couple guests for an evaluation. To size your Oracle VM server hardware and
Oracle VM server pools, you will need document a) the resource requirements of all of your virtual machines b) your organization’s
high-availability and disaster-recovery requirements and c) your organization’s separation of duty requirements for the Oracle VM
users and groups.
To size your Oracle VM hardware, first calculate the CPU, memory, and storage requirements for all of your guests. Understanding the
CPU, memory, and storage requirements for all of your guests allows you to accurately determine the CPU, memory, and storage
requirements for your Oracle VM servers and Oracle VM server pools.
Consider, for example, the case of virtualizing one 11g database that requires 16 CPUs, 128GB of memory, and 1TB of storage. The
11g database guest could run on a single Oracle VM server with 4 quad core CPUs (16 cores), 132GB of memory and 1TB of local or
remote storage. The Oracle VM server in the example would run the 11g database guest without oversubscribing CPU or memory
resources. Please note that dom0 requires a minimum of 512MB of memory, therefore the Oracle VM server in the example allocated
132GB of memory for the virtual machine plus 4GB for the overhead in dom0. If the example Oracle VM server uses local storage, it
would be a single server Oracle VM pool. If the Oracle VM server uses a shared storage repository, the Oracle VM server could be a
pool member in a multiserver pool.
Tip: An Oracle VM Manager placement policy can be used to restrict the 11g guest to a specific Oracle VM server.
To virtualize two 11g databases, each database with 16 CPUs and 128GB of memory with 1TB of storage, the two 11g database
guests could both run on a single Oracle VM server with 4 six core CPUs (24 cores), 260GB of memory, and 2TB of local or remote
storage. Running two 11g database guests each with 16 CPUs and 128GB on a single server with 4 six core CPUs (24 cores) would
49 of 243 18.04.2011 16:35

oversubscribe the Oracle VM server by 8 cores, which may not be an option if your database workload is CPU bound. An example of
not oversubscribing CPUs with two 11g database guests would be to run each guest on a dedicated Oracle VM server with 4 quad
core CPUs, 132 GB of memory, and 1TB of storage.
High-Availability and Disaster-Recovery Requirements

To help calculate the numbers of guests, Oracle VM servers, and Oracle VM server pools for your Oracle VM environment, you will
need to understand your organization’s high-availability and disaster-recovery requirements. High availability is an integral data
center strategy that allows organizations to meet service level agreements (SLAs) by minimizing or eliminating planned and
unplanned downtime. An SLA will specify the levels of availability that determine which high-availability strategy will allow you to
meet your availability SLA. For example, a mission-critical application would have an SLA that requires operational continuity in the
event of system failure. Conversely, a non-mission-critical application might have an SLA that allows several hours of downtime.
Oracle offers a wide variety of high-availability solutions for databases, applications, and operating systems that offer different levels
of availability. For example, RAC is Oracle’s database high-availability solution that offers operational continuity in the event of node
failure. Oracle DataGuard and Oracle ApplicationGuard are two other Oracle high-availability solutions that offer operational
continuity in the event of failure.
Oracle VM has two high-availability features; a) guest HA and b) Live Migration. Oracle VM HA automatically restarts guests when;
a) a guest hangs or b) when an Oracle VM pool member fails or restarts. Oracle VM HA minimizes unplanned downtime by restarting
guests. Live Migration is used to eliminate planned downtime by migrating running guests from one Oracle VM pool member to
another during a maintenance event, for example, for repairs or an upgrade. Both HA and Live Migration require a pool configuration
with a minimum of two Oracle VM servers with sufficient memory to run all the guests on one host and a shared storage repository.
An organization’s high-availability and disaster-recovery requirements will directly affect the numbers of guests, Oracle VM servers,
and Oracle VM server pools required to meet your availability SLA. For example, a corporate policy that states that a mission-critical
database requires operational continuity will require a clustering solution such as RAC. From an Oracle VM perspective, supporting a
database that requires operational continuity requires one dedicated Oracle VM server per RAC node for production environments. If
disaster recovery is a requirement for an Oracle VM environment, the number of Oracle VM servers at the disaster-recovery site
would be the minimum number of Oracle VM servers required to run all the guests in the event of an outage of the primary data
center.
Separation of Duties
Separation of duties is also a consideration that affects the number of Oracle VM servers and Oracle VM server pools in an Oracle VM
environment. For example, many organizations require separation of duties between development and production environments or
separation of duties based on geography. Oracle VM supports role-based access control that can be used to isolates Oracle VM
resources such as guests and Oracle VM pools based on user and group membership. Oracle VM role-based access control can isolate
access to guests within a pool or isolate access to an entire a pool based on user and group membership. For example, role-based
access controls could be created for a group named “Development”; these controls could restrict access to resources such as guests,
ISO files, templates, and Oracle VM pools used by the Development group.
Isolating resources within a pool, for example, isolating guests for groups a and b, would not affect the number of Oracle VM servers
within a pool. But isolating resources at the Oracle VM pool level, for example creating a production pool for the Production group
and development pool for the Development group, would require dedicated Oracle VM servers for each group, which, in turn, would
require additional Oracle VM servers and Oracle VM pools.
Understanding your organization’s separation of duties requirements will help to accurately determine the total number of Oracle VM
servers and Oracle VM pools within your Oracle VM environment.
Oracle VM Server CPU Requirements

Oracle VM server supports two modes of virtualization, paravirtualization mode and hardware-virtualization mode. To support
hardware-virtualization mode, the Oracle VM server requires either an Intel CPU with VT (virtualization technology) extensions or an
AMD CPU with SVM extensions (also called AMD‐V). The system BIOS settings on the Oracle VM server will need to be edited to
enable hardware-virtualization support. Hardware-virtualization support is disabled by default. If the CPU does not support hardware
virtualization, paravirtualization can be used for Red Hat Enterprise Linux and Oracle Enterprise Linux guests.
To support Live Migration between Oracle VM pool members, each pool member should have CPUs of the same CPU family and
model. Attempts at Live Migration between two Oracle VM servers with CPUs that are not of the same CPU family and model may fail.
Although the CPUs should be of the same family and model to support Live Migration, each Oracle VM server may have a different
number of sockets and cores.
Tip: To validate the CPU family and model of an Oracle VM server’s CPUs, view /proc/cpuinfo on your Oracle VM server.
To effectively size the number of CPUs and cores for an Oracle VM server, the first step is to count the total number of guest CPUs.
For example, when a guest is allocated a virtual CPU, the virtual CPU is actually a physical CPU core allocated from an Oracle VM
server. A guest with eight virtual CPUs will be allocated eight CPU cores from an Oracle VM server.
Oracle VM server supports oversubscribing CPUs, which means that a single Oracle VM server can overallocate its CPU cores. For
example, an Oracle VM server with four 6 core CPUs (24 cores) could allocate more than 24 cores to guests. Oversubscribing CPUs
with CPU-bound workloads, such as the Oracle Database or RAC, can quickly lead to nasty performance problems. Oversubscribing
CPUs should be used with workloads that are not CPU bound to allow greater utilization of the Oracle VM server hardware. For
example, the Oracle SOA Suite is an Oracle application that is not CPU bound. Oversubscribing CPUs with Oracle SOA Suite guests
would allow greater utilization of the Oracle VM server hardware.
Oracle VM Server Memory Requirements
50 of 243 18.04.2011 16:35

By default, each Oracle VM server reserves 512MB of memory for dom0. The average memory overhead for each running guest on a
dom0 is approximately 20MB plus 1% of the guest’s memory size. The remaining physical memory can be allocated to guests.
The memory requirement for a guest depends on the guest’s workload, but will not vary from the memory required for the same
workload running on bare metal. For example, if you would like to run two Oracle Database 11g guests on a single Oracle VM server
and each guest requires 64GB of memory, the Oracle VM server would have to have a minimum of 130GB of memory for the guests
plus 512MB for dom0.
Note: That 512MB of memory that is reserved for dom0 is configurable by editing the “dom0_mem=” parameter in /boot/grub
/menu.lst file.
The 512MB fixed memory overhead for dom0 is rarely an issue except with servers that only have 1–4GB of memory. There is a
noteworthy difference between the memory overhead of paravirtualized guests verses hardware-virtualized guests. The overhead for
hardware-virtualized guests is much higher than for paravirtualized guests due to the internal data structures, for example, the use of
shadow page tables and dedicated QEMU processes for each guest. The overhead for paravirtualized guests is approximately 8MB
per guest, regardless of the amount of memory assigned to the guest.
Tip: To list the total amount of memory on the system type “xm info | grep mem” in dom0.
To effectively size the amount of memory for an Oracle VM server, the first step is to calculate the total amount of memory for all of
the guests that could run the Oracle VM server. Do not forget to add the overhead for dom0 and the overhead for each guest, that is,
20MB per guest plus 1% of the each guest’s memory size. The total memory requirements for the guests plus dom0’s memory
requirements is the amount of memory required for any Oracle VM server. If HA or Live Migration is used, each Oracle VM server in
the pool would need to have enough memory to run all the guests that could run on a given server at any time.
To determine the amount of memory for an Oracle VM server pool, is is necessary to consider the additional memory overhead from
an HA event or a Live Migration. For example, in an HA-enabled pool with two servers it would be necessary to have enough memory
on each server to run all of the guests in the pool in the event of an Oracle VM server failure.
Oracle VM 2.x does not officially support memory overcommit, which means that an Oracle VM server equipped with xGB of memory
can only allocate the available memory. To be able to support an HA event or Live Migration in an Oracle VM server pool, each Oracle
VM pool member must have enough free memory to be able to accept any new guests. For example, in an HA-enabled pool with two
servers, if one server fails, the available server must have enough free memory to run the guests from the failed server. If an HA event
occurrs between two pool members and the target server does not have enough free memory for the guests, the guests will be
blocked from starting on the target Oracle VM server.
Note: Oracle VM 2.2/Xen 3.4.0 ships with the experimental Xenballoond memory overcommit feature, although Xenballoond memory
overcommit is not enabled or supported by Oracle.
In January 2009, Dan Magenheimer from Oracle announced the "Transcendent Memory" project, "tmem" for short. Tmem will
improve on VMware’s long-available but fatally flawed mechanism for "time-sharing" physical memory between virtual machines,
commonly known as "ballooning .” The results of tmem will be better memory utilization and fewer disk accesses, which, in turn, will
lead to higher performance and greater virtual-machine density per Oracle VM server. Tmem may be included in the Oracle VM 3.0
release.
Oracle VM Server Storage Requirements
Unless the Oracle VM server is booting from SAN, some form of local storage is required. A default Oracle VM 2.2 server installation
creates a “local” OCFS2 virtual machine file system that is mounted under /var/ovs/mount/UUID and linked to /OVS. Using a local
storage repository restricts pool membership to one Oracle VM server without Live Migration or HA functionality. To increase the
capacity of an Oracle VM pool past one Oracle VM server, the addition of a shared back-end storage repository is required.
To determine your storage requirements for a single- or multiple-server Oracle VM server pool, calculate the disk requirements for all
of your guests, ISO files, and templates. To account for growth, consider provisioning at least 30% to 50% more storage for your
Oracle VM storage repositories than the expected size.
Oracle VM’s installation program does not provide the ability to configure shared storage repositories. Storage administration for
storage arrays must be configured after the installation of the server.
Oracle VM Server Network Requirements

Oracle VM server uses Xen bridging to set up the networking for guest traffic. The Xen bridge configuration allows all domUs to
appear on the network as individual hosts. All virtual network device communication is routed through a Xen bridge, which operates
at layer 2 of the OSI model, effectively acting as an L2 virtual switch.
To ensure that dom0 and your Xen bridges provide sufficient throughput and availability for guests, consider bonding multiple
network interfaces to increase throughput and availability. For example, mode4 network bonding (802.3AB), running with tagged
VLANs (802.1Q) allows better than single-wire-speed transfers from multiple hosts.
Oracle VM’s installation program does not include network interfaces bonding and VLAN configurations. Network interfaces bonding
and VLAN configurations must be done after the installation of the server.
Oracle VM Server Boot From SAN Requirements

the boot from SAN option, type “boot: linux mpath [enter]” from the installation boot prompt. The installation boot prompt is
displayed at the first Oracle VM Server installation screen. Typing “boot: linux mpath [enter]” tells the installer to use the device-
51 of 243 18.04.2011 16:35

mapper-multipath drivers.
Before a boot from SAN installation is started, ensure that the blade chassis or servers are configured to support boot from SAN.
Next, provision, zone and mask at least two LUNs. One unique LUN per server is the the boot LUN. The boot LUN is where Oracle VM
server is installed. Do not use the second LUN during the installation. After the installation, the second LUN will be formatted with
OCFS2 and configured as the shared root storage repository. The shared root storage repository is the virtual machine file system that
is shared between all of the Oracle VM server pool members. To enable the boot from SAN installation option, when the server boots
using the Oracle VM ISO, it is necessary to pass the mpath parameter. The next example shows how to pass the mpath parameter
from the Oracle VM server boot prompt.
boot: linux mpath [enter]
Typing “boot: linux mpath [enter]” from the boot prompt at the first Oracle VM Server screen tells the installer to use the device-
mapper-multipath drivers. Once the installer is using the device-mapper-multipath drivers, all zoned and maked LUNs will be visable
during the installation process. Be sure to only select the boot LUN for the installation!
CD-ROM Oracle VM Server Pool Member Installation

This section discusses an Oracle VM server installation for an Oracle VM pool member. The difference between an installation for an
Oracle VM server pool member and an Oracle VM server with local storage is that the Oracle VM pool member does not need a
dedicated /OVS partition. The pool member installation described below will remove the default /OVS partition and reallocate the disk
space from the deleted /OVS partition to the / partition. A default Oracle VM server installation will allocate most of the disk space to
the /OVS partition. The local disk requirements foran Oracle VM pool member installation is roughly 2GB.
Oracle VM Server Boot From SAN

the boot from SAN option, type “boot: linux mpath [enter]” from the installation boot prompt. The installation boot prompt is
displayed at the first Oracle VM Server installation screen. Typing “boot: linux mpath [enter]” tells the installer to use the device-
mapper-multipath drivers.
Before a boot from SAN installation is started, ensure that the blade chassis or servers are configured to support boot from SAN.
Next, provision, zone and mask at least two LUNs. One unique LUN per server is the the boot LUN. The boot LUN is where Oracle VM
server is installed. Do not use the second LUN during the installation. After the installation, the second LUN will be formatted with
OCFS2 and configured as the shared root storage repository. The shared root storage repository is the virtual machine file system that
is shared between all of the Oracle VM server pool members. To enable the boot from SAN installation option, when the server boots
using the Oracle VM ISO, it is necessary to pass the mpath parameter. The next example shows how to pass the mpath parameter
from the Oracle VM server boot prompt.
Typing “boot: linux mpath [enter]” from the boot prompt at the first Oracle VM Server screen tells the installer to use the device-
mapper-multipath drivers. Once the installer is using the device-mapper-multipath drivers, all zoned and maked LUNs will be visable
during the installation process. Be sure to only select the boot LUN for the installation!
Installing Oracle VM Server From Other Sources

Oracle VM server can be installed directly from a bootable CDROM, as well as from the Oracle VM server media files (aka the Oracle
VM server installation tree) that have been staged on a) an Oracle VM server's local hard drive b) on an NFS share c) on an FTP
server and/or d) on a web server. To install Oracle VM server from the Oracle VM media files, from the installation boot prompt type
"boot: linux askmethod [enter]. Typing "linux askmethod [enter] from the installation boot prompt will enable the Install Method
installation screen. From the Install Method screen you can select to install Oracle VM server from a) Local CDROM b) Hard Disk c)
NFS image d) FTP or e) HTTP.
screen.
To install Oracle VM server using a CD-ROM:

1. Insert the Oracle VM server media into the CD-ROM drive.
2. Boot the server with the Oracle VM server media in the CD-ROM drive.
3. The Oracle VM Server screen is displayed, as shown in Figure 3.
Figure 3 shows the Oracle VM Server screen.
52 of 243 18.04.2011 16:35

4. a) To Install Oracle VM server on the local hard drive: Press the Enter key to start the install program. If the Enter key is not
pressed for one minute, the install program will automatically start.
4. b) To install Oracle VM server to Boot from SAN: To enable the boot from SAN option, type “linux mpath [enter]” at the boot
prompt. Typing “boot: linux mpath [enter]” tells the installer to use the device-mapper-multipath drivers. The next example show
how to enable the boot from SAN option.
Typing boot: linux mpath [enter] will continue the installation process.
4. c) To Install Oracle VM server from Other Sources: From the installation boot prompt type "boot: linux askmethod [enter] to
enable the Install Method installation screen. From the Install Method installation screen, select and enter the details for the Hard
drive, NFS image, FTP or HTTP installation media.
The next example show how to enable the Install Method installation screen.
boot: linux askmethod [enter]
Typing boot: askmethod [enter] will continue the installation process.
The CD Found screen

On the CD Found screen, you can test the media for errors. To test the media, use the Tab key to select the OK button and press
Enter. Once the media test is completed, any errors will be reported. To skip the media test and continue with the install, use the Tab
key to select the Skip button and press Enter to continue.
Figure 4 shows the CD Found screen.
The Keyboard Selection screen

On the Keyboard Selection screen, use the Tab key to select the list of keyboard models. Then use the UP and DOWN keys (↑ or ↓)
to select the desired keyboard model. The keyboard that is selected becomes the default keyboard for dom0. Next, use the Tab key to
select OK, and press Enter to continue.
53 of 243 18.04.2011 16:35

Figure 5 shows the Keyboard Selection screen.
Warning screen
If you see the Warning screen, use the Tab key to select the Yes button, then press Enter to continue.
Figure 6 shows the Warning screen.
Partitioning Type screen

The Partitioning Type screen offers the following four partitioning options:
Remove all partitions and create a new default partition layout

Remove all Linux partitions and create a new default partition layout
Use the free space on selected drives to create a new default partition layout
Create a custom partition layout
Use the Tab key to select the Remove all partitions and create a new default partition layout option. Ensure that the
appropriate drive is select in the Which drive(s) do you want to use for this installation section. Use the Tab key to select the
OK button to continue.
Figure 7 shows the Partitioning Type screen.
54 of 243 18.04.2011 16:35

Warning screen
Since we selected the Remove all partitions and create a new default partition layout option, a Warning screen is displayed to
confirm that we want to remove the partition(s), including all of the data contained on any of the selected partitions. Use the Tab key
to select the YES button to continue.
Review Partition Layout screen

On the Review Partition Layout screen, use the Tab key to select the YES button to continue to the Partitioning screen.
Figure 9 shows the Review Partition Layout screen.
Partitioning screen
On the Partitioning screen, use the Tab key to select the /OVS Mount Point, then use the Tab key to select the Delete button.
Press Enter to continue.
Figure 10 shows the Partitioning screen.
Confirm Delete screen

On the Confirm Delete screen,use the Tab key to select the Delete button, then press Enter to continue.
55 of 243 18.04.2011 16:35

Figure 11 shows the Confirm Delete screen.
Partitioning screen
On the Partitioning screen, use the Tab key to select the / Mount Point, then use the Tab key to select the Edit button. Press
Enter to continue.
Add Partition screen

On the Add Partition screen, use the Tab key to select the Fill all available space option. Next, press the Space bar to select the
Fill all available space option. Use the Tab key to select the OK button to proceed.
Figure 13 shows the Add Partition screen
Partitioning screen
On the Partitioning screen, use the Tab key to select the OK button. Press Enter to continue.
56 of 243 18.04.2011 16:35

The Boot Loader Configuration screen

On the Boot Loader Configuration screen, use the Tab key to select the Master Boot Record (MBR) or the First sector of boot
partition as the location to install the boot loader. For this example, we have selected the Master Boot Record (MBR) option.Next,
use the Tab key to select the OK button and press Enter to continue.
Figure 15 shows the Boot Loader Configuration screen.
Oracle VM Server Management Interface screen

On the Oracle VM Server Management Interface screen, use the Tab key to select the network interface that will be dedicated for
the dom0 and the Oracle VM agent management traffic. Once the network interface is selected, use the Tab key to select the OK
button and press Enter to continue.
Note: dom0’s management interface defaults to eth0, which is controlled in the /etc/ovs-config file. The dom0 management interface
can be changed after the installation.
Figure 16 shows the Oracle VM Server Management Interface screen.
The IPv4 Configuration for eth0 screen

The IPv4 Configuration for eth0 screen offers the following three options:
Dynamic IP configuration (DHCP)
57 of 243 18.04.2011 16:35

Manual address configuration

IP Address and Prefix (netmask)
If your Oracle VM server will use DHCP to assign its IP address, select the Dynamic IP configuration (DHCP) option.To select the
Dynamic IP configuration (DHCP) entry, use the Tab key to highlight the Dynamic IP configuration (DHCP) entry, then use the
Space bar to select the Dynamic IP configuration (DHCP) entry. Use the Tab key to select the OK button to continue.
If your Oracle VM server will use a static IP address, select the Manual address configuration entry.To select the Manual address
configuration entry use the Tab key to highlight the Manual address configuration entry, then use the Space bar to select the
Manual address configuration entry. Next, use the Tab key to enter the IP Address and Prefix (netmask). Use the Tab key to select
the OK button to continue.
Figure 17 shows the IPv4 Configuration for eth0 screen.
The Miscellaneous Network Settings screen

On the Miscellaneous Network Settings screen, use the Tab key to select the Gateway, Primary DNS and optional Secondary
DNS to enter the networking settings for your environment. Use the Tab key to select the OK button and press Enter to continue.
Figure 18 shows the Miscellaneous Network Settings screen.
Hostname Configuration screen

On the Hostname Configuration screen, select one of the following two options:
automatically via DHCP

manually
If the machine uses DHCP to assign its hostname, select the automatically via DHCP option. Then, use the Tab key to select the OK
button to continue.
To assign a hostname for your Oracle VM server, select the manually option and enter the fully qualified domain name (FQDN) in the
text box. Then, use the Tab key to select the OK button to continue.
Figure 19 shows the Hostname Configuration screen.
58 of 243 18.04.2011 16:35

Time Zone Selection screen

On the Time Zone Selection screen select the System clock uses UTC option to use Coordinated Universal Time (UTC), then use
the Tab key and the UP or DOWN key (↑ or ↓) to select the time zone closest to your Oracle VM server’s physical location. Next, use
the Tab key to select the OK button and press Enter to continue.
Figure 20 shows the Time Zone Selection screen.
Oracle VM Agent password screen

On the Oracle VM Agent password screen, enter the password for the Oracle VM agent in the Password field. In the Password
(confirm) field, reenter the password. Use the Tab key to select the OK button and press Enter to continue. If the two passwords do
not match, the installation program will ask you to reenter the passwords.
The Oracle VM agent password is used by Oracle VM Manager and the Oracle VM Management Pack to dispatch commands and to
retrieve pool-status data. The Oracle VM agent password can be changed after the installation using Oracle VM Manager or the
Oracle VM Management Pack and from dom0 by typing “service ovs-agent configure”.
Figure 21 shows the Oracle VM Agent password screen
Root Password screen

On the Root Password screen, enter a password with at least six characters for the root user in the Password field. In the Password
59 of 243 18.04.2011 16:35

Figure 22 shows the Root Password screen.
Installation to begin screen

On the Installation to begin screen, select OK and press Enter to continue.
Figure 23 shows the Installation to begin screen.
Complete screen
When the Complete screen appears, remove the Oracle VM Server media from the CD-ROM drive and press Enter to reboot the
Oracle VM server.
Note: Remain at the Oracle VM servers console until the server reboots in order to accept the End User License Agreement, which is
displayed after the server reboots. For example, if you ssh in to the server after the reboot, you will not be presented with the End
User License Agreement. If the End User License Agreement is not accepted after the reboot, the Oracle VM agent will not be
started.
Figure 24 shows the Complete screen.
60 of 243 18.04.2011 16:35

End User License Agreement screen

On the End User License Agreement screen use the UP or DOWN key (↑ or ↓) to read the End User License Agreement. Then,
use the Tab key to select the Agree button and press Enter to continue.
TIP: If you ssh in to the server after the reboot, you will not be presented with the End User License Agreement. If the End User
License Agreement is not accepted after the reboot, the Oracle VM agent will not be started.
Figure 25 shows the End User License Agreement screen.
Oracle VM Server login prompt screen

On the Oracle VM Server login prompt screen, enter the root username and the password to access the dom0 console.
Figure 26 shows the Oracle VM Server login prompt screen.
Post Installation Checklist

1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the server and pool
configurations will not be hampered by unpatched bugs.
2. Ensure that all the Oracle VM servers’ clocks are synchronized using NTP.
First, open the “/etc/ntp.conf” file by typing “vi /etc/ntp.conf” and validate that at least two available NTP servers entries are listed.
The next example shows two bold NTP server entries in an ntp.conf file.
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server myntp1.com
server myntp2.com
Ping each NTP server listed in the ntp.conf file from each Oracle VM server to ensure network connectivity.
Next, type "ntpstat" on each Oracle VM server to validate the NTP configuration. The next example shows the output from typing the
ntpstat command on an Oracle VM server that has its time synchronized to an NTP server with the IP address of 192.168.4.251.
# ntpstat
synchronized to NTP server (192.168.4.251)
at stratum 4
time correct to within 54 ms
polling server every 1024 s
Finally, validate that the time, date and time zone on each Oracle VM server as well as on the Oracle VM Manager host is
61 of 243 18.04.2011 16:35

synchronized by typing the "date" command.
3. All Oracle VM servers have consistent name resolution using DNS with both forward and reverse lookups.
First, open the “/etc/resolv.conf” file by typing “vi /etc/resolv.conf” and validate that two available DNS servers are listed. The next
example shows two DNS servers listed in a resolve.conf file.
# vi /etc/resolve.conf
nameserver <MY DNS SERVER1 IP ADDRESS>
From each Oracle VM server ping each DNS server listed in the resolv.conf file to ensure network connectivity.
Next, validate the forward and reverse lookups for each Oracle VM pool member and the Oracle VM Manager host using the “host”
command. For example, to validate server2's forward lookup from server1 type “host server2” as shown in the next example.
# host server2
server2 has address
192.168.4.6
Next, to validate server2's reverse lookup from server1 type “host 192.168.4.6” as shown in the next example.
# host 192.168.4.6
6.4.168.192.in-addr.arpa domain
name pointer
server2
Note: Using hosts files without DNS is not advised and may produce unpredictable results.
4. The Oracle VM server’s host name in the /etc/hosts file must be associated with the Oracle VM server's public IP address. If an
Oracle VM pool member's host name is associated with 127.0.0.1, the cluster.conf file will be malformed and the Oracle VM pool will
not be operational. The next example shows the improper syntax from an Oracle VM server's hosts file entry.
127.0.0.1 servername.com servername

localhost.localdomain localhost
The next example shows the proper syntax for an Oracle VM server’s hosts file entry.
127.0.0.1 localhost.localdomain
localhost
192.168.4.8 servername.com
servername
5. ocfs2 network connectivity between all Oracle VM server pool members must be operational before creating a multiple server pool.
Check the ocfs2 network connectivity between all Oracle VM pool members by typing "nc -zv <myoraclevmserver1> 7777". For
example, if you have two Oracle VM servers named ovs1 and ovs2, from ovs1 type "nc -zv ovs2 7777". Typing "nc -zv ovs2 7777" from
ovs1 should return "succeeded!". If you receive a "failed: Connection refused" message between any Oracle VM servers, something
(firewall, switch, router, cable, etc..) is restricting communication between the hosts.
The iptables firewall on an Oracle VM server may be blocking the ocfs2 connectivity. If iptables is disabled and allowing all
connections, the output from typing “iptables -L” will look like the next example.
# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy

ACCEPT)
Chain OUTPUT (policy ACCEPT)

If typing “iptables -L” lists firewall rules, you can a) disable iptables by typing "service iptables stop && chkconfig iptables off" or b)
add the following bold iptables rule to the /etc/sysconfig/iptables file prior to the last line on all Oracle VM pool members.
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 7777 -j ACCEPT

-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
After you have added the above bold iptables rule, restart the iptables service by typing "service iptables restart".
CD-ROM Oracle VM Server Installation with Local Storage
62 of 243 18.04.2011 16:35

This section discusses a default Oracle VM installation with local storage. A default Oracle VM server installation will allocate most of
the disk space to the /OVS partition. The difference between an installation for an Oracle VM server with local storage and a pool
member is that the pool member does not need an /OVS partition with dedicated storage. A pool member installation removes the
default /OVS partition and reallocates the disk space from the /OVS partition to the / partition.
Installing Oracle VM Server From Other Sources

Oracle VM server can be installed directly from a bootable CDROM, as well as from the Oracle VM server media files (aka the Oracle
VM server installation tree) that have been staged on a) an Oracle VM server's local hard drive b) on an NFS share c) on an FTP
server and/or d) on a web server. To install Oracle VM server from the Oracle VM media files, from the installation boot prompt type
"boot: linux askmethod [enter]. Typing "linux askmethod [enter] from the installation boot prompt will enable the Install Method
installation screen. From the Install Method screen you can select to install Oracle VM server from a) Local CDROM b) Hard Disk c)
NFS image d) FTP or e) HTTP.
screen.
To install Oracle VM server using a CD-ROM:

1. Insert the Oracle VM server media into the CD-ROM drive.
2. Boot the server with the Oracle VM server media in the CD-ROM drive.
3. The Oracle VM Server screen is displayed, as shown in Figure 31.
Figure 27 shows the Oracle VM Server screen.
4. a) To Install Oracle VM server on the local hard drive: Press the Enter key to start the install program. If the Enter key is not
pressed for one minute, the install program will automatically start.
4. b) To Install Oracle VM server from Other Sources: From the installation boot prompt type "boot: linux askmethod [enter] to
enable the Install Method installation screen. From the Install Method installation screen, select and enter the details for the Hard
drive, NFS image, FTP or HTTP installation media.
The next example show how to enable the Install Method installation screen.
boot: linux askmethod [enter]
Typing boot: askmethod [enter] will continue the installation process.
The CD Found screen

On the CD Found screen, you can test the media for errors. To test the media, use the Tab key to select the OK button and press
Enter. Once the media test is completed, any errors will be reported. To skip the media test and continue with the install, use the Tab
key to select the Skip button and press Enter to continue.
Figure 28 show the CD Found screen.
63 of 243 18.04.2011 16:35

The Keyboard Selection screen

On the Keyboard Selection screen, use the Tab key to select the list of keyboard models. Then use the UP or DOWN key (↑ or ↓) to
select the desired keyboard model. The keyboard that is selected becomes the default keyboard for dom0. Next, use the Tab key to
select OK, then press Enter to continue.
Figure 29 shows the Keyboard Selection screen.
Warning screen
If you see the Warning screen, use the Tab key to select the Yes button, then press Enter to continue.
Partitioning Type screen

The Partitioning Type screen offers the following four partitioning options:
Remove all partitions and create a new default partition layout

Remove all Linux partitions and create a new default partition layout
Use the free space on selected drives to create a new default partition layout
Create a custom partition layout
64 of 243 18.04.2011 16:35

Use the Tab key to select the Remove all partitions and create a new default partition layout option. Ensure that the
appropriate drive is select in the Which drive(s) do you want to use for this installation section. Use the Tab key to select the
OK button to continue.
Figure 31 shows the Partitioning Type screen.
Warning screen
Since we selected the Remove all partitions and create a new default partition layout option, a Warning screen is displayed to
confirm that we want to remove the partition(s), including all of the data contained on any of the selected partitions. Use the Tab key
to select the YES button to continue.
Review Partition Layout screen

From the Review Partition Layout screen, use the Tab key to select the NO button and press Enter to continue.
Figure 33 shows the Review Partition Layout screen.
The Boot Loader Configuration screen

On the Boot Loader Configuration screen, use the Tab key to select the Master Boot Record (MBR) or the First sector of boot
65 of 243 18.04.2011 16:35

partition as the location to install the boot loader. For this example, we selected the Master Boot Record (MBR) option.Use the
Tab key to select the OK button and press Enter to continue.
Figure 34 shows the Boot Loader Configuration screen.
Oracle VM Server Management Interface screen

On the Oracle VM Server Management Interface screen, use the Tab key to select the network interface that will be dedicated for
the dom0 and the Oracle VM agent management traffic. Once the network interface is selected, use the Tab key to select the OK
button and press Enter to continue.
Note: dom0’s management interface defaults to eth0, which is controlled in the /etc/ovs-config file. The dom0 management interface
can be changed after the installation.
Figure 35 shows the Oracle VM Server Management Interface screen.
The IPv4 Configuration for eth0 screen

The IPv4 Configuration for eth0 screen offers the following three options:
Dynamic IP configuration (DHCP)

Manual address configuration
IP Address and Prefix (netmask)
If your Oracle VM server will use DHCP to assign its IP address, select the Dynamic IP configuration (DHCP) option.You can select
the Dynamic IP configuration (DHCP) entry by using the Tab key to highlight the Dynamic IP configuration (DHCP) entry then
use the Space bar to select the Dynamic IP configuration (DHCP) entry. Use the Tab key to select the OK button to continue.
If your Oracle VM server will use a static IP address, select the Manual address configuration entry.You can select the Manual
address configuration entry by using the Tab key to highlight the Manual address configuration entry, then use the Space bar to
select the Manual address configuration entry. Next, use the Tab key to enter the IP Address and Prefix (netmask). Use the Tab
key to select the OK button to continue.
Figure 36 shows the IPv4 Configuration for eth0 screen.
66 of 243 18.04.2011 16:35

The Miscellaneous Network Settings screen

On the Miscellaneous Network Settings screen, use the Tab key to select the Gateway, Primary DNS, and optional Secondary
DNS to enter the networking settings for your environment. Use the Tab key to select the OK button and press Enter to continue.
Figure 37 shows the Miscellaneous Network Settings screen.
Hostname Configuration screen

On the Hostname Configuration screen, select one of the following two options:
automatically via DHCP

manually
If the machine uses DHCP to assign its hostname, select the automatically via DHCP option. Use the Tab key to select the OK
button to continue.
To assign a hostname for your Oracle VM server, select the manually option and enter the fully qualified domain name (FQDN) in the
text box. Use the Tab key to select the OK button to continue.
Figure 38 shows the Hostname Configuration screen.
Time Zone Selection screen
67 of 243 18.04.2011 16:35

On the Time Zone Selection screen select the System clock uses UTC option to use Coordinated Universal Time (UTC), then use
the Tab key and the UP or DOWN key (↑ or ↓) to select the time zone closest to your Oracle VM server’s physical location. Use the
Tab key to select the OK button and press Enter to continue.
Figure 39 shows the Time Zone Selection screen.
Oracle VM Agent password screen

On the Oracle VM Agent password screen, enter the password for the Oracle VM agent in the Password field. In the Password
The Oracle VM agent password is used by Oracle VM Manager and the Oracle VM Management Pack to dispatch commands and to
retrieve pool status data. The Oracle VM agent password can be changed after the installation using Oracle VM Manager or the
Oracle VM Management Pack, or from dom0 by typing “service ovs-agent configure”.
Figure 40 shows the Oracle VM Agent password screen
Root Password screen

On the Root Password screen, enter a password with at least six characters for the root user in the Password field. In the Password
Figure 41 shows the Root Password screen.
68 of 243 18.04.2011 16:35

Installation to begin screen

On the Installation to begin screen, select OK and press Enter to continue.
Figure 42 shows the Installation to begin screen.
Complete screen
When the Complete screen appears, remove the Oracle VM Server media from the CD-ROM drive and press Enter to reboot the
Oracle VM server.
TIP: Remain at the Oracle VM servers console until the server reboots to be able to accept the End User License Agreement, which is
displayed after the server reboots. For example, if you ssh in to the server after the reboot you will not be presented with the End
User License Agreement. If the End User License Agreement is not accepted after the reboot, the Oracle VM agent will not be
started.
Figure 43 shows the Complete screen.
End User License Agreement screen

On the End User License Agreement screen, use the UP or DOWN key (↑ or ↓) to read the End User License Agreement. Use
the Tab key to select the Agree button and press Enter to continue.
69 of 243 18.04.2011 16:35

TIP: If you ssh in to the server after the reboot you will not be presented with the End User License Agreement. If the End User
License Agreement is not accepted after the reboot the Oracle VM agent will not be started.
Figure 44 shows the End User License Agreement screen.
Oracle VM Server login prompt screen

On the Oracle VM Server login prompt screen, enter the root username and the password to access the dom0 console.
Figure 45 shows the Oracle VM Server login prompt screen.
Post Installation Checklist

1. The Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the server and pool
configurations will not be hampered by unpatched bugs.
2. Ensure that all the Oracle VM servers’ clock is synchronized using NTP.
server myntp1.com
server myntp2.com
# ntpstat
at stratum 4
time correct to within 54 ms
3. The Oracle VM server must have consistent name resolution using DNS with both forward and reverse lookups.
70 of 243 18.04.2011 16:35

# host server2
server2 has address
192.168.4.6
# host 192.168.4.6
name pointer
server2

localhost
servername
Oracle VM Server PXE/Kickstart Installation

This section starts with a discussion of an Enterprise Linux boot server configuration using RPM packages from the Unbreakable
Linux Network (ULN). After the boot server configuration, we will learn how to create and stage a kickstart file to automate an Oracle
VM server installation. Next, we will discuss the PXE boot client requirements and configurations. The section concludes with
PXE/kickstart installation troubleshooting tips.
This section discusses how to configure an Enterprise Linux boot server with DHCP, TFTP, and HTTP. The boot server allows a
bare-metal system to receive an IP address via DHCP, load a kernel via TFTP, and boot without an operating system. Next, the
kickstart file orchestrates an automated Oracle VM server installation. Once the installation is completed, a personalized Oracle VM
server is booted and can be added to a pool using Oracle VM Manager or the Oracle VM Management Pack.
The DHCP service will be configured with network-specific details along with the IP addresses of the Oracle VM servers you will
PXE/kickstart install. You will need to change the example network details with your specific environmental networking details.
The TFTP service will be configured with a /tftpboot directory populated with:
1. The pxelinux.0 bootloader program in /tftpboot

2. The vmlinuz compressed Linux kernel in /tftpboot
3. The initrd.img initial RAM disk in /tftpboot/ovm/2.2
4. A subdirectory named pxelinux.cfg with the PXE boot files. The example will have the following two PXE boot files:
C0A804C7
C0A804C6
The HTTP server will be configured with default settings and used to host the Oracle VM media files and the kickstart files.
HTTP Services Configuration

This section starts with the installation and configuration of Apache. The section concludes with the steps to mount and copy the
Oracle VM installation media files (aka, the Oracle VM server installation tree) to the /var/www/html/oraclevm2.2/ directory.
Installing Apache from a registered Enterprise Linux host from the Unbreakable Linux Network is accomplished by typing “up2date -i
httpd” while logged in as root or using sudo. Once Apache is installed, configure Apache to automatically start by typing “chkconfig
httpd on”. Next, start Apache by typing “service httpd start”. The next example shows how to install, configure and start Apache.
71 of 243 18.04.2011 16:35

# sudo up2date -i httpd

# sudo chkconfig httpd on
# sudo service httpd start
Once the “up2date -i httpd”, “chkconfig httpd on” and “service httpd start” commands have finished, test your Apache server by
pointing a web browser to the fully qualified domain name (FQDN) or the IP address of the Apache server. You will see the default
Apache test page as shown in Figure 46.
Tip: If you don’t see the default Apache test page, check if iptables is blocking http traffic on the Apache host. Consider disabling
iptables to test Apache by typing “sudo /sbin/service iptables stop”.
The “If you are the website administrator:” section of the default Apache test page explains how to disable the test page and to enable
the default web root directory at /var/www/html. To disable the Apache test page and to enable the default web root at /var/www
/html, access the Apache server and comment out all the entries in the /etc/httpd/conf.d/welcome.conf file. The next example shows
the default /etc/httpd/conf.d/welcome.conf file.
# This configuration file enables the default "Welcome"

# page if there is no default index page present for
# the root URL. To disable the Welcome page, comment
# out all the lines below.
#
<LocationMatch "^/+$">
Options -Indexes
ErrorDocument 403 /error/noindex.html
</LocationMatch>
The next example shows the /etc/httpd/conf.d/welcome.conf file with all the entries commented out.
# This configuration file enables the default "Welcome"

# page if there is no default index page present for
# the root URL. To disable the Welcome page, comment
# out all the lines below.
#
#<LocationMatch "^/+$">
# Options -Indexes
# ErrorDocument 403 /error/noindex.html
#</LocationMatch>
To test the new configuration restart Apache, as shown in the next example.
# sudo service httpd restart

Stopping httpd: [ OK ]
Starting httpd: [ OK ]
To view the default root directory of your Apache server located at /var/www/html refresh your browser or point a browser to the fully
qualified domain name (FQDN) or the IP address of the Apache server. You will be presented with the Apache root directory as shown
in Figure 47.
72 of 243 18.04.2011 16:35

How to Make the Oracle VM Server Installation Tree Available for a PXE/Kickstart Installation
This section discusses how to make the Oracle VM server installation tree available for a PXE/kickstart installation on a boot server in
the /var/www/html/ovs2.2 directory. All of the steps below are completed on the boot server using sudo or root.
1. Create a directory named ovs2.2 in /var/www/html, by typing “mkdir -p /var/www/html/ovs2.2”.

2. Copy the Oracle VM server ISO file to the Apache server.
3. From the directory with the ISO file, mount the ISO file, for example in /tmp/mnt by typing “mkdir –p /tmp/mnt” and then “mount
-o loop,ro OracleVM-Server-2.2.0.iso /tmp/mnt/”.
4. Copy the installation files to the /var/www/html/ovs2.2 directory by typing “cp –avr /tmp/mnt/* /var/www/html/ovs2.2”.
5. Change the ownership of the files in the ovs2.2 directory by typing “chown -R apache:apache /var/www/html/ovs2.2”.
The next example shows how to create the directories, mount and copy the installation files and change the ownership of the ovs2.2
directory.
# sudo mkdir -p /var/www/html/ovs2.2

# sudo mkdir -p /tmp/mnt
# sudo mount -o loop,ro OracleVM-Server-2.2.0.iso /tmp/mnt/
# sudo cp -avr /tmp/mnt/* /var/www/html/ovs2.2
# sudo chown -R apache:apache /var/www/html/ovs2.2
To confirm that the Oracle VM server installation tree is available for the PXE/kickstart installation via HTTP, point your browser to
the ovs2.2 directory, that is, http://<FQDN>/ovs2.2. You will see the installation files as shown in Figure 48.
DHCP Service Configurations

In this section, we will install and configure DHCP with the IP addresses of the Oracle VM servers that will be PXE booted and
installed using a kickstart file.
1- The first step is to install the DHCP service. Using sudo or root, type “up2date -i dhcp” as shown in the next example.
# sudo up2date -i dhcp
2- Next, we will configure the startup parameters for the dhcpd daemon, by typing “chkconfig --list dhcpd”. Then, type “chkconfig
dhcpd on” to configure the dhcpd daemon to start at runlevels 2, 3, 4 and 5. Next, type “chkconfig --list dhcpd” to validate the dhcpd
73 of 243 18.04.2011 16:35

runlevels, as shown in the next example.
# sudo chkconfig --list dhcpd

dhcpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
# sudo chkconfig dhcpd on
# sudo chkconfig --list dhcpd
dhcpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
3- Next, we will configure the DHCP server’s /etc/dhcpd.conf file. The default DHCP configuration file is located at /etc/dhcpd.conf.
An example dhcpd.conf file is located at /usr/share/doc/dhcp-3.0.5/dhcpd.conf.sample.
Replace the bold section with your environmental details.
ddns-update-style none;
allow booting; # support PXE booting
allow bootp; # respond to bootp queries
subnet 192.168.4.0 netmask 255.255.255.0 {

option routers 192.168.4.254;
option subnet-mask 255.255.255.0;
option nis-domain "example.com";
option domain-name "example.com";
option domain-name-servers 192.168.4.11;
option time-offset -28800; # Pacific Standard Time
default-lease-time 120;
max-lease-time 120;
next-server 192.168.4.11;
pool {
range dynamic-bootp 192.168.4.199 192.168.4.230;
}
#Oracle VM 2.2 Kickstart boxes
group {
filename "pxelinux.0";
host ovs2
{ hardware ethernet 00:30:48:7F:44:6E; fixed-address 192.168.4.199; }
host ovs3
{ hardware ethernet 00:30:48:7F:35:0A; fixed-address 192.168.4.198; }
}
}
The example dhcpd.conf file will configure the DHCP server to respond to DHCP requests on the 192.168.4.0 network with a netmask
of 255.255.255.0, as shown in the next example.
subnet 192.168.4.0 netmask 255.255.255.0 {
Replace the 192.168.4.0 network address and the 255.255.255.0 subnet mask with your network address and subnet mask.
The next section of the dhcpd.conf file configures the router, subnet mask, nis-domain, domain name, name server, and time zone.
option routers 192.168.4.254;

option subnet-mask 255.255.255.0;
option nis-domain "example.com";
option domain-name "example.com";
option domain-name-servers 192.168.4.11;
option time-offset -28800; # Pacific Standard Time
Replace the router, subnet mask, nis-domain (if applicable), domain name (optional), name server (DNS), and time zone with your
environmental details.
Tip: Time zone details are listed in the /usr/share/zoneinfo/ directory.
74 of 243 18.04.2011 16:35

The next section lists the next-server entry, which is the IP address of your boot server.
next-server 192.168.4.11;
Replace 192.168.4.11 with the IP address of your boot server.
The next section of the dhcpd.conf file shows the IP address range the DHCP server will assign to DHCP clients. For example, the
DHCP server will assign IP addresses from 192.168.4.199 to 192.168.4.230.
range dynamic-bootp 192.168.4.199 192.168.4.230;
Replace the 192.168.4.199 192.168.4.230 address range with your DHCP IP address range.
The next section of the dhcpd.conf file lists a descriptive name, MAC address, and the fixed IP address of each Oracle VM server that
will PXE boot.
host ovs2
host ovs3
{ hardware ethernet 00:30:48:7F:35:0A; fixed-address 192.168.4.198; }
Replace the host section with a descriptive name for your Oracle VM server, for example, “host server-name”. Next, replace the MAC
address and the fixed IP address with your server’s details.
Tip: If your servers have an OS installed, type “ifconfig -a” for nix hosts or “ipconfig /all” for Windows to list each NICs MAC address.
If your servers do not have an OS, enable the PXE boot option in the system BIOS. Once PXE boot is enabled, you should be able to
see the MAC addresses during the system startup.
4- Once you have the entries in the dhcpd.conf file, restart the DHCP service by typing “service dhcpd restart”, as shown in the
next example.
# sudo service dhcpd start

Starting dhcpd: [ OK ]
Note: It’s necessary to restart the DHCP service to recognize any modifications made to the dhcpd.conf file.
TFTP Service Configurations

In this section we will configure the tftp service.
The tftp service is managed by xinetd. The default xinetd configuration disables tftp. To enable the tftp service, edit /etc/xinetd.d/tftp
and change the “disable = off” line to “disable = on”.
Once the tftp service is enabled, configure the startup parameters for the xinetd daemon by typing “chkconfig xinetd on”. Typing
“chkconfig xinetd on” configures the xinetd daemon to start at runlevels 2, 3, 4, and 5. Next, type “chkconfig --list xinetd” to list
xinetd’s runlevels, as shown in the next example.
# sudo chkconfig xinetd on

# sudo chkconfig --list xinetd
xinetd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Next, restart the xinetd service by typing “service xinetd restart”, as shown in the next example.
# sudo service xinetd restart

Stopping xinetd: [ OK ]
Starting xinetd: [ OK ]
To confirm that the tftp service is running, type “netstat -l -u | grep tftp”, as shown in the next example.
75 of 243 18.04.2011 16:35

# netstat -l -u | grep tftp

udp 0 0 *:tftp *:*
If no output is displayed after typing “netstat -l -u | grep tftp”, the tftp service is not running. The example confirms that the tftp
service is up.
Tip: Consider disabling iptables during the testing phase by typing “sudo /sbin/service iptables stop”.
How to Create the PXE boot Files

In this section, we will prepare the PXE boot files on the boot server in the /tftpboot directory. The tftp service will be configured with
a /tftpboot directory populated with:
a bootloader program “pxelinux.0” in /tftpboot

a compressed Linux kernel “vmlinuz” in /tftpboot
an initial RAM disk “initrd.img” in /tftpboot/ovm/2.2
a subdirectory “pxelinux.cfg” with PXE boot files for two Oracle VM servers:
C0A804C7
C0A804C6
In the next section, we will prepare the PXE boot files in the /tftpboot directory. All of the steps below are completed on the boot
server using sudo or root.
1. Create the /tftpboot/ovm/2.2/ directory by typing “mkdir -p /tftpboot/ovm/2.2/”.

2. Create the /tftpboot/pxelinux.cfg directory by typing “mkdir -p /tftpboot/pxelinux.cfg”
3. Copy the bootloader program from the boot server in /usr/lib/syslinux/pxelinux.0 to the /tftpboot/ directory by typing “cp /usr/lib
/syslinux/pxelinux.0 /tftpboot/”.
4. Copy the initrd.img file located on the <Oracle VM Server media>/images/pxeboot/initrd.img directory to the /tftpboot/ovm/2.2/
directory by typing cp <Oracle VM Server media>/images/pxeboot/initrd.img /tftpboot/ovm/2.2/.
5. Copy the vmlinuz file from the <Oracle VM Server media>/images/pxeboot/ vmlinuz to the /tftpboot directory by typing “cp
<Oracle VM Server media>/images/pxeboot/ vmlinuz /tftpboot/”.
The next example shows the commands from Steps 1 through 5.
# sudo mkdir -p /tftpboot/ovm/2.2/

# sudo mkdir -p /tftpboot/pxelinux.cfg
# sudo cp /usr/lib/syslinux/pxelinux.0 /tftpboot/
# sudo cp <Oracle VM Server media>/images/pxeboot/initrd.img /tftpboot/ovm/2.2/
# sudo cp <Oracle VM Server media>/images/pxeboot/ vmlinuz /tftpboot/
The next example show the contents of the /tftpboot directory after Steps 1 through 5.
# tree /tftpboot/
/tftpboot/
|
|-- ovm
| `-- 2.2
| `-- initrd.img
|-- pxelinux.0
|-- pxelinux.cfg
`-- vmlinuz
6. Next, we will create two unique PXE boot files. Each PXE boot file has a unique name that represents the hex number of the IP
address of the server. A PXE boot file contains the PXE boot and kickstart configurations. The PXE boot configurations are
identical for all Oracle VM servers and allow the bare-metal systems to load a kernel via TFTP and boot without an operating
system. The kickstart configuration is different for each PXE boot file. Each PXE boot file parses a unique kickstart file that
personalizes the kickstart installation.
To convert an IP address to the hex value for a PXE boot file, use the “gethostip” program by typing “gethostip -x <IP ADDRESS>”.
The next example shows how to list the hex values for 192.168.4.199 and 192.168.4.198.
# gethostip -x 192.168.4.199
C0A804C7
# gethostip -x 192.168.4.198
C0A804C6
76 of 243 18.04.2011 16:35

The first PXE boot file is named C0A804C7 and is for the Oracle VM server in the dhcpd.conf file named ovs2. The next example
shows ovs2’s entry in the dhcpd.conf file.
host ovs2
The next example shows how to get the name of the PXE boot file for ovs2, that is, 192.168.4.199.
# gethostip -x 192.168.4.199
C0A804C7
The second PXE boot file named is C0A804C6 and is for the Oracle VM server in the dhcpd.conf file named ovs3. The next example
shows ovs3’s entry in the dhcpd.conf file.
host ovs3
{ hardware ethernet 00:30:48:7F:35:0A; fixed-address
192.168.4.198; }
The next example shows how to get the name of the PXE boot file for ovs3, that is 192.168.4.198.
# gethostip -x 192.168.4.198
C0A804C6
Next, create the PXE boot file for ovs2 (192.168.4.199) by typing “vi /tftpboot/pxelinux.cfg/C0A804C7”. Enter the configurations
shown below and save the file.
# vi /tftpboot/pxelinux.cfg/C0A804C7
default ovsboot
label ovsboot
kernel vmlinuz
append initrd=ovm/2.2/initrd.img ks=http://192.168.4.11/ovs2-ks.cfg ksdevice=eth0
:wq!
Typing “vi /tftpboot/pxelinux.cfg/C0A804C7” creates a PXE boot file C0A804C7 in /tftpboot/pxelinux.cfg/. The default line is required.
You can change the description from the default line, that is, replace ovsboot with any descriptive name. The label line is also
required. You can change the description from the label line, that is, replace ovsboot with any name. The kernel vmlinuz line
configures the PXE boot system to load the compressed Linux kernel “vmlinuz”. The append initrd=ovm/2.2/initrd.img
ks=http://192.168.4.11/ovs2-ks.cfg ksdevice=eth0 line configures a) the initial RAM disk “initrd.img” and b) the kickstart file
ovs2-ks.cfg on the boot server. Pressing the Esc key followed by :wq! saves the C0A804C7 file.
Note: The kickstart file can be staged using http, ftp, or nfs, as shown in the next example.
ks=http://192.168.4.11/ovs3-ks.cfg
ks=ftp:// 192.168.4.11/ovs3-ks.cfg
ks=nfs:192.168.4.11/ovs3-ks.cfg
Next, create a PXE boot file for ovs3 (192.168.4.198) by typing “vi /tftpboot/pxelinux.cfg/ C0A804C6”. Enter the configurations shown
below and save the file.
vi /tftpboot/pxelinux.cfg/C0A804C6
default ovsboot
label ovsboot
kernel vmlinuz
:wq!
Finally, make sure all files and directories in /tftpboot have the right permission set. The next example shows how to set the correct
permissions for the files and directories in /tftpboot.
77 of 243 18.04.2011 16:35

# find /tftpboot/ -type f -exec chmod 644 {} \;

# find /tftpboot/ -type d -exec chmod 755 {} \;
The next example shows all of the directories and files in the /tftpboot directory.
# tree /tftpboot/
/tftpboot/
|
|-- ovm
| `-- 2.2
| `-- initrd.img
|-- pxelinux.0
|-- pxelinux.cfg
| `-- C0A804C7
| `-- C0A804C6
`-- vmlinuz
Kickstart File Examples

This section discusses how to create, stage, test, and troubleshoot a kickstart file to automate an Oracle VM server installation. The
section starts with an overview of a default Oracle VM anaconda kickstart file followed by three example kickstart files. Then, we will
learn how to generate encrypted passwords for a kickstart file. Next, we review with how to stage a kickstart file on a boot server for
a PXE/kickstart installation. The section concludes with PXE client configurations and PXE/kickstart installation troubleshooting.
Each Oracle VM server installation generates a generic anaconda kickstart file in the /root directory, named anaconda-ks.cfg. The
kickstart file is used by anaconda during the installation process to define the installation parameters. The default anaconda-ks.cfg file
lists the answers to each Oracle VM server installation question and field entry required by the installation program. If any of the
required fields are missing from a kickstart file the installation halts and waits for input.
Example 1 shows a default Oracle VM anaconda kickstart file. The bold sections need to be modified to create a kickstart file for your
environment.
Example 1
# Kickstart file automatically generated by anaconda.
install
cdrom
lang en_US.UTF-8
network --device eth0 --bootproto static --ip 192.168.4.6 --netmask 255.255.255.0 --gateway 192.168.4.254 --nameserver
192.168.4.11 --hostname ovs1.sf.itnc.com
ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx
ovsmgmntif eth0
rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx
firewall --enabled --port=21:tcp --port=22:tcp --port=53:udp --port=53:tcp --port=80:tcp --port=2049:tcp --port=5900-5950:tcp
--port=8002:tcp --port=8003:tcp --port=8899:tcp --port=7777:tcp
authconfig --enableshadow --enablemd5
selinux --disabled
timezone --utc America/Los_Angeles
bootloader --location=mbr --dom0_mem=569 --driveorder=sda
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
#clearpart --all --drives=sda
#part /boot --fstype ext3 --size=100 --ondisk=sda
#part / --fstype ext3 --size=3072 --ondisk=sda
#part /OVS --fstype ocfs2 --size=1024 --grow --ondisk=sda
#part swap --size=1024 --ondisk=sda
%packages
@office
@admin-tools
@editors
@text-internet
@gnome-desktop
@dialup
@core
@base
@games
78 of 243 18.04.2011 16:35

@java
@base-x
@graphics
@printing
@sound-and-video
@ovs-virtualization
@graphical-internet
tftp
bridge-utils
squashfs-tools
The default kickstart file must be modified before you can test a PXE/kickstart installation. The next list show the sections of the
default kickstart file that need to be modified for your environment.
cdrom
#clearpart --all --drives=sda
#part /boot --fstype ext3 --size=100 --ondisk=sda
#part / --fstype ext3 --size=3072 --ondisk=sda
#part /OVS --fstype ocfs2 --size=1024 --grow --ondisk=sda
#part swap --size=1024 --ondisk=sda
%packages
Tip: To review an exhaustive list of kickstart file options, please consult the following resources:
http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Installation_Guide-en-US/s1-kickstart2-options.html
http://fedoraproject.org/wiki/Anaconda/Kickstart
Example 2 is a commented kickstart file that was modified from the default anaconda kickstart file and will be used to install an
Oracle VM server pool member named ovs2.sf.itnc.com. The Oracle VM server ovs2.sf.itnc.com will not have a local storage
repository, that is, an /OVS directory. The bold sections need to be modified for your environment.
Note: Timezone details and entries are listed in the /usr/share/zoneinfo/ directory.
Example 2
### Installation
Install
### Keyboard layout
lang en_US.UTF-8
keyboard us
### Skips using X windows
skipx
### Installation source
url --url http://192.168.4.11/ovs2.2/
### Network configurations
### Oracle VM server agent password
### Oracle VM agent management interface
ovsmgmntif eth0
### Root password
### dom0 iptables configuration
### Authentication settings
### SELinux configuration
selinux –disabled
### Timezone settings
### Bootloader configuration
### Automatic reboot after install
#reboot
## Partitioning
79 of 243 18.04.2011 16:35

clearpart --linux
part /boot --fstype=ext3 --size=256
part / --fstype=ext3 --size=1 --grow
part swap --fstype=swap –recommended
## Packages
%packages
@core
@base
@ovs-virtualization
tftp
bridge-utils
Note: Example 2 uses a web server to host the installation source. The installation source can be hosted using http, nfs, or ftp. The
next list shows http, nfs and ftp entries in a kickstart file hosting the installation source.
url --url http://192.168.4.11/ovs2.2/

nfs --server=192.168.4.11 --dir=/ovs2.2/
url --url=ftp:// 192.168.4.11/ovs2.2/
Example 3 was modified from the default anaconda kickstart file and is used to install an Oracle VM server pool member named
ovs3.sf.itnc.com. ovs3.sf.itnc.com will not have an /OVS directory. The bold sections need to be modified for your environment.
Example 3
install
key --skip
lang en_US.UTF-8
keyboard us
skipx
url --url http://192.168.4.11/ovs2.2/
ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxx
ovsmgmntif eth0
rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxx
selinux --disabled
#reboot
clearpart --linux
part /boot --fstype=ext3 --size=256
part / --fstype=ext3 --size=1 --grow
part swap --fstype=swap --recommended
%packages
@core
@base
@ovs-virtualization
tftp
bridge-utils
Example 4 was modified from the default anaconda kickstart file and is used to install a stand-alone Oracle VM server named
ovs1.sf.itnc.com. ovs1.sf.itnc.com will have a local storage repository in the /OVS directory. The bold sections need to be modified for
your environment.
Example 4
install
key --skip
lang en_US.UTF-8
keyboard us
skipx
url --url http://192.168.4.11/ovs2.2/
ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
ovsmgmntif eth0
rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
80 of 243 18.04.2011 16:35


selinux --disabled
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
#reboot
clearpart --all --drives=sda
part /boot --fstype ext3 --size=100 --ondisk=sda
part / --fstype ext3 --size=3072 --ondisk=sda
part /OVS --fstype ocfs2 --size=1024 --grow --ondisk=sda
part swap --size=1024 --ondisk=sda
%packages
@core
@base
@ovs-virtualization
tftp
bridge-utils
Note: None of the example kickstart files will automatically reboot the server after the installation. Uncomment the reboot section of
the kickstart file to have the Oracle VM server automatically reboot after the installation. After a successful PXE/kickstart installation,
either disable the PXE boot feature from the BIOS or comment out the Oracle VM server’s entries in the dhcpd.conf file to disable
PXE boot and to preclude the system from performing additional PXE/kickstart installations.
How to Generate an Encrypted Password with grub-md5-crypt

This section reviews how to generate the encrypted passwords for the root user account and the ovs-agent account for an Oracle VM
server kickstart file. The following list shows the two password entries in an Oracle VM server kickstart file that require encrypted
passwords.

To generate encrypted passwords for the root user account and the ovs-agent account, use the grub-md5-crypt program. From an
Enterprise Linux host , while logged in as root or using sudo, type “grub-md5-crypt”. You will be prompted for a password and then
asked to confirm the password. The password will not be visible as you type it. The grub-md5-crypt program will then print out the
MD5-encrypted password.
Note: The Oracle VM installation program requires at least six characters each for the root and Oracle VM agent passwords.
The following example shows the output from grub-md5-crypt.
# grub-md5-crypt
Password:
Retype password:
xxxxxxxxxxxxxxxxxxxxxxxxxx
Once you have generated the encrypted passwords for the root user account and the ovs-agent account, place the encrypted
passwords in the kickstart file and stage the kickstart file on the boot server.
How to Stage a Kickstart File for an Oracle VM Server PXE/kickstart Installation

The section discusses how to stage a kickstart file on a web server for an Oracle VM server PXE/kickstart installation.
Kickstart files can be staged using http, ftp, or nfs. The next example shows a PXE boot file named C0A804C7 that points to a
kickstart file (ovs2-ks.cfg) staged on a web server (192.168.4.11).
# vi /tftpboot/pxelinux.cfg/C0A804C7
default ovsboot
label ovsboot
kernel vmlinuz
:wq!
Note: Kickstart files can be staged using http, ftp, or nfs, as shown in the next example.
ks=http://192.168.4.11/ovs3-ks.cfg
ks=ftp:// 192.168.4.11/ovs3-ks.cfg
ks=nfs:192.168.4.11/ovs3-ks.cfg
81 of 243 18.04.2011 16:35

To stage kickstart files to a web server, copy the kickstart files to the directory specified in the PXE boot file. The above example uses
the web root directory at /var/www/html, although the example could have used any subdirectory with a descriptive name, such as
http://192.168.4.11/kickstart-files/ovs2-ks.cfg.
Figure 49 shows the staged installation tree in the ovs2.2/ directory along with the kickstart file ovs2-ks.cfg and ovs3-ks.cfg in the
web root directory, that is, the /var/www/html of the example 192.168.4.11 boot/web server.
Note: A kickstart file can be staged in any directory on a web, ftp, or nfs server.
PXE Boot Client Configuration

This section discusses how to configure a system’s network interface card (NIC) to be a PXE boot client.
The final step in preparing a PXE boot environment is to enable each system on which you want to perform a PXE/kickstart
installation to boot using PXE. To enable a system’s PXE boot capability, reboot the system and access the system BIOS. From the
system BIOS, locate and enable the PXE boot feature. In some cases, it may be necessary to change the boot order and place PXE
boot at the top of the list.
Tip: Many systems use the "F12" hotkey to access the boot methods selection screen, while others systems require BIOS access to
enable PXE boot. Consult your vendor’s website for details about your hardware.
You are now ready to test your PXE/kickstart installation.
PXE/Kickstart Oracle VM Server Installation and Troubleshooting Tips

This section discusses Oracle VM server PXE/Kickstart installation troubleshooting tips.
If you run into problems when testing your boot server and PXE/kickstart configurations, check /var/log/messages, /var/log/httpd
/access_log and /var/log/httpd/error_logs on the boot server for errors. The /var/log/messages, /var/log/httpd/access_log and /var/log
/httpd/error_log log files on the boot server are where all the DHCP, TFTP and HTTP errors are logged. The next list shows a
PXE/kickstart check list.
PXE/kickstart check list
Check that your boot server is ready to answer requests from the PXE clients.
Is your tftp service is running? From the boot server, type “netstat -l -u | grep tftp”, as shown in the next example to confirm that the
tftp server is running.
# netstat -l -u | grep tftp

udp 0 0 *:tftp *:*
If no output is displayed after typing “netstat -l -u | grep tftp”, the tftp service is not running. The example confirms that the tftp
service is up.
Check if iptables is blocking traffic on the boot server.
Consider disabling iptables to test the boot server by typing “sudo /sbin/service iptables stop”.
Check the BIOS of each PXE client to ensure that PXE boot is enabled.
When the system boots, do you see the NIC looking for an IP address via DHCP? If not, PXE boot is not enabled.
Check that your PXE client is connected to the same network as the boot server.
If the PXE client is located across a router from the boot server, the PXE client may not be able to receive an IP address from the
DHCP service.
How to Register and Update an Oracle VM Server from the Oracle Unbreakable Linux Network
This section discusses how to register and update an Oracle VM server from the Oracle Unbreakable Linux Network. The section
starts with a brief introduction to the Oracle Unbreakable Linux support program and the Oracle Unbreakable Linux Network (ULN).
The section concludes with an overview of how to register and update an Oracle VM server from the Oracle Unbreakable Linux
Network using the up2date program.
Tip: Even after a fresh installation of Oracle VM, it’s advisable to patch the system before any testing to avoid troubleshooting
patched bugs.
The Oracle Unbreakable Linux Support Program

The Oracle Unbreakable Linux support program is an add-on component of Oracle’s enterprise support package for Oracle VM and
Enterprise Linux. The Oracle Unbreakable Linux support program uses the same support infrastructure, support phone number and
My Oracle Support account for Oracle VM and Enterprise Linux as with all other Oracle technologies. Using Oracle support for Oracle
VM and Enterprise Linux allows an Oracle support service request (SR) to transition between groups with issues that require
cross-stack collaboration. For example, if you open a service request for an application issue and the root cause is at the virtualization
82 of 243 18.04.2011 16:35

layer, the service request will automatically transition between the application and Oracle VM support engineers.
The Oracle Unbreakable Linux Network is a cloud resource for Oracle Unbreakable Linux support customers that hosts the Oracle
VM and Enterprise Linux RPM repositories, including software patches, updates and fixes. The Oracle Unbreakable Linux Network’s
web portal is located at http://linux.oracle.com. The Oracle Unbreakable Linux Network web portal provides a dashboard and
management interface for all registered Oracle VM and Enterprise Linux systems and RPM channels. The Oracle Unbreakable Linux
Network repositories are used to patch and install RPMs for Oracle VM and Enterprise Linux systems. Oracle Unbreakable Linux
support customers have the option to patch and install RPMs for Oracle VM and Enterprise Linux systems from the Oracle
Unbreakable Linux Network using the up2date program, or from a local yum repository.
Note: Before you can access the Oracle Unbreakable Linux Network it's necessary to create an Oracle Single Signon account. Your
existing My Oracle Support (MOS) Oracle Single Signon account will not work with the Oracle Unbreakable Linux Network until the
account has been registered with the Oracle Unbreakable Linux Network. Click the Register link at the the Oracle Unbreakable Linux
Network portal to a) create a new Oracle Single Signon account or to b) associate your existing Oracle Single Signon account with the
Oracle Unbreakable Linux Network.
Figure 50 highlights the Oracle Unbreakable Linux Network web portal.
The Oracle Unbreakable Linux Network and My Oracle Support, formerly Metalink, are entirely separate systems, accessed by
different URLs, and use different customer service identifiers (CSIs). My Oracle Support is used to interface with Oracle’s enterprise
support organization, whereas the Oracle Unbreakable Linux Network is used to monitor registered Oracle VM and Enterprise Linux
systems and RPMs. A valid customer service identifier (CSI) for Oracle VM or Enterprise Linux is required to access the RPMs at the
Oracle Unbreakable Linux Network. The customer service identifiers for Oracle VM and Enterprise Linux are only valid for the Oracle
Unbreakable Linux Network, not for My Oracle Support.
Note: The customer service identifiers for Oracle VM and Enterprise Linux cannot be entered at the My Oracle Support portal.
Oracle VM Server Update Options

There are three options for updating an Oracle VM Server. The first option is to use the up2date program to register and update
Oracle VM servers from the Oracle Unbreakable Linux Network. The up2date program communicates with the Oracle Unbreakable
Linux Network over an SSL encrypted connection and downloads and installs all requested packages in an RPM format. Consult
up2date’s man page by typing “man up2date” from dom0 as root to view a comprehensive list of command augments.
Note: Oracle Unbreakable Linux Network access requires a valid Oracle VM customer service identifier.
Figure 51 shows an Oracle VM server being updated from the Oracle Unbreakable Linux Network using the up2date program.
83 of 243 18.04.2011 16:35

The second option is to update Oracle VM servers using a local yum repository. A local yum repository can be hosted on any
Enterprise Linux system that has been registered with the Unbreakable Linux Network with internet access and Apache. Local yum
repositories are populated and synchronized with RPMs that are hosted at the Unbreakable Linux Network using a script and a local
cron job. The up2date program or the yum program can be used with a local yum repository.
Figure 52 shows an Oracle VM server being updated from a local yum repository.
The third option is boot an Oracle VM server with the latest Oracle VM media in the CD-ROM drive and to select the update option
from the boot prompt. Selecting the update option from the boot prompt will update the Oracle VM Server to the version of the Oracle
VM server media. Once the server has been updated, the system must be patched from the Oracle Unbreakable Linux Network or
from a local yum repository.
In the next section, we discuss the process for registering and updating an Oracle VM Server. The examples will all be performed
from dom0 as root using the up2date command.
up2date Program Proxy Configurations

The up2date program defaults may need to modified to meet your environmental and business requirements. For example, if you’re
using a proxy server, you will need to edit the up2date configuration to enter your proxy details. To edit the up2date program defaults,
from dom0 as root type “up2date --configure”. Typing “up2date --configure” lists the up2date program defaults. There are five proxy
configurations that can be edited to allow access from your Oracle VM server to the internet. The following list shows the up2date
proxy configuration items with their default settings and item numbers.
3. enableProxy No
4. enableProxyAuth No
11. httpProxy
21. proxyPassword
22. proxyUser
To edit an up2date program item type “up2date –configure” then type the item number. Then, type C to clear the default value or type
q to quit without saving. Next, type the new value and press Enter to save the new value and to exit. If you need to enter multiple
values, separate them with semicolons (;).
Another default up2date configuration for Oracle VM is to not update the Oracle VM server’s kernel. The default configuration avoids
a “blind” kernel update, which could affect any installed programs or third-party drivers that are dependent on a specific kernel
version. If the default kernel configuration is modified to allow kernel updates, previous kernels are not removed, which allows a
system to be rebooted into the previous kernel if things go wrong.
up2date Program Kernel Configurations

To enable kernel updates, clear the “pkgSkipList” and the “pkgsToInstallNotUp” items by typing the item number and then typing C
to clear the item list. Next, select Enter to save and exit up2date. The following list shows the default “pkgSkipList” and the
“pkgsToInstallNotUp” configurations.
19. pkgSkipList ['kernel*']

20. pkgsToInstallNotUp ['kernel', 'kernel-modules', 'kernel-devel']
The next example shows the default up2date --config configurations with the proxy and kernel items in bold.
# up2date --config
0. adminAddress ['root@localhost']
1. debug No
2. disallowConfChange ['noReboot', 'sslCACert', 'useNoSSLForPackages', 'noSSLSe
3. enableProxy No
4. enableProxyAuth No
5. enableRollbacks No
6. fileSkipList []
7. forceInstall No
8. gpgKeyRing /etc/sysconfig/rhn/up2date-keyring.gpg
84 of 243 18.04.2011 16:35

9. headerCacheSize 40
10. headerFetchCount 10
11. httpProxy
12. isatty Yes
13. keepAfterInstall No
14. networkRetries 5
15. noBootLoader No
16. noReboot No
17. noReplaceConfig Yes
18. noSSLServerURL http://linux-update.oracle.com/XMLRPC
19. pkgSkipList ['kernel*']
20. pkgsToInstallNotUp ['kernel', 'kernel-modules', 'kernel-devel']
21. proxyPassword
22. proxyUser
23. removeSkipList ['kernel*']
24. retrieveOnly No
25. retrieveSource No
26. rhnuuid 3cfb2ee2-6a22-11dd-9022-001c23b73c3a
27. serverURL https://linux-update.oracle.com/XMLRPC
28. showAvailablePacka No
29. sslCACert /usr/share/rhn/ULN-CA-CERT
30. storageDir /var/spool/up2date
31. systemIdPath /etc/sysconfig/rhn/systemid
32. updateUp2date Yes
33. useGPG Yes
34. useNoSSLForPackage No
35. useRhn Yes
36. versionOverride
Enter number of item to edit <return to exit, q to quit without saving>:
Import Oracle’s GPG Key

Before an Oracle VM server can connect to the Oracle Unbreakable Linux Network, Oracle’s GPG key must be imported using the
rpm program. To import the Oracle’s GPG key, from dom0 as root type “rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY”, as shown in
the next example.
#rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY
Once the GPG key has been imported, the Oracle VM Server can be registered at the Oracle Unbreakable Linux Network by typing
“up2date” to start the registration process. The registration process requires you to select a user name and password and to enter a
valid Oracle VM Support Identifier number (CSI). The user name and password that are selected during the registration process
become the user name and password to log on to the Oracle Unbreakable Linux Network. The first time you log on and authenticate
your account on the Oracle Unbreakable Linux Network, you will be prompted to convert your user name and password to an Oracle
SSO user name and password. Oracle SSO accounts can be used across Oracle’s web properties, for example on the Oracle Technical
Network (OTN). You can change your user name and password after you have authenticated your account by clicking the Profile link.
The Oracle Unbreakable Linux Network Registration Process
The following list shows the six steps to register an Oracle VM host with the Oracle Unbreakable Linux Network:
1. Review the Unbreakable Linux Privacy Statement
2. Register a User Account
3. Register a System Profile—Hardware
4. Register a System Profile—Packages
5. Send Profile Information to the Unbreakable Linux Network
6. Finished Registration
Step 1. Review the Unbreakable Linux Privacy Statement

From the Review the Unbreakable Linux Privacy Statement screen use the Alt key to select the Next tab, once the Next tab is
selected press the Enter key to proceed.
Figure 53 shows the Review the Unbreakable Linux Privacy Statement screen.
85 of 243 18.04.2011 16:35

Step 2. Register a User Account

On the Register a User Account screen, enter a User name, Password, Password confirmation and a valid Oracle VM CSI
number. Use the Alt key to select the Next tab, and then press the Enter key to proceed.
Figure 54 shows the Register a User Account screen.
Step 3. Register a System Profile—Hardware

On the Register a System Profile—Hardware screen, accept the defaults and use the Alt key to select the Next tab. Once
the Next tab is selected, press the Enter key to proceed.
Note: The information gathered from the system profile step is saved in your user profile at the Oracle Unbreakable Linux Network.
Figure 55 shows the Register a System Profile—Hardware screen.
Step 4. Register a System Profile— Packages

On the Register a System Profile—Packages screen, accept the defaults and use the Alt key to select the Next tab. Once
the Next tab is selected, press the Enter key to proceed.
86 of 243 18.04.2011 16:35

Figure 56 shows the Register a System Profile—Packages screen.
Step 5. Send Profile Information to the Unbreakable Linux Network

From the Send Profile Information to the Unbreakable Linux Network screen, accept the defaults and use the Alt key to select
the Next tab. Once the Next tab is selected, press the Enter key to proceed.
Figure 57 shows the Send Profile Information to the Unbreakable Linux Network screen.
Step 6 Finished Registration

On the Finished Registration screen, accept the defaults and use the Alt key to select the Next tab. Once the Next tab is selected,
press the Enter key to proceed.
Figure 58 shows the Finished Registration screen.
The Oracle VM server has been successfully registered.
Once a system is registered type “up2date -u” to update the system.
Watch this space

87 of 243 18.04.2011 16:35

Chapter 5: Oracle VM Manager Sizing, Installation, Updates

and the Oracle VM Manager CLI
Oracle VM Manager Introduction

Oracle VM Manager Resource Requirements
Oracle VM Manager Installation Prerequisites
Oracle VM Manager Installation
Deploying the Oracle VM Manager Template
Updating and Patching Oracle VM Manager
Oracle VM Manager Database Backup and Restoration
The Oracle VM Manager Command Line Interface
Oracle VM 2.1.5 to 2.2 Upgrade
Table of Contents
...Supporting Large Oracle VM Manager User Communities
...libaio RPM Package
...Name Resolution
...Host Firewall & Port Test
...SMTP Server
...Oracle VM Manager Guest Console
Required Installation Passwords and Password Complexity Requirements
Applying the 2.2.16 Oracle VM Manager Patch
Updating Oracle VM Manager using the Installation Media
...OVS Database and oc4jadmin Accounts and Passwords
...Updating Oracle VM Manager
...Oracle VM Manager Database Backup
...Oracle VM Manager Database Restore
...Oracle VM Manager Template Prerequisites
...SMTP Server
...User Accounts, Passwords and Password Complexity Requirements
...HTTP and Oracle VM Manager XE Database listening Ports
...Oracle VM Server Cluster Configurations
Download and Deploy the Oracle VM Manager Template
...Running the Deploy_Manager_Template.sh Script
The Oracle VM Manager Command Line Interface Introduction
Downloading the Oracle VM Manager Command Line Interface from ULN
Oracle VM Manager Command Line Interface Installation & Configuration
Oracle VM Manager Command Line Interface Command Examples
...Bulk Commands and Batch Scripting
...Guest Backup
...Create a Server Pool
Programming with the Oracle VM Manager Command Line Interface

Oracle VM Manager is a bare bones stand-alone management solution for Oracle VM servers, Oracle VM server pools and Oracle VM
guests. Oracle VM Manager is a light weight private source cluster ready Oracle Application Development Framework 10g (ADF)
application that installs on Enterprise Linux. Oracle VM Manager also has a command line interface that allows Oracle VM Manager
administrative tasks to be perform from the command line or executed using scripts.
Oracle VM Manager is supported on a physical or virtual Enterprise Linux operating system. Oracle VM Manager is a great candidate
for virtualization on Oracle VM or Oracle VM VirtualBox. Virtualizing Oracle VM saves on hardware costs and improves application
flexibility while reducing data center space.
Note: Oracle VM Manager is not supported and should not be installed in Oracle VM server's dom0.
List 1 shows the supported Enterprise Linux operating systems and virtualization platforms for Oracle VM Manager.
88 of 243 18.04.2011 16:35

Oracle Enterprise Linux 4 update 5 or later

Red Hat Enterprise Linux 4 or later
Oracle VM for x86
Oracle VM VirtualBox
Oracle VM Manager is distributed in three formats; a) as an ISO file that installs on Enterprise Linux b) as a pre-packaged Oracle VM
template (with the CLI) and c) as the Oracle VM Management Pack plug-in.
Oracle VM Manager is considered the leading edge development platform for the Oracle VM Management Pack, which is an Oracle
Enterprise Manager plug-in. Oracle VM Manager 2.1.2 was ported to Oracle Enterprise Manager 10.2.0.5 and Oracle VM Manager
2.2 was ported to Oracle Enterprise Manager 11g R1. Expect at least a six month lag for Oracle to port a new Oracle VM Manager
release to the Oracle VM Management Pack.
Note: The Oracle VM Management Pack is licensed software; Oracle VM Server and Oracle VM Manager are not licensed software.
Figure 1 highlights the Oracle VM Manager 2.2 user interface.
The next chart shows the matrix of supported Oracle VM Manager and Oracle VM server combinations.
OVM 2.1.0 2.1.1 2.1.2 2.1.5 2.2.0
OVS --------+------+------+-----+------
2.1.0 O | O | O | O | O
2.1.1 X | O | O | O | O
2.1.2 X | X | O | O | O
2.1.5 X | X | X | O | O
2.2.0 X | X | X | X | O
2.2.1 X | X | X | X | O
Note: Oracle VM Manager can manage many Oracle VM server pools, e.g. Oracle VM Manager 2.2 can manage one pool of Oracle
VM 2.1.x servers, another pool of Oracle VM 2.2 servers, and many more server pools; But within the same server pool, all the servers
must have the same version of Oracle VM server and the Oracle VM agent. Mixing Oracle VM 2.1.x and 2.2.x servers in the same pool
is not supported.

Oracle's recommends 2 GB of memory and one 1.83 GHz CPU for VM Manager. Oracle's minimum resource recommendations for VM
Manager is appropriate for a small Oracle VM pool with a hand full of Oracle VM servers without the Command Line Interface.
Oracle VM Manager resource requirements can be quickly tested with Oracle VM Manager user interface (UI) page response times.
For example, if the Oracle VM Manager user interface is painfully slow or if you experience page timeouts adding memory and CPUs
will speed up the user interface and eliminate page timeouts.
Tip: Oracle VM Manager on VMware is performance challenged and will need substantially more CPU and memory than the minimum
89 of 243 18.04.2011 16:35

recommendations. Oracle VM Manager is a great fit running on a virtualized Enterprise Linux operating system on an Oracle VM
server.
Table 1 lists Oracle's recommended minimum resource requirements for Oracle VM Manager 2.x.
Description Minimum
Resources
Memory 2 GB
Processor 1 1.83 GHz

CPU
Swap Space 2 GB
Hard Disk 4 GB
Space
The memory and CPU requirements for Oracle VM Manager 2.x scales up as the number of Oracle VM pool members grow.
Table 2 shows the recommended CPU and memory allocation for three different Oracle VM Manager deployments.
Number of Memory CPU Swap

Oracle VM
Server Pool
Members
< 10 2 GB 1 1.83 GHz 2 GB
> 10 4 GB 2 2 GHz 8 GB
> 32 8 GB 4 2 GHz 8 GB
Supporting Large Oracle VM Manager User Communities

If Oracle VM Manager will be supporting over 10 users ensure that the OVM Manager host has at least 2.5G of RAM with the
following oc4j script modifications.
On the OVM Manager host, edit the “/opt/oc4j/bin/oc4j” file and change this line:
OC4J_JVM_ARGS="-XX:PermSize=256m -XX:MaxPermSize=512m"
to:
OC4J_JVM_ARGS="-XX:PermSize=512m -XX:MaxPermSize=1024m -Xms2048m -Xmx2048m"
Next, restart the OC4J service on the Oracle VM Manager hosts by typing "service oc4j restart".
libaio RPM Package

Oracle VM Manager uses the Oracle Database 10g Express Edition as the default data repository. The Oracle Database 10g Express
Edition requires the libaio package release 0.3.96 or above.
To validate if your Enterprise Linux system has the libaio 0.3.96 or above package, as root “type rpm -q libaio” as shown in the next
example.
# rpm -q
libaio
libaio-
0.3.106-3.2
If you get no result from typing “rpm -q libaio” you will need to install the libaio package. To install libaio from the Oracle
Unbreakable Linux network type “up2date -i libaio” as shown in the next example.
# up2date -i
libaio
Name Resolution
Before you install Oracle VM Manager ensure that the Oracle VM Manager server's host name is properly entered in the /etc/hosts
file. The host name, i.e. fully qualified domain name (FQDN) must be associated with the public IP address.
The next example shows an “improper” Enterprise Linux /etc/hosts file entry.
127.0.0.1 servername.com servername localhost.localdomain
90 of 243 18.04.2011 16:35

localhost
The next example shows the proper syntax for an Enterprise Linux /etc/hosts file entry.
127.0.0.1 localhost.localdomain localhost

Host Firewall & Port Test

Oracle VM Manager will reserve tcp ports 8888, 8899 and 4443 for the Oracle VM Manager application. Before installing Oracle VM
Manager on the Oracle VM Manager host, check that ports 8888 and 8899 are free. As root type “netstat -na |grep 8888” and then
type “netstat -na |grep 8899”. If you do not get a response by typing “netstat -na |grep 8888” and “netstat -na |grep 8899” the ports
are free. If you do get a response the ports must be released before you can install Oracle VM Manager.
If your Enterprise Linux host uses iptables you will need to allow tcp traffic on ports 8888, 8899 and 4443 to allow access to the
Oracle VM Manager portal.
To allow tcp traffic on ports 8888, 8899 and 4443 on the Oracle VM Manager host, as root or with sudo type the following three
commands:
# system-config-securitylevel-tui -q
-p 8888
-p 8899
-p 4443
SMTP Server
Oracle VM Manager has several user notification features that require the use of an SMTP server. During the Oracle VM Manager
installation you will be asked to a) enter the SMTP server IP address or fully qualified domain name and b) the SMTP port and c) a
valid email address and password for the Oracle VM Manager admin account.
Entering the SMTP details during the installation is optional and can be configured after the installation by using the
“update_email.sh” script on the Oracle VM Manager host or by editing the values in Oracle VM Manager repository database table
ovs_sys_value under the OVS schema.
Oracle VM Manager Guest Console

The semi integrated Oracle VM Manager guest console requires the Oracle VM Manager host to have the TightVNC package
installed. The TightVNC package allows non-Linux users, i.e.Windows or MAC clients to use the TightVNC guest console.
The TightVNC package can be downloaded from: http://oss.oracle.com/oraclevm/manager/RPMS/
The next example shows how to download and install the TightVNC package on the Oracle VM Manager host as root. The TightVNC
package can be downloaded and installed in any directory, i.e. /tmp on the Enterprise Linux host.
# wget http://oss.oracle.com/oraclevm/manager/RPMS/tightvnc-
java-1.3.9-3.noarch.rpm
# rpm -ivh tightvnc-java-version.noarch.rpm
Note: If you use Firefox on Linux to access the Oracle VM Manager user interface and would like to use the TightVNC guest console
you will also need to install the Oracle VM Manager console plug in (ovm-console) on your Linux workstation. The ovm-console
package is available at http://oss.oracle.com/oraclevm/manager/RPMS/.
Required Installation Passwords and Password Complexity Requirements

During the Oracle VM Manager Installation you will be prompted for a total of five required passwords.
List 2 shows the five required passwords.
The password for the Oracle XE SYS and SYSTEM accounts.

The password for the Oracle VM Manager OVS database schema.
The password for the OC4J oc4jadmin account.
The password for the Web Service keystore file.
The password for the admin account.
List 3 shows the password complexity requirements for the required passwords.
91 of 243 18.04.2011 16:35

The password must begin with an uppercase or lowercase letter.

The password may include the following characters:
Numbers (such as 1, 2, 3, and so on), letters (from a to z, A to Z), and underscores (_).
Examples of valid passwords: Password01, Password_123, password.
Tip: Select and write down all the passwords before you start the installation. Unfortunately you may not be able to use the same
password for all four services because the Web Service has a slightly different password policy.

An Oracle VM Manager installation takes approximately 5-15 minutes depending on the performance of the Oracle VM Manager host.
During the Oracle VM Manager installation, you are required to set the following ports. Please note that the installer program will
provide defaults.
The Oracle XE HTTP port. The default port number is 8080.

The Oracle XE listening port. The default port number is 1521.
1. Download the Oracle VM Manager ISO file from the Oracle eDelivery Linux portal.
2. Log in to the Oracle VM Manager host as root.
3. Copy the Oracle VM Manager ISO file to a directory on the Oracle VM Manager host, i.e. to the /tmp directory.
4. Create a directory to mount the ISO file, i.e. mkdir -p /tmp/mount-point.
5. Mount the ISO file by typing “mount -o loop,ro OracleVM-Manager-version.iso mount-point”
6. Change into the directory where the ISO file is mounted, i.e. “cd /tmp/mount-point.
7. Type “sh runInstaller.sh” to start the Oracle VM Manager installation and then type “1” to select the “Install Oracle VM Manager”
option as shown in the next example.
# sh runInstaller.sh
Welcome to Oracle VM Manager 2.2
Please enter the choice: [1|2|3]

1. Install Oracle VM Manager
2. Uninstall Oracle VM Manager
3. Upgrade Oracle VM Manager
1
Starting Oracle VM Manager 2.2 installation …
8. From the ”Do you want to install a new database or use an existing one? [1|2]” prompt type ”1” to select the ”Install a new Oracle
XE database on localhost” option as shown below.
Do you want to install a new database or use an existing one? [1|2]
1. Install a new Oracle XE database on localhost
2. Use an existing Oracle database in my network
1
Prepare to install the Oracle XE database ...

Checking the supported platforms ... Done
Checking the prerequisite packages are installed ... Done
Checking the available disk space ... Done
Installing the oracle-xe-univ package (rpm) now ...

Done
Oracle Database 10g Express Edition Configuration

-------------------------------------------------
This will configure on-boot properties of Oracle Database 10g Express
Edition. The following questions will determine whether the database should
be starting upon system boot, the ports it will use, and the passwords that
will be used for database accounts. Press <Enter> to accept the defaults.
Ctrl-C will abort.
9. Press the enter key to select the default HTTP port 8080 for Oracle Application Express.
Specify the HTTP port that will be used for Oracle Application Express [8080]:
10. Press the enter key to select the default database listener port 1521.
Specify a port that will be used for the database listener [1521]:
11. Type a secure password for the database accounts.

Specify a password to be used for database accounts. Note that the same
password will be used for SYS and SYSTEM. Oracle recommends the use of
different passwords for each database account. This can be done after
initial configuration:
92 of 243 18.04.2011 16:35

Confirm the password:
12. Type y and press enter to configure the Oracle Database 10g Express Edition to start on boot.
Do you want Oracle Database 10g Express Edition to be started on boot (y/n) [y]:y
Starting Oracle Net Listener...Done

Configuring Database...Done
Starting Oracle Database 10g Express Edition Instance...Done
Installation Completed Successfully.
To access the Database Home Page go to "http://127.0.0.1:8080/apex"
Checking the availability of the database …
Set default database schema to 'OVS'.
12. Type a secure password for the OVS account.

Please enter the password for account 'OVS':
Creating the Oracle VM Manager database schema ...Done
Installing the ovs-manager package (rpm) ...

Done
Installing the oc4j package (rpm) ...

Done
Please enter the password for account 'oc4jadmin':

Starting OC4J ... Done.

To access the OC4J Home Page and change the password go to http://127.0.0.1:8888/em
Deploying Oracle VM Manager application to OC4J container.

Creating connection pool ... Done
Creating data source ... Done
Deploying application help ... Done
Deploying application ... Done
13. Type a secure password for the keystore password for the Web Service.
Please enter the keystore password for the Web Service:
Setting keystore password for Web Service ... Done
14. You can enable HTTPS for the Oracle VM Manager user interface by typing Y, or n if you do not want to enable HTTPS. If you
select Y a private SSL certificate will be generated and used to secure the Oracle VM Manager user interface. In the example we
typed Y to enable HTTPS.
Do you want to use HTTPS access for Oracle VM Manager (Y|n)?y
Configuring OC4J to use HTTPS ... Done

Stopping OC4J ... Done
Starting OC4J ... Done
15. Type a secure password for the default Oracle VM Manager admin account. You will use the admin user name and the admin
password to log into the Oracle VM Manager user interface.
Please enter the password for the default account 'admin':

Configuring SMTP server …
16. Type the outgoing SMTP mail server IP address or the fully qualified domain name (optional).
Please enter the outgoing SMTP mail server (e.g. - mail.abc.com, mail.abc.com:25): mail.my.domain.net:25
Mail server checking, may need some time, please wait...
Setting the SMTP server to mail.my.domain.net...
Done
17. Type the e-mail address for the admin account.

Please enter an e-mail address for account 'admin': my.admin.account@my.domain.com
Confirm the e-mail address: my.admin.account@my.domain.com
The console feature is not enabled by default.
93 of 243 18.04.2011 16:35

For detailed setup, refer to Oracle VM Manager User's Guide
Installation of Oracle VM Manager completed successfully.
To access the Oracle VM Manager 2.2 home page go to:

https://<Oracle VM Manager Host>:4443/OVS
To access the Oracle VM Manager web services WSDL page go to:

https://<Oracle VM Manager Host>:4443/OVSWS/LifecycleService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/ResourceService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/PluginService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/ServerPoolService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/VirtualMachineService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/AdminService.wsdl
To access the Oracle VM Manager help page go to:

https://<Oracle VM Manager Host>:4443/help/help
If you selected HTTP:

http://<Oracle VM Manager Host>:8888/OVS

http://<Oracle VM Manager Host>:8888/OVSWS/LifecycleService.wsdl
http://<Oracle VM Manager Host>:8888/OVSWS/ResourceService.wsdl
http://<Oracle VM Manager Host>:8888/OVSWS/PluginService.wsdl
http://<Oracle VM Manager Host>:8888/OVSWS/ServerPoolService.wsdl
http://<Oracle VM Manager Host>:8888/OVSWS/VirtualMachineService.wsdl
http://<Oracle VM Manager Host>:8888/OVSWS/AdminService.wsdl

http://<Oracle VM Manager Host>:8888/help/help
The Oracle VM Manager application was successfully installed and can be accessed by typing https://<Oracle VM Manager
Host>:4443/OVS if you selected HTTPS or http://<Oracle VM Manager Host>:8888/OVS if you did not select HTTPS in a web
browser. Enter the admin user name and the admin password to log into the Oracle VM Manager portal.
List 2 shows the supported Web browsers for the Oracle VM Manager user interface.
Microsoft Internet Explorer 5.5 or later

Firefox 1.0.4 or later (Linux and Windows)
Safari 2.x (Mac)
If you configured the SMTP server the admin account will receive the following email:
OVM Administrator
https://<Oracle VM Manager Host>:4443/OVS

https://<Oracle VM Manager Host>:4443/OVSWS/LifecycleService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/ResourceService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/PluginService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/ServerPoolService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/VirtualMachineService.wsdl
https://<Oracle VM Manager Host>:4443/OVSWS/AdminService.wsdl

https://<Oracle VM Manager Host>:4443/help/help
Figure 2 shows the Oracle VM Manager Log in page.
94 of 243 18.04.2011 16:35

The Oracle Database Express Edition portal can be accessed from the Oracle VM manager server locally by entering
http://127.0.0.1:8080/apex or remotely by entering the ip address or the FQDN followed by :8080/apex, i.e. http://<Oracle VM
Manager Host>:8080/apex in a web browser. You can use the SYS, SYSTEM or the OVS user account and the associated password to
log in to the Oracle Database Express Edition portal.
Figure 3 shows the Oracle Database Express Edition portal login page.
The Application Server Control portal is also part of the Oracle VM Manager install. The oc4jadmin account can be maintained from
the Application Server Control portal. The Application Server Control portal can only be access locally from the Oracle VM Manager
host by typing http://127.0.0.1:8888/em. Use the oc4jadmin account and password to access the Application Server Control portal.
Figure 4 shows the Application Server Control login page.
Applying the 2.2.16 Oracle VM Manager Patch

The Oracle VM Manager 2.2.16 patch provides several Oracle VM Manager updates and bug fixes. The Oracle VM Manager 2.2.16
patch is available from the Unbreakable Linux Network in the Oracle Software for Enterprise Linux channels.
Table 3 lists the ovs-manager-2.2-16.noarch package and the channels where the ovs-manager-2.2-16.noarch package is hosted.
Table 3
95 of 243 18.04.2011 16:35

Package Channel Description

ovs-manager-2.2-16.noarch Oracle Software for Enterprise Linux 4 (i386) Oracle Software for Enterprise Linux 4 (i386)
ovs-manager-2.2-16.noarch Oracle Software for Enterprise Linux 4 (x86_64) Oracle Software for Enterprise Linux 4 (x86_64)
ovs-manager-2.2-16.noarch Oracle Software for Enterprise Linux 5 (x86_64) Oracle Software for Enterprise Linux 5 (x86_64)
ovs-manager-2.2-16.noarch Oracle Software for Enterprise Linux 5 (i386) Oracle Software for Enterprise Linux 5 (i386)
While applying the ovs-manager-2.2-16.noarch package you will be prompted for a total of three passwords.
List 4 shows the three passwords.
The password for the Oracle XE SYS account.

The next example shows how to install the ovs-manager-2.2-16.noarch package as root or using sudo.
Updating Oracle VM Manager using the Installation Media

Unlike Oracle VM Server, which has multiple update options Oracle VM Manager is updated by downloading the media from Oracle’s
eDelivery portal and executing an installation script. Oracle VM media is freely downloadable from Oracle’s eDelivery system at
http://edelivery.oracle.com/linux.
Updating Oracle VM Manager is accomplished by running the Oracle VM Manager installation program on the Oracle VM Manager
host and to select the upgrade installation option. The Oracle VM Manager installation program will ask for the passwords for the
existing Oracle VM Manager OVS database account and the oc4jadmin account. The Oracle VM Manager OVS database account and
the oc4jadmin account passwords are selected during the Oracle VM Manager installation process. The account passwords for the
Oracle VM Manager OVS database account and the oc4jadmin account can be managed from their respected web portals.
Note: The Oracle Database Express Edition portal and the Application Server Control portal are installed by default with the Oracle
VM Manager portal. The Oracle VM Manager Template does not enclude the Oracle Database Express Edition portal and the
Application Server Control portal.
OVS Database and oc4jadmin Accounts and Passwords

The OVS account can be maintained from the Oracle Database Express Edition portal. The Oracle Database Express Edition portal is
a part of the Oracle VM Manager install and can be accessed locally by entering http://127.0.0.1:8080/apex or remotely by entering
the ip address or the FQDN followed by :8080/apex, i.e. http://OracleVM_Manager_fqdn:8080/apex in a web browser.
The next Figure shows the Oracle Database Express Edition portal login page.
The Application Server Control portal is a part of the Oracle VM Manager install. The oc4jadmin account can be maintained from the
Application Server Control portal. The Application Server Control portal can only be accessed locally from the Oracle VM server
console by entering http://127.0.0.1:8888/em in a local web browser.
The next Figure shows the Application Server Control portal login page.
96 of 243 18.04.2011 16:35

Updating Oracle VM Manager

The following section will review each of the steps of a successful Oracle VM Manager upgrade.
The update process requires you to select the upgrade option from the install prompt as well as enter the passwords for the Oracle
VM Manager OVS database account and the Oracle VM Manager oc4jadmin account.
Step 1 is to download the ISO image from http://edelivery.oracle.com/linux.
Once the ISO image is downloaded and placed on the Oracle VM Manager host or made available from a network share then it should
be mounted as root user.
The following example shows how to mount the Oracle VM Manager ISO image from the Oracle VM Manager server console as root
user.
# mount -o ro,loop OracleVM-Manager-2.1.2.iso mnt/
# cd mnt
# ls
EULA LICENSE readme.txt runInstaller.sh scripts source TRANS.TBL
Step 2 is to execute the runInstaller.sh script within the Oracle VM Manager root directory from the Oracle VM Manager server
console as root.
The following example shows how to execute the runInstaller.sh from the Oracle VM Manager server console as root user.
Step 3 is to select the Upgrade option (number 3) from the install prompt. The next example shows the installation choice prompt.
Step 4 asks if you would like to proceed or cancel the upgrade. You can enter y to proceed or n to cancel the upgrade. The following
example shows the upgrade acceptance prompt.
Are you sure you want to upgrade Oracle VM Manager from version
2.1.1 to 2.1.2 ? [y|N] y
Step 5 is to enter the OVS password. The following example shows the OVS installation password prompt.
Please enter the password for database account 'OVS':
Step 6 is to enter the oc4jadmin password. The following example shows the oc4jadmin installation password prompt.
Step 7 is to enter yes or no to backup the Oracle VM Manager database. This step is optional, i.e. entering no will not abort the
upgrade. The following example shows the Oracle VM Manager database backup prompt.
Would you like to back up the Oracle VM Manager database?
Now back up the Oracle VM Manager database to /usr/lib/oracle/xe/app
/oracle/product/10.2.0/server/dump-2008-10-06-05-24-25-PM.dmp
A successful upgrade will exit with the following informational warnings.
Export terminated successfully with warnings.
Done.
Now upgrading the database schema ... Done.
Now upgrading the Oracle VM Manager application ...

Done.
Upgrade Oracle VM Manager successfully.
Please check the log /var/log/ovm-manager/upgrade_2.1.1_2.1.2.log for
more information.
97 of 243 18.04.2011 16:35

To access the Oracle VM Manager 2.1.2 home page go to:

http://ovm01.com:8888/OVS

http://ovm01.com:8888/help/help
#
Step 8 is to validate the Oracle VM Manager version number from the Oracle VM Manager home page. The Oracle VM home page
URL can be accessed locally via http://127.0.0.1:8888/OVS or from a remote web browser via http://ip_address_of_host:8888/OVS or
http://fqdn:8888/OVS. From the Oracle VM Manager home page you can validate the version number from the footer of the home
page.
Oracle VM Manager Database Backup

The Oracle VM Manager batabase can be backed up using the backup.sh script. Before you backup the Oracle VM Manager database,
access Oracle VM Manager and ensure that the status of all of the Oracle VM servers is "Running". You will need the following details
for the backup procedure a) the password for database account 'OVS', which was assigned during the Oracle VM Manager install, b)
the path for dump file, i.e. /tmp/ovs.dmp and c) the path for log file, i.e. /tmp/ovs.log. We will conclude Step 1 with a complete Oracle
VM Manager database restore.
The following example shows how to execute the “/opt/ovs-manager-2.2/bin/backup.sh” script from the Oracle VM Manager host as
root user.
# sh /opt/ovs-manager-2.2/bin/backup.sh
Next to backup the Oracle VM Manager database select the Backup Oracle VM Manager option (number 1) from the backup
prompt. The next example shows the backup selection prompt.
sh /opt/ovs-manager-2.2/bin/backup.sh
Welcome to Oracle VM Manager
Completed backup:
Please enter the choice: [1|2]

1. Back up Oracle VM Manager
2. Restore Oracle VM Manager
1
The next example reviews a complete Oracle VM Manager database backup. Access the Oracle VM Manager server console as root
and type "sh /opt/ovs-manager-2.2/bin/backup.sh" to start the Oracle VM Manager database backup as shown in the following
example.
Completed backup:

1
Back up data now ...

Please enter the path and filename for the dump file (eg /tmp/ovs.dmp):/tmp/ovs.dmp
Please enter the path and filename for the log file (eg /tmp/ovs.log):/tmp/ovs.log
SQL*Plus: Release 10.2.0.1.0 - Production on Sat Oct 17 10:00:51 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
SQL>
PL/SQL procedure successfully completed.
SQL> Disconnected from Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
Export: Release 10.2.0.1.0 - Production on Sat Oct 17 10:00:52 2009
98 of 243 18.04.2011 16:35

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
Export done in US7ASCII character set and AL16UTF16 NCHAR character set
server uses AL32UTF8 character set (possible charset conversion)
. exporting pre-schema procedural objects and actions
. exporting foreign function library names for user OVS
. exporting PUBLIC type synonyms
. exporting private type synonyms
. exporting object type definitions for user OVS
About to export OVS's objects ...
. exporting database links
. exporting sequence numbers
. exporting cluster definitions
. about to export OVS's tables via Conventional Path ...
. . exporting table OVS_AGENT 2 rows exported
EXP-00091: Exporting questionable statistics.
. . exporting table OVS_ALERT 11 rows exported
. . exporting table OVS_CATEGORY 1 rows exported
. . exporting table OVS_CDROM 5 rows exported
. . exporting table OVS_CDROM_RESOURCE 0 rows exported
. . exporting table OVS_GROUP 4 rows exported
. . exporting table OVS_HD_TEMP
. . exporting table OVS_IMG_OS 0 rows exported
. . exporting table OVS_IMG_TEMP
. . exporting table OVS_LOCK 19 rows exported
. . exporting table OVS_MAP 35 rows exported
. . exporting table OVS_NIC_TEMP
. . exporting table OVS_OS_RESOURCE 17 rows exported
. . exporting table OVS_PARTNER 0 rows exported
. . exporting table OVS_PREFERRED_SERVER 0 rows exported
. . exporting table OVS_PRIVILEGE 4 rows exported
. . exporting table OVS_PRIVILEGE_ROLE 5 rows exported
. . exporting table OVS_RESOURCE 4 rows exported
. . exporting table OVS_ROLE 3 rows exported
. . exporting table OVS_SERVER 2 rows exported
99 of 243 18.04.2011 16:35

. . exporting table OVS_SITE 1 rows exported

. . exporting table OVS_STATISTIC 16593 rows exported
. . exporting table OVS_SYS_VALUE 8 rows exported
. . exporting table OVS_TASK 0 rows exported
. . exporting table OVS_USER 19 rows exported
. . exporting table OVS_USER_GROUP 75 rows exported
. . exporting table OVS_USER_ROLE 19 rows exported
. . exporting table OVS_USER_SITE 19 rows exported
. . exporting table OVS_VD_IMG 39 rows exported
. . exporting table OVS_VIRTUAL_DISK 73 rows exported
. . exporting table OVS_VM_GEN_INFO 25 rows exported
. . exporting table OVS_VM_IMG 25 rows exported
. . exporting table OVS_VM_NETWORK 25 rows exported
. . exporting table OVS_VM_SNAPSHOT 0 rows exported
. . exporting table OVS_WORKSPACE_TEMP
. exporting synonyms
. exporting views
. exporting stored procedures
. exporting operators
. exporting referential integrity constraints
. exporting triggers
. exporting indextypes
. exporting bitmap, functional and extensible indexes
. exporting posttables actions
. exporting materialized views
. exporting snapshot logs
. exporting job queues
. exporting refresh groups and children
. exporting dimensions
. exporting post-schema procedural objects and actions
. exporting statistics
Connected to:
SQL>
100 of 243 18.04.2011 16:35

Done
Please check the log file /tmp/ovs.log
#
Oracle VM Manager Database Restore

The Oracle VM Manager batabase can be restored from a backup using the backup.sh script. Before you restore the Oracle VM
Manager database, access Oracle VM Manager and ensure that the status of all of the Oracle VM servers is "Running".
The following details are required to restore the Oracle VM Manager databasee a) the password for database account 'OVS', which
was assigned during the Oracle VM Manager install, b) the path for dump file, i.e. /tmp/ovs.dmp and c) the path for log file, i.e.
/tmp/ovs.log.
The following example shows how to execute the “/opt/ovs-manager-2.2/bin/backup.sh” script from the Oracle VM Manager server
console as root user.
Next, to backup the Oracle VM Manager database, select the Restore Oracle VM Manager option (number 2) from the backup
Completed backup:

2
The next example reviews a complete Oracle VM Manager database restore. Access the Oracle VM Manager server console as root
and type "sh /opt/ovs-manager-2.1/bin/backup.sh" to start the Oracle VM Manager database restore as shown in the following
example.
The next example reviews an Oracle VM Manager database restore procedure. Access the Oracle VM Manager server console as root
and type "sh /opt/ovs-manager-2.1/bin/backup.sh" as shown in the following example.

2
Restore data now ...

Please enter the password for database account 'SYS':
Import: Release 10.2.0.1.0 - Production on Sun Oct 18 11:15:24 2009
Export file created by EXPORT:V10.02.01 via conventional path
Warning: the objects were exported by OVS, not by you
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses AL32UTF8 character set (possible charset conversion)
. . importing table "OVS_AGENT" 2 rows imported
. . importing table "OVS_ALERT" 11 rows imported
. . importing table "OVS_CATEGORY" 1 rows imported
. . importing table "OVS_CDROM" 5 rows imported
. . importing table "OVS_CDROM_RESOURCE" 0 rows imported
. . importing table "OVS_GROUP" 4 rows imported
. . importing table "OVS_IMG_OS" 0 rows imported
. . importing table "OVS_LOCK" 19 rows imported
101 of 243 18.04.2011 16:35

. . importing table "OVS_MAP" 35 rows imported

. . importing table "OVS_OS_RESOURCE" 17 rows imported
. . importing table "OVS_PARTNER" 0 rows imported
. . importing table "OVS_PREFERRED_SERVER" 0 rows imported
. . importing table "OVS_PRIVILEGE" 4 rows imported
. . importing table "OVS_PRIVILEGE_ROLE" 5 rows imported
. . importing table "OVS_RESOURCE" 4 rows imported
. . importing table "OVS_ROLE" 3 rows imported
. . importing table "OVS_SERVER" 2 rows imported
. . importing table "OVS_SITE" 1 rows imported
. . importing table "OVS_STATISTIC" 16593 rows imported
. . importing table "OVS_SYS_VALUE" 8 rows imported
. . importing table "OVS_TASK" 0 rows imported
. . importing table "OVS_USER" 19 rows imported
. . importing table "OVS_USER_GROUP" 75 rows imported
. . importing table "OVS_USER_ROLE" 19 rows imported
. . importing table "OVS_USER_SITE" 19 rows imported
. . importing table "OVS_VD_IMG" 39 rows imported
. . importing table "OVS_VIRTUAL_DISK" 73 rows imported
. . importing table "OVS_VM_GEN_INFO" 25 rows imported
. . importing table "OVS_VM_IMG" 25 rows imported
. . importing table "OVS_VM_NETWORK" 25 rows imported
. . importing table "OVS_VM_SNAPSHOT" 0 rows imported
Import terminated successfully without warnings.
Done
#

This section will review how to deploy the Oracle VM Manager Template. The section starts with a review of the Oracle VM Manager
Template prerequisites, followed with the procedure to deploy the Oracle VM Manager Template on an Oracle VM server.
Oracle VM Manager Template Prerequisites

Table 4 shows the disk, storage, RAM and networking prerequisits for an Oracle VM server hosting the Oracle VM Manager Template.
Oracle VM Server Minimum Requirements

Resource
Requirements
Disk Space The Oracle VM Manager Template requires at

least 4GB of free space on the Oracle VM
server for the compressed and uncompressed
files.
Storage Repositories The Oracle VM Manager Template can be

configured on an Oracle VM server with a
local OCFS2 storage repository or with a
shared OCFS2 or NFS storage repository. If
the Oracle VM server is using a shared
storage repository, mount only the root
repository before running the
Deploy_Manager_Template.sh script. Only
after the Oracle VM Manager Template has
been deployed can extended storage
repositories be configured and mounted.
RAM The Oracle VM Manager Template requires at

least 2GB of free memory on the Oracle VM
Server.
Networking The Oracle VM Server will need to have a

working Xen bridge for the Oracle VM
Manager Template. The Oracle VM Manager
Deploy_Manager_Template.sh script will list
the “Available network bridges” and ask to
select a bridge for the Oracle VM Manager
Teample. The xen bridge that is selected, i.e.
xenbr0, xenbr1, xenbr2, etc.. must be active
and accessable on the network.
102 of 243 18.04.2011 16:35

Note: The Oracle VM Manager Template

requires a static IP address that will be
assigned to the Oracle VM Manager template
virtual machine. The Oracle VM Manager
Deploy_Manager_Template.sh script will ask
for the following network details for the
Oracle VM Manager virtual machine:
* Static IP address
* Netmask
* Default Gateway IP address
* DNS Server IP address
* Hostname
SMTP Server
Oracle VM Manager has several user notification features that require the use of an SMTP server. During the Oracle VM Manager
installation you will be asked to a) enter the SMTP server IP address or fully qualified domain name and b) the SMTP port and c) a
valid email address and password for the Oracle VM Manager admin account.
Entering the SMTP details during the template configuration is optional and can be configured after the template is deployed using
the “update_email.sh” script or by editing the values in Oracle VM Manager repository database table ovs_sys_value under the OVS
schema.
Tip: The Deploy_Manager_Template.sh script can be run with options “--skipsmtpcheck” and “--skipmailcheck” to skip the SMTP
server check and email check.
User Accounts, Passwords and Password Complexity Requirements

The Oracle VM Manager Template configuration will create two operating system users with default passwords; root with password
ovsroot and oracle with password oracle. The default passwords for the root and oracle accounts should be chaged once the template
is deployed. The oracle account belongs to the oinstall and dba groups.
The default vnc console password for the Oracle VM Manager Template is oracle.
During the Oracle VM Manager Installation you will be prompted for a total of six required passwords.
List 5 shows the six required passwords.
The password for the Oracle XE SYS and SYSTEM accounts.

The password for the Web Service keystore file.
The password for the admin account.
The Oracle VM agent password.
Tip: The Oracle VM agent password can be reset by typing “service ovs-agent configure” from dom0 as root.
List 6 shows the password complexity requirements for the required passwords.
The password must begin with an uppercase or lowercase letter.

The password may include the following characters:
Numbers (such as 1, 2, 3, and so on), letters (from a to z, A to Z), and underscores (_).
Examples of valid passwords: Password01, Password_123, password.
Tip: Select and write down all the passwords before you start the installation. Unfortunately you may not be able to use the same
password for all four services because the Web Service has a slightly different password policy.
HTTP and Oracle VM Manager XE Database listening Ports

During the Oracle VM Manager Template configuration phase the installer asks to set the XE HTTP port and XE Database port. The
installer program will provide defaults for the XE HTTP port and XE Database port.
The Oracle XE HTTP port. The default port number is 8080.

The Oracle XE listening port. The default port number is 1521.
Oracle VM Server Cluster Configurations

If the Oracle VM Manager Template is being deployed on an Oracle VM server that had been in an Oracle VM server pool, the Oracle
VM servers cluster configurations must be cleaned before deploying the template.
103 of 243 18.04.2011 16:35

To clean an Oracle VM server's cluster configurations it is necessary to a) empty the /etc/ocfs/cluster.conf file b) delete and recreate
the local BerkleyDB and c) run the cleanup.py script to stop o2cb heartbeat, offline o2cb, remove o2cb configuration file, umount
ovs-agent storage repositories and to cleanup ovs-agent local database.
To clear /etc/ocfs/cluster.conf file type “cat /dev/null> /etc/ocfs2/cluster.conf” from dom0, as shown in the next example.
# cat /dev/null> /etc/ocfs2

/cluster.conf
To remove the local BerkleyDB first type “service ovs-agent stop”, which stops the Oracle VM agent. Next, type “rm -fr /etc/ovs-
agent/db/*” to delete the BerkleyDB. Finally, type “service ovs-agent start” to start the Oracle VM agent, which also recreate a new
local BerkleyDB.
To stop the o2cb heartbeat, offline o2cb, remove o2cb configuration file, unmount ovs-agent storage repositories and to cleanup
ovs-agent local database, type "/opt/ovs-agent-2.3/utils/cleanup.py" and then type “y” as shown in the next example.
# /opt/ovs-agent-2.3/utils
/cleanup.py
This is a cleanup script for
ovs-agent.
It will try to do the following:
*) stop o2cb heartbeat

*) offline o2cb
*) remove o2cb configuration file
*) umount ovs-agent storage
repositories
*) cleanup ovs-agent local database
Would you like to continue? [y/N] y

Cleanup done.
Download and Deploy the Oracle VM Manager Template

It takes approximately 15 to 30 minutes to download and deploy the Oracle VM Manager Template, depending on download
bandwidth, and the performance of the Oracle VM host.
List 7 reviews the five steps to deploy the Oracle VM Manager Template.
1. Download the Oracle VM Manager template named “Oracle VM Manager 2.2.0 for x86 (32 bit) with Oracle Linux 5.4” from
“http://edelivery.oracle.com/linux”. Note: Do not unzip the V21186-01.zip file until Step 2.
2. Copy the zip file named V21186-01.zip to your Oracle VM server's /OVS/running_pool directory.
3. Unzip the V21186-01.zip file. The V21186-01.zip file contains two file, Deploy_Manager_Template.sh and
OVM_EL5U4_X86_OVM_MANAGER_PVM.tgz. Note: Do not untar the OVM_EL5U4_X86_OVM_MANAGER_PVM.tgz file.
4. From dom0 as root type “sh Deploy_Manager_Template.sh”. The Deploy_Manager_Template.sh script is used to check the
prerequisite and gather all the details to configure and create the Oracle VM Manager virtual machine.
5. After the Deploy_Manager_Template.sh script completes and the Oracle VM Manager template is deployed, you will need to wait
about 3 to 5 minutes for the Oracle VM Manager application to start. Once the Oracle VM Manager application is running log in
to the Oracle VM Manager URL that is displayed at the end of the Deploy_Manager_Template.sh script using the admin account
and the password youe selected durring the installation. You can ssh to the Oracle VM Manager Template using the IP address
assigned durring the configuration phase using the root account with the default “ovsroot” password.
Running the Deploy_Manager_Template.sh Script

The next example shows a complete Deploy_Manager_Template.sh script session run as root from dom0. The Oracle VM server in the
example was configed with a shared root storage repository. The Deploy_Manager_Template.sh and
OVM_EL5U4_X86_OVM_MANAGER_PVM.tgz files are in the /OVS/running_pool directory.
Tip: The Deploy_Manager_Template.sh script can be run with options “--skipsmtpcheck” and “--skipmailcheck” to skip the SMTP
server check and email check.
# sh Deploy_Manager_Template.sh --skipsmtpcheck --skipmailcheck

Starting prerequisite checking ...
Checking the Oracle VM Server ...

Done
Checking the xend status ...running
Done
Checking the Oracle VM Agent version ...Done
Checking the Oracle VM Agent status ...
Done
104 of 243 18.04.2011 16:35

Checking the Oracle VM Server free memory: 15623 MB

Done
Checking the available disk space ...
[ R ] 6a16e7f3-36aa-475e-9181-37f65c944487 => ca-fileserver4:/vol/vol1_nfs_demo
*** Storage repositories initialized.
**ca-fileserver4.us.oracle.com:/vol/vol1_nfs_scdemo mounted, set as cluster root.
[ R ] 6a16e7f3-36aa-475e-9181-37f65c944487 => ca-fileserver4:/vol/vol1_nfs_demo
You have 90 GB in /var/ovs/mount/6A16E7F336AA475E918137F65C944487
Checking the running pool directory .../var/ovs/mount/6A16E7F336AA475E918137F65C944487/running_pool
Done
Checking the HA cluster prerequisite ...
Please enter the password for Oracle VM Agent: cluster_precheck...2.3
Done
Checking whether Oracle VM Agent DB is clean ...
Done
Setting up the network for Oracle VM Manager ...

Oracle VM Manager needs to be configured using a static IP address. Follow
the prompts to provide your network settings for Oracle VM Manager.
Press any key to continue...

Available network bridges:
0) xenbr0 Link encap:Ethernet HWaddr 00:1F:29:68:55:FC
inet addr:10.211.0.115 Bcast:0.0.0.0 Mask:255.255.252.0
1) xenbr1 Link encap:Ethernet HWaddr 00:1F:29:68:55:B4
Please choose one of the network bridges(default is xenbr0):
You selected the network bridge: xenbr0
Enter static IP address: 10.211.0.216
Enter netmask: [255.0.0.0] 255.255.252.0
Enter gateway: 10.211.0.1
Enter DNS server: 139.185.51.1
Enter hostname (e.g, host.domain.com): [ca-scdemo-ovm2.2.1-01]

Generating parameter file ...
Note following password setting will be used for SYS and SYSTEM.
Oracle recommends the use of different passwords for each database account.
This can be done after initial configuration.
Specify a password to be used for database accounts:

Please enter the password for account 'OVS':


Please enter the keystore password for Web Service:

Do you want to use HTTPS access for Oracle VM Manager (Y|n)? n
Please enter the password for the default account 'admin':

Set Oracle VM Server Information:

Enter the Server Pool Name: HQ
Enter the Oracle VM Server login user name: root

Enter the Oracle VM Server login password:
Do you want to enable HA(y|N)? y
Please confirm the following information:

Server Pool name : HQ
Oracle VM Server name : ca-scdemo1
Oracle VM Server login user name : root
Enable HA : true
Is the Oracle VM Server information correct(Y|n)? Y

Untar the Oracle VM Manager template tarball ...
105 of 243 18.04.2011 16:35

Done
Move the parameter file to Manager.img
Done
Create Oracle VM Manager virtual machine ...
Done
The Oracle VM Manager virtual machine is booting. To finish the Oracle VM
Manager configuration, connect to the virtual machine console using any
VNC Viewer from a desktop machine via the command:
vncviewer ca-scdemo1:5900
After Oracle VM Manager has been successfully deployed, access the Oracle
VM Manager home page at:
http://10.211.0.216:8888/OVS
The Oracle VM Manager Command Line Interface Introduction

The Oracle VM Manager Command Line Interface is an Oracle VM Manager add on that is available to Oracle Unbreakable Linux
support customers via the Oracle Unbreakable Linux Network. The Oracle VM Manager Command Line Interface offers a
command-line interface for Oracle VM Manager. The Oracle VM Manager Command Line Interface allows Oracle VM Manager
functions to be executed from the command-line. Oracle VM Manager Command Line Interface commands can be scripted allowing
automation of administrative tasks, such as checking status, performing lifecycle management, and executing bulk actions.
List 8 highlights the most common use cases for the Oracle VM Manager Command Line Interface:
Guest, server and server pool lifecycle management:
Power On, Power Off, Clone, Save as Template, Import, Migrate, Pause, Unpause, Suspend, Resume and Delete virtual
machines
Manage virtual machine resources, including ISO files, virtual machine templates and virtual machine images
Manage Oracle VM Manager users, and Oracle VM Manager groups
Create and configure server pools
Manage the Oracle VM Agent
Checking the status of guests, server, and server pools:
Get a list of running guests in server pool

Get a list of active pool servers
Diagnose and troubleshoot issues with guests, servers, and server pools
Get configuration and status information for guests, servers and server pools
Performing bulk operations:
With multiple guests

With multiple servers
With multiple server pools
The Oracle VM Manager Command Line Interface (ovmcli) is a stand-alone application that is written in Python that leverages the
Oracle VM Manager Web Services API to communicate with Oracle VM Manager. The Oracle VM Manager Command Line Interface
can be installed on any Oracle Enterprise Linux or RHEL host with connectivity to your Oracle VM Manager system. The Oracle VM
Manager Command Line Interface can be accessed from a local or remote console.
Figure 6 illustrates the Oracle VM Manager Command Line Interface intra-component communication and firewall requirements.
List 9 highlights the Oracle VM Manager Command Line Interface communication ports and system passwords:
106 of 243 18.04.2011 16:35

Oracle VM Manager:
The default HTTP port of Oracle VM Manager UI is 8888.

The default HTTPS port of Oracle VM Manager UI is 4443
Remote access to the Command Line Interface hosts via ssh on port 22
Guest console access uses ports 5900 through 5999.
Guest console access requires a password without a user name.
The Oracle VM Manager Command Line Interface has one prerequisite, namely the python-ZSI RPM. The ovmcli and python-ZSI
RPMs are hosted at ULN. The ovmcli RPM is located in the el5_i386_oracle_addons and the el5_x86_64_oracle_addons channel. The
python-ZSI RPM is located in the el5_i386_addons and el5_x86_64_addons channel.
Downloading the Oracle VM Manager Command Line Interface from ULN

In this section we will walk through the procedure to download the Oracle VM Manager Command Line Interface. We will start with a
visit to ULN, to search for, and download, the ovmcli and python-ZSI RPMs. After the download section, we will walk through the
installation and configuration of the Oracle VM Manager Command Line Interface on an Oracle Enterprise Linux host. If you have
already downloaded the RPMs from ULN, please skip to the next section.
Note: ULN is password protected. To access ULN you will need a valid Oracle Unbreakable Linux support contract and a CSI number.
Support contracts for Oracle VM can be purchased from your Oracle VM sales representative or directly from the Oracle Store. Once
you have a valid support contract, you can register at the ULN portal or you can register by running the “up2date –register” command
from one of your Oracle VM servers. Please review Oracle VM: How to update an Oracle VM Server for details about registering an
Oracle VM server with ULN.
The Oracle VM Manager Command Line Interface (ovmcli) is hosted at ULN in the el5_i386_oracle_addons and
el5_x86_64_oracle_addons channels. The Oracle VM Manager Command Line Interface prerequisite python-ZSI is hosted in the
el5_i386_addons and el5_x86_64_addons channel.
Table 5 shows the Oracle VM Manager Command Line Interface package names with the associated ULN channel names.
Package ULN Channel
el5_i386_oracle_addons and
ovmcli-1.0-1.el5.noarch.rpm
el5_x86_64_oracle_addons
el5_i386_addons and
python-ZSI-2.1-a1.el5.noarch.rpm
el5_x86_64_addons
1- Point your browser to http://linux.oracle.com as shown in Figure 7. Enter you user name and password and click the Login button
to access ULN.
2- From the ULN home page click on the Channels tab as shown in Figure 8.
107 of 243 18.04.2011 16:35

3- From the Channels page click the i386_oracle_addons channel for the x86 platform or click the el5_x86_64_oracle_addons
channel for the x86-64 platform to download the appropriate ovmcli RPM.
In our example we are looking for the el5_x86_64_oracle_addons channel which is not displayed on the current page. We will need
to access the next page to locate the el5_x86_64_oracle_addons channel. Click the Next button at the bottom of the page as shown
in Figure 9 to display the next page.
4- From the Channels page click the el5_x86_64_oracle_addons channel as shown in Figure 10.
108 of 243 18.04.2011 16:35

5- From the Oracle Software addons for Enterprise Linux 5 (X86-64) page, click the Packages link as shown in Figure 11.
6- From the Oracle Software addons for Enterprise Linux 5 (X86-64) > Packages page, click the ovmcli-1.0-1.el5.noarch link
as shown in Figure 12 to display the ovmcli-1.0-1.el5.noarch RPM Package Details page.
109 of 243 18.04.2011 16:35

7- From the Oracle Software addons for Enterprise Linux 5 (X86-64) > Packages > ovmcli-1.0-1.el5.noarch page, click the ovmcli-
1.0-1.el5.noarch.rpm link as shown in Figure 13 to download the ovmcli-1.0-1.el5.noarch RPM.
Note: The File list link lists the details about all the files for the ovmcli-1.0-1.el5.noarch RPM. The Requires link lists the
requirements for the ovmcli-1.0-1.el5.noarch RPM. The Provides link lists what the ovmcli-1.0-1.el5.noarch RPM provides, i.e.
the ovmcli. The Obsoletes link lists what application(s) the ovmcli-1.0-1.el5.noarch RPM obsoletes. The Conflicts link lists the
conflicts for the ovmcli-1.0-1.el5.noarch RPM.
8- From the Oracle Software addons for Enterprise Linux 5 (X86-64) > Packages > ovmcli-1.0-1.el5.noarch page, click the
Channel link as shown in Figure 14 to return to the Channel page to download the python-ZSI RPM.
9- From the Channels page locate and click the el5_x86_64_addons channel link as shown in Figure 15 to access the Enterprise
Linux 5 Add ons (X86-64) > Packages page.
10- From the Oracle Enterprise Linux 5 Add ons (x86-64) > Packages page click the python-ZSI-2.1-a1.el5.noarch.rpm link as
shown in Figure 16 to display the Enterprise Linux 5 Add ons (X86-64) > Packages > python-ZSI-2.1-a1.el5.noarch.rpm page.
110 of 243 18.04.2011 16:35

11- From the Enterprise Linux 5 Add ons (X86-64) > Packages > python-ZSI-2.1-a1.el5.noarch.rpm page, click the python-
ZSI-2.1-a1.el5.noarch.rpm link as shown in Figure 17 to download the python-ZSI-2.1-a1.el5.noarch.rpm RPM.
Note: The File list link lists the details about all the files for the python-ZSI-2.1-a1.el5.noarch.rpm RPM. The Requires link lists
the requirements for the python-ZSI-2.1-a1.el5.noarch.rpm RPM. The Provides link lists what the python-ZSI-2.1-
a1.el5.noarch.rpm RPM provides, i.e. python-ZSI. The Obsoletes link lists what application(s) the python-ZSI-2.1-
a1.el5.noarch.rpm RPM obsoletes. The Conflicts link lists the conflicts for the python-ZSI-2.1-a1.el5.noarch.rpm RPM.
In the Oracle VM Manager Command Line Interface Download section we visted ULN and downloaded the ovmcli and python-ZSI
RPM. In the next section we will walk through the installation and configuration of the Oracle VM Manager Command Line Interface.
Oracle VM Manager Command Line Interface Installation & Configuration

In the Oracle VM Manager Command Line Interface installation and configuration section we will first walk through the Oracle VM
Manager Command Line Interface installation followed by the Oracle VM Manager Command Line Interface configuration.
The Oracle VM Manager Command Line Interface installation is a quick and simple process that requires two RPMs, a) ovmcli and b)
python-ZSI. As soon as you have downloaded both RPMs and a) placed them on the host or b) mounted a share with the RPMs you can
install the Command Line Interface RPMs by running “rpm –Uvh ovmcli-1.0-1.el5.noarch.rpm python-ZSI-2.1-a1.el5.noarch.rpm” on
the Enterprise Linux host as root.
Table 6 shows a successful Oracle VM Manager Command Line Interface installation.

# rpm -Uvh ovmcli-1.0-1.el5.noarch.rpm python-ZSI-2.1-a1.el5.noarch.rpm
Preparing... ########################################### [100%]
1:python-ZSI ########################################### [ 50%]
2:ovmcli ########################################### [100%]
111 of 243 18.04.2011 16:35

#
Once you have installed the Oracle VM Manager Command Line Interface you can configure the Oracle VM Manager Command Line
Interface by running "ovm config" as root. The Oracle VM Manager Command Line Interface configuration will ask for the following
five pieces of information as shown in List 10.
1. Oracle VM Manager hostname

2. Oracle VM Manager port number (the default is 8888)
3. Deploy path (use the default)
4. Location of vncviewer (required for vncviewer command, e.g. /usr/bin/vncviewer – i.e. run ”which vncviewer” to find the path to
vncviewer)
5. Enable or disable HTTPS support (depends on Oracle VM Manager setup)
The next example shows a complete "ovm config" session.

# ovm config
This is a wizard for configuring the Oracle VM Manager Command Line Interface.
CTRL-C to exit.
Oracle VM Manager hostname: ovm1.sf.seo.com
Oracle VM Manager port number: 8888
Deploy path (leave blank for default):
Location of vncviewer (leave blank to skip): /usr/bin/vncviewer
Enable HTTPS support? (Y/n): n
Configuration complete.
Please rerun the Oracle VM Manager Command Line Interface.
#
You have successfully configured the Oracle VM Manager Command Line Interface. The Oracle VM Manager Command Line Interface
is now ready for use and can be accessed from local or remote console by typing "ovm".
The next section introduces how to use the Oracle VM Manager Command Line Interface starting with a single command, then a bulk
command and concluding with batch script examples.
Oracle VM Manager Command Line Interface Command Examples

This section will start with an introduction to using the Oracle VM Manager Command Line Interface. After the Oracle VM Manager
Command Line Interface introduction we will walk through various example Oracle VM Manager Command Line Interface use cases
starting with single operation command followed by bulk commands and batch scripting examples.
The Oracle VM Manager Command Line Interface can be accessed from the local or remote console from the server hosting the
Oracle VM Manager Command Line Interface. Commands can be executed directly from the command line or from within the Oracle
VM Manager Command Line Interface shell.
The Oracle VM Manager Command Line Interface shell can be accessed by typing “ovm –u username –p your_password shell” as
shown in the following example:
# ovm -u username -p password shell
ovm>
You would substitute username with a valid Oracle VM Manager user name and substitute password with a valid password to
authenticate into the shell.
Once you have successfully authenticated in to the Oracle VM Manager Command Line Interface shell you will be presented with the
“ovm>” prompt. From the “ovm>” prompt you can enter the desired subcommands. From the Oracle VM Manager Command Line
Interface shell, type “ovm help”, to display an abbreviated list of the help messages, as shown below.
ovm> help
Usage: subcommand [suboptions]
Oracle VM Manager Command Line Interface
Subcommands:
group --- Group management

img --- Virtual machine image management
iso --- ISO image management
sd --- Shared virtual disk management
svr --- Server management
svrp --- Server pool management
tmpl --- Virtual machine template management
user --- User management
val --- System parameter management
vm --- Virtual machine management
config --- CLI configuration
connect --- Connect to Oracle VM Manager from CLI shell
exit --- Exit
help --- Show help
112 of 243 18.04.2011 16:35

shell --- Launch an interactive shell

use --- Specify a server pool to use
vncviewer --- Start a VNC console
"help <subcommand>" displays help message for that subcommand.

"help all" displays complete list of subcommands.
ovm>
Tip: Help messages for subcommands, i.e. "help <subcommand>" will display the help message for that subcommand. The "help all"
command displays a complete list of all the subcommands.
Subcommands can also be run directly from the command line outside of the Oracle VM Manager Command Line Interface shell. The
next example shows how to display the help message from the command line, not from the Oracle VM Manager Command Line
Interface shell.
# ovm -u username -p password help
Usage: ovm [options] subcommand [suboptions]
Oracle VM Manager Command Line Interface
Subcommands:
group --- Group management

img --- Virtual machine image management
iso --- ISO image management
sd --- Shared virtual disk management
svr --- Server management
svrp --- Server pool management
tmpl --- Virtual machine template management
user --- User management
val --- System parameter management
vm --- Virtual machine management
config --- CLI configuration
connect --- Connect to Oracle VM Manager from CLI shell
exit --- Exit
help --- Show help
shell --- Launch an interactive shell
use --- Specify a server pool to use
vncviewer --- Start a VNC console
#
To recap, subcommands can be entered from the Oracle VM Manager Command Line Interface shell or directly from the server’s
console. Table 3 shows the same “help” command run from the Oracle VM Manager Command Line Interface shell and from the
Oracle VM Manager Command Line Interface server’s console.
Command Line Interface shell Server console
# ovm -u username -p password shell # ovm -u username -p password help

ovm> help
Bulk Commands and Batch Scripting

In this section, we will start with an introduction to Oracle VM Manager Command Line Interface bulk commands and batch scripting.
After the introduction, we will walk through two Oracle VM Manager Command Line Interface use case examples. The first use case
will show how to suspend and then resume four guests, to accommodate a backup schedule. The last example will show how to add an
Oracle VM server to an existing server pool.
Note: We will be adding new example Oracle VM Manager Command Line Interface use cases to this chapter on a regular basis. Stay
tuned!
Bulk commands can be run from the command line or via batch scripts. For example, if you wanted to suspend multiple guests with a
single command i.e. “vm suspend”, this could be accomplished by typing multiple “vm suspend” commands separated by a semi-colon.
Before we execute our fist “vm suspend” bulk command, lets check the status of our virtual machines using the “vm ls” command. The
“vm ls” command can be entered from the Oracle VM Manager Command Line Interface shell or directly from the console.
The next example shows the “vm ls” command entered at the server console which will list the status of all of the virtual machines in
your Oracle VM server farm.
# ovm -u admin -p password vm ls
Name Size(MB) Mem VCPUs Status Server_Pool
node4 13313 256 1 Running SF-HQ
113 of 243 18.04.2011 16:35


The same command can be executed from the Oracle VM Manager Command Line Interface shell as shown in the next example.
# ovm -u admin -p password shell
Type "help" for a list of commands.
ovm> vm ls
ovm>
Now that we have checked that status of the guests in our Oracle VM farm we will suspend four “Running” guests named node1,
node2, node3, and node4 with a single bulk command. Enter the bulk command from the command line, not the Oracle VM Manager
Command Line Interface shell to suspend all four guests as shown in the next example.
# ovm -u admin -p password vm suspend -s serverpool_name -n node1; ovm -u admin -p password vm suspend -s serverpool_name -n
node2; ovm -u admin -p password vm suspend -s serverpool_name -n node3; ovm -u admin -p password vm suspend -s
serverpool_name -n node4
Suspending.
Suspending.
Suspending.
Suspending.
#
The above example shows four separate “vm suspend” commands separated by a semi-colon. You would replace the password,
serverpool_name and hostnames with yours to use this bulk command example in your environment.
Tip: The help message for “vm suspend” can be displayed by typing “ovm -u admin -p password help vm suspend” or from the shell by
typing “help vm suspend”.
Let’s validate that the status of the four virtual machines has changed, from “Running” to “Suspended”, by typing “ovm -u admin -p
password vm ls” from the server console, as shown in the next example.
node4 13313 256 1 Suspended SF-HQ
The above “vm ls” example validated that our “vm suspend” bulk command was successfully completed.
Next we will review how to create a vmsuspend batch script. If you would like to test the vmsuspend batch script, you will need to
resume your suspended guest. We will cover how to resume your guests in great detail in a couple of paragraphs. If you need to
resume your guests to test our next example vmsuspend batch script, please run the following bulk command from the server console.
# ovm -u admin -p password vm resume -s serverpool_name -n node1; ovm -u admin -p password vm resume -s serverpool_name -n
node2; ovm -u admin -p password vm resume -s serverpool_name -n node3; ovm -u admin -p password vm resume -s
Resuming.
Resuming.
Resuming.
Resuming.
#
The same “vm suspend” bulk command could be run as a batch script. The commands in an Oracle VM Manager Command Line
Interface batch script are executed in sequence, identical to how sqlplus runs a sql script. There is no error handling with Oracle VM
Manager Command Line Interface batch scripts which means when the previous command finishes, regardless of success or failure,
the next command will run.
The first step to create an Oracle VM Manager Command Line Interface batch script is to create a file. Let’s call the example file
vmsuspend. Save the example commands shown below in the vmsuspend file.
vm suspend -s serverpool_name -n node1;

Now that we have created the file vmsuspend, we can run the vmsuspend file as a batch script in one of two ways; this assumes that
the vmsuspend file is located in the current working directory, i.e. /home/user/vmsuspend. You can run the batch script from the
server console or from the Oracle VM Manager Command Line Interface shell.
The next example shows how to run our example vmsuspend batch script from the server console.
# ovm shell -s vmsuspend
Login: admin
114 of 243 18.04.2011 16:35

Password:
Suspending.
Suspending.
Suspending.
Suspending.
#
The above example runs our batch script from the servers console using the “-s” or script option. As shown in the example, it is
necessary to authenticate into the shell to run the vmsuspend batch script. Once the script completes you are dropped back to the
server console.
Our other option to run the vmsuspend batch script is from the Oracle VM Manager Command Line Interface shell. The next example
shows how to run the vmsuspend batch script from the Oracle VM Manager Command Line Interface shell. The example assumes that
the vmsuspend file is located in the current working directory, i.e. /home/user/vmsuspend.
ovm> @vmsuspend
Suspending.
Suspending.
Suspending.
Suspending.
ovm>
The above example runs our batch script from the Oracle VM Manager Command Line Interface shell. Once the script completes, you
are dropped back to the Oracle VM Manager Command Line Interface shell prompt.
Let’s validate the status of the four virtual machines by typing “vm ls” from the Oracle VM Manager Command Line Interface shell as
shown on the next example.
# ovm> vm ls
Typing “vm ls” from the Oracle VM Manager Command Line Interface shell validated that our previous vmsuspend batch script did
successfully run and suspend all four of the guests. You can also list the status of your virtual machines from the server console, as
shown in the next example.
Now that we have reviewed running a “vm suspend” bulk command and a vmsuspend batch script along with “vm ls”, let’s examine
how to resume our guests. First we will review a “vm resume” bulk command, followed with a vmresume batch script.
Next, we will resume four “Suspended” guests named node1, node2, node3, and node4 with a single bulk command. Enter the bulk
command from the command line, not the Oracle VM Manager Command Line Interface shell, to resume all four guests as shown in
the next example.
# ovm -u admin -p password vm resume -s serverpool_name -n node1; ovm -u admin -p password vm resume -s serverpool_name -n
node2; ovm -u admin -p password vm resume -s serverpool_name -n node3; ovm -u admin -p password vm resume -s
Resuming.
Resuming.
Resuming.
Resuming.
#
You can validate the status of the four virtual machines by typing “ovm -u admin -p password vm ls” from the server console or “vm ls”
from the Oracle VM Manager Command Line Interface shell.
The same “vm resume” bulk command could be run as a batch script. As discussed in the “vm suspend” section the commands in an
Oracle VM Manager Command Line Interface batch script are executed in sequence, identical to how sqlplus runs a sql script. There
is no error handling with Oracle VM Manager Command Line Interface batch scripts which means when the previous command
finishes, regardless of success or fail, the next command will run.
The first step to create our example “vm resume” Oracle VM Manager Command Line Interface batch script, is to create a file. Let’s
call the example file vmresume. Save the example commands shown below in the vmsuspend file.
vm resume -s serverpool_name -n node1;

115 of 243 18.04.2011 16:35

Now that we have created the file vmresume, we can run the vmreseume file as a batch script in one of two ways; this assumes that
the vmreseume file is located in the current working directory, i.e. /home/user/vmresume. You can run the batch script from the server
console or from the Oracle VM Manager Command Line Interface shell.
The next example shows how to run our example vmresume batch script from the server console.
# ovm shell -s vmresume
Login: admin
Password:
Resuming.
Resuming.
Resuming.
Resuming.
#
The above example runs our batch script from the servers console using the “-s” or script option. As shown in the example it is
necessary to authenticate into the shell to run the vmsuspend batch script. Once the script completes, you are dropped back to the
server console.
Our other option to run the vmresume batch script is from the Oracle VM Manager Command Line Interface shell. The next example
shows how to run the vmresume batch script from the Oracle VM Manager Command Line Interface shell. The example assumes that
the vmresume file is located in the current working directory, i.e. /home/user/vmresume.
ovm> @vmresume
Resuming.
Resuming.
Resuming.
Resuming.
ovm>
The above example runs our batch script from the Oracle VM Manager Command Line Interface shell. Once the script completes, you
are dropped back to the Oracle VM Manager Command Line Interface shell prompt.
You can validate the status of the four virtual machines by typing “ovm -u admin -p password vm ls” from the console or “vm ls” from
the Oracle VM Manager Command Line Interface shell.
We have just completed our review of the “vm suspend” and “vm resume” bulk command and batch scripting. The next example
expands from our previous “vm suspend” and “vm resume” with a detailed use case called Guest Backup. The Guest Backup use case
will use a batch script that suspends four guests, pauses for 120 seconds and then resumes the guests to accommodate an Oracle VM
repository scheduled backup job.
Guest Backup
Our first example use case “Guest Backup”, builds upon what we learned in the previous section with regards to the vm suspend and
vm resume examples. The use case shows us how to create a batch script to suspend four guests, pause for 120 seconds and then
resume the guests to accommodate a scheduled snapshots/cloning of an Oracle VM extended repository.
There are numerous strategies to back up Oracle VM guests. For example, you can use the same agent based backup solution you
have always used for your physical machines or you could leverage the snapshots/cloning functionality from your storage solution.
Our goal, with the Guest Backup use case, is to provide an Oracle VM Manager Command Line Interface solution to work with the
snapshots/cloning functionality from your existing storage.
To capture a clean snapshot or backup of a running guest, the guest should be in the suspended state when the snapshot or backup
job is executed. When a guest is suspended, the status of the guest operating system is written to disk and removed from system
memory. Conversely, when a guest operating system is paused, the system state continues to reside in memory. For the Guest Backup
use case we will use the “vm suspend” and “vm resume” commands to ensure that the status of the guest operating system is written
to disk and removed from system memory for the duration of the snapshot or backup job.
Prerequisites:
A storage solution that supports snapshots or cloning.

Our example batch script named vmbackup will be scheduled to run each night at 3:00 AM as a cron job.
The vmbackup job will run 2 minutes before the scheduled snapshot or clone job.
The target guests are in the Running state.
List 11 shows the work flow to execute the Guest Backup use case.
1. Create the vmbackup batch script

2. Test the vmbackup batch script
3. Automate the vmbackup batch script with cron.
1- The first step is to create an Oracle VM Manager Command Line Interface batch script on the machine hosting the Oracle VM
Manager Command Line Interface. Let’s create a file named vmbackup. Save the example commands shown below in the vmbackup
116 of 243 18.04.2011 16:35

file.

sleep 120;
2- Next we will test the example vmbackup batch script from the server console, as shown in the example.
# ovm -u admin -p password shell -s vmbackup
Suspending.
Suspending.
Suspending.
Suspending.
Sleeping for 120 seconds
Resuming.
Resuming.
Resuming.
Resuming.
#
3- The final step is to automate the vmbackup batch script to run at 3:00 AM every day with cron, as shown in the example.
# crontab –e
* 3 * * * ovm -u admin -p password shell -s vmbackup
:wq!
#
We have just completed our first use case that showed us how to create a batch script to suspend four guests, pause for 120 seconds,
and then resume the guests to accommodate a scheduled snapshot or backup job of an Oracle VM extended repository.
Create a Server Pool

The next example batch script shows how to create a server pool.
vm ls;
svrp ls;
svrp new -H ovstest19.cn.test.com --serverpool_name=ovstest19 -a -A oracle -U root -P password -L BJ -D "ovstest19 serverpool";
use -s ovstest19;
svrp info;
sleep 10;
svrp status;
svr ls;
Programming with the Oracle VM Manager Command Line Interface

This section will provide an example of a more advanced method of programming with the Oracle VM Manager Command Line
Interface commands.
You can program Oracle VM Manager Command Line Interface commands to determine whether a command is successfully executed
or not. You’re also able to get the return result of commands in a programming friendly way.
The following example script will a) Create a serverpool b) Register all the discoverable ISOs c) Create a hvm guest by one of the ISOs
and d) Check the status of the guest created, if the status is "Running", then power off the guest.
from ovmcli.Ovm import Ovm
from ovmcli.errorcode import *
import time
ovm = Ovm(username='admin', password='oracle')

_ = ovm.runcmd
serverpool_name = "ovstest19"
vm_name = "hvmbyiso1"
server_ip = "ovstest19.cn.oracle.com"
# Delete a serverpool
if OVM_OK == _("svrp del -s %s --force" % serverpool_name):
print "OK serverpool_del"
# Create a serverpool
117 of 243 18.04.2011 16:35

cmd = 'svrp new -H %s --serverpool_name=%s -a -A oracle -U root -P oracle -L BJ ' % (server_ip, serverpool_name)
if OVM_OK == _(cmd):
print "OK serverpool_create"
# Discover and Register ISOs

discoverable_isos = _("iso disc -s %s" % serverpool_name)
for iso in discoverable_isos:
cmd = "iso reg -s %s -g el51 -l %s" % (serverpool_name, iso)
print "OK iso reg %s" % iso
isos = _("iso ls -s %s" % serverpool_name)
iso = isos[0][1]
# Create a VM
cmd = 'vm new --method=iso -x -s %s -n %s -c 1 -i %s -o Other -y 512 -d 1024' % (serverpool_name, vm_name, iso)
print "OK vm new"
while 1:
cmd = "vm status -s %s -n %s" % (serverpool_name, vm_name)
status = _(cmd)
if type(status) == type([]):
vm_status = status[0]
if vm_status != 'Running':
print "vm_status:" + vm_status
print "sleep 10s"
time.sleep(10)
else:
print vm_status
cmd = 'vm poweroff -s %s -n %s' % (serverpool_name, vm_name)
print "OK poweroff"
break

In this chapter we will review the steps to upgrade all of the Oracle VM components from version 2.1.5 to version 2.2. The chapter
starts with an introduction to the upgrade process for Oracle VM Manager and Oracle VM server. Next we review the upgrade options
and upgrade prerequisites for both Oracle VM Manager and Oracle VM server. Next we will review the Unbreakable Linux Network to
support the up2date upgrade option. The chapter concludes with a review of the steps to upgrade Oracle VM Manager and Oracle VM
server from version 2.1.5 to version 2.2.
Table of Contents
Oracle VM 2.1.5 to 2.2 Upgrade Processes
Oracle VM Manager Upgrade Process
Oracle VM Server Upgrade Options
The Unbreakable Linux Network
Backup the OVS Repository
Download the Oracle VM Manager Media
Unzip and Mount the Oracle VM 2.2 ISO
Upgrade Oracle VM Manager
Oracle VM Server Upgrade using up2date
...Oracle VM Server Upgrade Prerequisites
...Oracle VM Server Upgrade Commands
...Reboot the Oracle VM Pool Master Server
...Reboot the None Master Oracle VM servers
...Enter the Server Pool Virtual IP using Oracle VM Manager
Oracle VM 2.1.5 to 2.2 Upgrade Processes

The upgrade process for Oracle VM consists of first upgrading Oracle VM Manager followed by the Oracle VM servers. Oracle VM
118 of 243 18.04.2011 16:35

Manager is upgraded by using the Oracle VM Manager 2.2 ISO file which is available from the eDelivery portal. Oracle VM servers
can be upgraded using either a) the bootable Oracle VM Server 2.2 ISO image or b) with up2date and the Unbreakable Linux Network
or c) using up2date or yum with a local yum repository.
Note: Please note that a rolling upgrade of the Oracle VM servers from version 2.1 to version 2.2 is not supported. You will need to
backup and then power off all your virtual machines before you upgrade the Oracle VM servers. A rolling upgrade is not supported,
due in part, to an upgrade of the OCFS2 file system.
List 1 shows the correct order to backup and upgrade all of the Oracle VM components from version 2.1.5 to version 2.2.
1. Back up the Oracle VM Manager database

2. Backup the OVS repository
3. Upgrade Oracle VM Manager from version 2.1.5 to version 2.2 using the Oracle VM 2.2.0 ISO
4. Upgrade the none master Oracle VM servers, i.e. the virtual machine and utility server pool members
5. Upgrade the Oracle VM server master agent server
6. Reboot the master agent server
7. Reboot the none master Oracle VM servers
8. If your server pools are HA enabled, access Oracle VM Manager and enter the server pool virtual VIP
Oracle VM Manager Upgrade Process

Oracle VM Manager is updated by downloading the media from Oracle’s eDelivery portal and executing an installation script. Oracle
VM media is freely downloadable from Oracle’s eDelivery portal.
The Oracle VM Manager update process will require you to select an installation option of either install, uninstall, or upgrade, as well
as entering the passwords for the existing Oracle VM Manager OVS database account and the oc4j admin account. The Oracle VM
Manager OVS database account and the oc4j admin account passwords are selected during the Oracle VM Manager installation
process. The account passwords for the Oracle VM Manager OVS database account and the oc4jadmin account can be managed from
their respected web portals.
The OVS account can be maintained from the Oracle Database Express Edition portal which is a part of the Oracle VM Manager
install. The Oracle Database Express Edition portal can be accessed from the Oracle VM manager server locally by entering
http://127.0.0.1:8080/apex or remotely by entering the ip address or the FQDN followed by :8080/apex, i.e.
http://OracleVM_Manager_fqdn:8080/apex in a web browser.
Figure 1 shows the Oracle Database Express Edition portal login page.
The Application Server Control portal is also part of the Oracle VM Manager install. The oc4j admin account can be maintained from
the Application Server Control portal. The Application Server Control portal can only be accessed locally from the Oracle VM server
console by entering http://127.0.0.1:8888/em in a local web browser.
Figure 2 shows the Application Server Control login page.
Oracle VM Server Upgrade Options

There are two options to ugrade Oracle VM servers. The first option is to use the up2date command from the Oracle VM server’s
dom0 console. Using the up2date command requires Unbreakable Linux Network access. The second option is to upgrade the system
to the latest version, using a bootable Oracle VM server ISO image. Oracle VM server ISO is freely downloadable from Oracle’s
eDelivery portal.
Figure 3 shows the Oracle VM server upgrade options.
119 of 243 18.04.2011 16:35

The difference between the two Oracle VM server upgrade options is a) the ability to perform incremental updates or b) platform
upgrades. Using the up2date command with the Unbreakable Linux Network allows Oracle VM servers to receive incremental
updates, patches, security fixes as well as upgrades. Using the bootable media requires the machine to be restarted and to select the
update option from the boot prompt. Selecting the update option from the boot prompt will update the Oracle VM server to the
version of the bootable media, e.g. from 2.1.5 to 2.2.
Figure 4 shows the System to Upgrade boot prompt. From the System to Upgrade boot prompt administrators can select to
Reinstall System or Oracle VM server 2.x to upgrade the system.
When the up2date command is executed from an Oracle VM server’s dom0 console, up2date connects to the ULN repository and
downloads the requested packages in RPM format. The up2date command communicates over the internet to “linux-
update.oracle.com” on port 443. Up2date then installs the packages on the registered Oracle VM server.
Please consult up2date’s man page by executing “man up2date” from the Oracle VM server’s dom0 console as root for a
comprehensive list of command augments.
The Unbreakable Linux Network

The Oracle Unbreakable Linux support program allows customers to open Oracle VM and Linux Service Requests (SRs) 24/7 as well
as access to the Unbreakable Linux Network. Existing Oracle support customers use the same support infrastructure i.e. support
phone number and web portal to log and track Oracle VM Service Requests (SR) as with Tech and App Service Requests. The
Unbreakable Linux Network is an online resource for Oracle Unbreakable Linux subscribers that offers access to Linux and Oracle
VM software patches, updates and fixes. The Unbreakable Linux Network can be accessed from registered Linux and Oracle VM hosts
with the up2date program as well as with a web browser, by pointing the browser to http://linux.oracle.com.
The Unbreakable Linux Network and My Oracle Support (formally Metalink) are two separate systems, accessed by different URLs
and user name and passwords. ULN is used to access Oracle VM and Linux patches, updates, and fixes, and My Oracle Support is
used to manage SRs.
Figure 5 shows the ULN portal.
120 of 243 18.04.2011 16:35

This section will review the steps to upgrade Oracle VM Manager. We start with a review of the Oracle VM Manager database backup
and restore procedure followed by the 2.1.5 to 2.2 Oracle VM Manager upgrade.

Our first step to upgrade Orace VM Manager is to backup our Oracle VM Manager database using the backup.sh script. Before you
backup the Oracle VM Manager database, access Oracle VM Manager and ensure that the status of all of the Oracle VM servers is
"Running". You will need the following details for the backup procedure a) the password for database account 'OVS', which was
assigned during the Oracle VM Manager install, b) the path for dump file, i.e. /tmp/ovs.dmp and c) the path for log file, i.e.
/tmp/ovs.log. We will conclude Step 1 with a complete Oracle VM Manager database restore.
Next to backup the Oracle VM Manager database select the Backup Oracle VM Manager option (number 1) from the backup
Completed backup:

1
The next example reviews a complete Oracle VM Manager database backup. Access the Oracle VM Manager server console as root
and type "sh /opt/ovs-manager-2.1/bin/backup.sh" to start the Oracle VM Manager database backup as shown in the following
example.
Completed backup:

1
Back up data now ...

Connected to:
SQL>
121 of 243 18.04.2011 16:35

Export: Release 10.2.0.1.0 - Production on Sat Oct 17 10:00:52 2009
Export done in US7ASCII character set and AL16UTF16 NCHAR character set
server uses AL32UTF8 character set (possible charset conversion)
. exporting pre-schema procedural objects and actions
. exporting foreign function library names for user OVS
. exporting PUBLIC type synonyms
. exporting private type synonyms
. exporting object type definitions for user OVS
About to export OVS's objects ...
. exporting database links
. exporting sequence numbers
. exporting cluster definitions
. about to export OVS's tables via Conventional Path ...
. . exporting table OVS_AGENT 2 rows exported
. . exporting table OVS_ALERT 11 rows exported
. . exporting table OVS_CATEGORY 1 rows exported
. . exporting table OVS_CDROM 5 rows exported
. . exporting table OVS_CDROM_RESOURCE 0 rows exported
. . exporting table OVS_GROUP 4 rows exported
. . exporting table OVS_HD_TEMP
. . exporting table OVS_IMG_OS 0 rows exported
. . exporting table OVS_IMG_TEMP
. . exporting table OVS_LOCK 19 rows exported
. . exporting table OVS_MAP 35 rows exported
. . exporting table OVS_NIC_TEMP
. . exporting table OVS_OS_RESOURCE 17 rows exported
. . exporting table OVS_PARTNER 0 rows exported
. . exporting table OVS_PREFERRED_SERVER 0 rows exported
. . exporting table OVS_PRIVILEGE 4 rows exported
. . exporting table OVS_PRIVILEGE_ROLE 5 rows exported
. . exporting table OVS_RESOURCE 4 rows exported
. . exporting table OVS_ROLE 3 rows exported
122 of 243 18.04.2011 16:35


. . exporting table OVS_SERVER 2 rows exported
. . exporting table OVS_SITE 1 rows exported
. . exporting table OVS_STATISTIC 16593 rows exported
. . exporting table OVS_SYS_VALUE 8 rows exported
. . exporting table OVS_TASK 0 rows exported
. . exporting table OVS_USER 19 rows exported
. . exporting table OVS_USER_GROUP 75 rows exported
. . exporting table OVS_USER_ROLE 19 rows exported
. . exporting table OVS_USER_SITE 19 rows exported
. . exporting table OVS_VD_IMG 39 rows exported
. . exporting table OVS_VIRTUAL_DISK 73 rows exported
. . exporting table OVS_VM_GEN_INFO 25 rows exported
. . exporting table OVS_VM_IMG 25 rows exported
. . exporting table OVS_VM_NETWORK 25 rows exported
. . exporting table OVS_VM_SNAPSHOT 0 rows exported
. . exporting table OVS_WORKSPACE_TEMP
. exporting synonyms
. exporting views
. exporting stored procedures
. exporting operators
. exporting referential integrity constraints
. exporting triggers
. exporting indextypes
. exporting bitmap, functional and extensible indexes
. exporting posttables actions
. exporting materialized views
. exporting snapshot logs
. exporting job queues
. exporting refresh groups and children
. exporting dimensions
. exporting post-schema procedural objects and actions
. exporting statistics
123 of 243 18.04.2011 16:35

Connected to:
SQL>
Done
#
We just executed a successful Oracle VM Manager database backup.

In the event that you need to restore the Oracle VM Manager database walk through the same procedure as in the above example and
select 2 to restore Oracle VM Manager database. You will need the following details for the restore procedure a) the password for
database account 'OVS', which was assigned during the Oracle VM Manager install, b) the path for dump file, i.e. /tmp/ovs.dmp and c)
the path for log file, i.e. /tmp/ovs.log.
Next, to backup the Oracle VM Manager database, select the Restore Oracle VM Manager option (number 2) from the backup
Completed backup:

2
The next example reviews a complete Oracle VM Manager database restore. Access the Oracle VM Manager server console as root
and type "sh /opt/ovs-manager-2.1/bin/backup.sh" to start the Oracle VM Manager database restore as shown in the following
example.
The next example reviews an Oracle VM Manager database restore procedure. Access the Oracle VM Manager server console as root
and type "sh /opt/ovs-manager-2.1/bin/backup.sh" as shown in the following example.

2
Restore data now ...

Please enter the password for database account 'SYS':
Import: Release 10.2.0.1.0 - Production on Sun Oct 18 11:15:24 2009
Export file created by EXPORT:V10.02.01 via conventional path
Warning: the objects were exported by OVS, not by you
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses AL32UTF8 character set (possible charset conversion)
. . importing table "OVS_AGENT" 2 rows imported
. . importing table "OVS_ALERT" 11 rows imported
. . importing table "OVS_CATEGORY" 1 rows imported
. . importing table "OVS_CDROM" 5 rows imported
. . importing table "OVS_CDROM_RESOURCE" 0 rows imported
124 of 243 18.04.2011 16:35

. . importing table "OVS_GROUP" 4 rows imported

. . importing table "OVS_IMG_OS" 0 rows imported
. . importing table "OVS_LOCK" 19 rows imported
. . importing table "OVS_MAP" 35 rows imported
. . importing table "OVS_OS_RESOURCE" 17 rows imported
. . importing table "OVS_PARTNER" 0 rows imported
. . importing table "OVS_PREFERRED_SERVER" 0 rows imported
. . importing table "OVS_PRIVILEGE" 4 rows imported
. . importing table "OVS_PRIVILEGE_ROLE" 5 rows imported
. . importing table "OVS_RESOURCE" 4 rows imported
. . importing table "OVS_ROLE" 3 rows imported
. . importing table "OVS_SERVER" 2 rows imported
. . importing table "OVS_SITE" 1 rows imported
. . importing table "OVS_STATISTIC" 16593 rows imported
. . importing table "OVS_SYS_VALUE" 8 rows imported
. . importing table "OVS_TASK" 0 rows imported
. . importing table "OVS_USER" 19 rows imported
. . importing table "OVS_USER_GROUP" 75 rows imported
. . importing table "OVS_USER_ROLE" 19 rows imported
. . importing table "OVS_USER_SITE" 19 rows imported
. . importing table "OVS_VD_IMG" 39 rows imported
. . importing table "OVS_VIRTUAL_DISK" 73 rows imported
. . importing table "OVS_VM_GEN_INFO" 25 rows imported
. . importing table "OVS_VM_IMG" 25 rows imported
. . importing table "OVS_VM_NETWORK" 25 rows imported
. . importing table "OVS_VM_SNAPSHOT" 0 rows imported
Import terminated successfully without warnings.
Done
#
We have just completed an Oracle VM Manager backup and restore.
Backup the OVS Repository

Next we will back up the OVS repository using your existing storage backup tools.
Download the Oracle VM Manager Media

Point you browser to http://edelivery.oracle.com/linux to download the Oracle VM Manager ISO files as shown in Figure 6. Click the
Continue button to proceed.
From the Registration page enter the following required information then click the Continue button to proceed:
Full Name (FIRST LAST)
Company Name
Email address
Select your country from the drop down list
Select the check box to agree to the Agreement Terms
Select the check box to accept the Export Restrictions
Figure 7 shows the Registration Page.
125 of 243 18.04.2011 16:35

From the Media Pack Search page select Oracle VM from the Select a Product Pack drop down box, then select the desired
platform either x86 or x86-64 from the Platform drop down box. Click the Go button to proceed.
Figure 8 shows the Media Pack Search page.
From the Media Pack Search results page select the Oracle VM 2.2 Media Pack radio button. Click the Oracle VM 2.2 Media
Pack link to proceed.
Figure 9 shows the Media Pack Search results page.
126 of 243 18.04.2011 16:35

From the Oracle VM 2.2 Media Pack page click the Oracle VM Manager 2.2 Download button to download the Oracle VM
Manager 2.2 media.
Figure 10 shows the Oracle VM 2.2 Media Pack page.
We have just completed the steps to download the Oracle VM 2.2 Media Pack from the Oracle eDelivery portal.
Unzip and Mount the Oracle VM 2.2 ISO

Now that you have the Oracle VM Manager 2.2 media, copy it over to your Oracle VM Manager server. From the Oracle VM Manager
server console as root, unzip the V18419-01.zip archive as shown in the next example.
#unzip V18419-01.zip
Archive: V18419-01.zip
inflating: OracleVM-Manager-2.2.0.iso, /*.sqljsp, /*.SQLJSP)
#
The following example shows how to mount the Oracle VM Manager ISO image from the Oracle VM Manager server console as root
user.
#mount -o ro,loop OracleVM-Manager-2.1.5.iso /mnt/
Upgrade Oracle VM Manager

The following section will start with a review of each of the steps of a successful Oracle VM Manager upgrade. The section will
conclude with a complete successful Oracle VM Manager upgrade session.
Before you proceed with the Oracle VM Manager 2.2 upgrade you will need the OVS password and the oc4jadmin password, which
127 of 243 18.04.2011 16:35

where both selected durring the Oracle VM Manager install.

The update process requires you to select an upgrade option from the install prompt, as well as enter the passwords for the Oracle
VM Manager OVS database account, and the Oracle VM Manager oc4jadmin account.
Access the Oracle VM server console as root and cd in to the directory where you mounted the ISO file. The following example shows
how to execute the runInstaller.sh from the Oracle VM Manager server console as root user.
Select the Upgrade Oracle VM Manager option (number 3) from the install prompt. The next example shows the installation
selection prompt.

3
Starting Oracle VM Manager 2.2 upgrade ...

Next, you are asked if you would like to backup the Oracle VM Manager database. If you select “N” from the install prompt, the
upgrade will be aborted. The following example shows the Oracle VM Manager database backup prompt.
Backup the database before upgrade is highly recommended, to backup the database now, choose 'N' and run:
/opt/ovs-manager-2.1/bin/backup.sh
Next you are asked if you would like to proceed or cancel the upgrade. You can enter y to proceed or N to cancel the upgrade. The
following example shows the upgrade acceptance prompt.
Are you sure you want to upgrade Oracle VM Manager from version 2.1.5 to 2.2 ? [y|N]
Next you must enter the OVS password. The following example shows the OVS installation password prompt.
Next enter the oc4jadmin password. The following example shows the oc4jadmin installation password prompt.
A successful upgrade will exit with the following informational warnings.

Done.
Please check the log /var/log/ovm-manager/upgrade_2.1.5_2.2.log for more information.

http://ovm03.sf.seo.com:8888/OVS

http://ovm03.sf.seo.com:8888/help/help
The following example show a complete Oracle VM Manager 2.2 upgrade session.

3
Starting Oracle VM Manager 2.2 upgrade ...
Backup the database before upgrade is highly recommended, to backup the database now, choose 'N' and run:
/opt/ovs-manager-2.1/bin/backup.sh
Are you sure you want to upgrade Oracle VM Manager from version 2.1.5 to 2.2 ? [y|N] y
Now upgrading OC4J ...

Starting OC4J ... Done.

Done.
128 of 243 18.04.2011 16:35


Please check the log /var/log/ovm-manager/upgrade_2.1.5_2.2.log for more information.

http://ovm03.sf.seo.com:8888/OVS

http://ovm03.sf.seo.com:8888/help/help
#
Oracle VM Server Upgrade using up2date
In this section we will review how to update an Oracle VM Server using up2date with the Unbreakable Linux Network. We will
conclude with a review on how to upgrade Oracle VM servers using up2date with the Unbreakable Linux Network.
It is essential to upgrade and boot the Oracle VM servers in the following order; first upgrade the Oracle VM virtual machine and
utlilty servers, followed by the Oracle VM pool master server. After all of the servers are servers upgraded, boot the Oracle VM pool
master server followed by the none pool master servers.
Note: You can determine the Oracle VM server roles i.e. virtual machine, utlilty and master servers by accessing Oracle VM Manager
and displaying the Servers tab as shown in Figure 11.
Oracle VM Server Upgrade Prerequisites

Before we upgrade any of the Oracle VM servers ensure that all of the virtual machines are off and verify the /etc/hosts and
/etc/resolv.conf file setting as explained in List 1.
List 1
1- Verify that the hostname in /etc/hosts is associated with the public IP address, not 127.0.0.1.
2- Verify that all of the Oracle VM servers have the proper entries in /etc/resolv.conf or if DNS is not used, make sure the correct
setting are in /etc/hosts. Please note that all the servers in the same pool must have the consistent name resolution, either by DNS or
by file (/etc/hosts).
The upgrade procees for all of the Oracle VM 2.1.5 pool memeber servers i.e. the virtual machine servers, the utlilty servers and the
pool master pool server is idential. We will first upgrade all of the none pool master servers, followed by the pool master pool server,
using the following commands.
Note: Before we upgrade any of the Oracle VM servers, ensure that all of the virtaul machines are powered off and backed up.
Oracle VM Server Upgrade Commands

First, from dom0 as root install the ovm22upgrade package as shown in the example.
# up2date ovm22upgrade
Once the ovm22upgrade package successfully completes, run the ovm22upgrade.py script as shown in the next example.
# /usr/local/sbin/ovm22upgrade.py
To complete the Oracle VM server upgrade run the following command, as shown in the next example.
# up2date -fu
Once you have upgraded all of the none pool master servers, upgrade the pool master pool server. Next we need to boot the pool
master pool server followed by the none pool master servers.
The next example shows each of the three steps from a sucessful Oracle VM server upgrade.
First install ovm22upgrade package as shown in the example.

# up2date ovm22upgrade
Fetching Obsoletes list for channel: ovm2_i386_latest...

########################################
Fetching rpm headers...

########################################
129 of 243 18.04.2011 16:35

Name Version Rel

----------------------------------------------------------
ovm22upgrade 1.0 1.0.19.el5 noarch
Testing package set / solving RPM inter-dependencies...

########################################
ovm22upgrade-1.0-1.0.19.el5 ########################## Done.
Preparing ########################################### [100%]
Installing...
1:ovm22upgrade ########################################### [100%]
Please run the following command to continue the upgrade:
/usr/local/sbin/ovm22upgrade.py
[root@ovs3 ~]# /usr/local/sbin/ovm22upgrade.py
Please stop all running Virtual Machine Guests before proceeding.
Next run python script as suggested by the install process as shown in the next example.
# /usr/local/sbin/ovm22upgrade.py
Running phase 1 of upgrade... done
Installing new ovs-release package for OVM 2.2
Fetching Obsoletes list for channel: ovm22_i386_bootstrap...

########################################
Fetching rpm headers...

########################################
Name Version Rel

----------------------------------------------------------
ovs-release 2.2 0.15 i386

########################################
ovs-release-2.2-0.15.i386.r ########################## Done.
Preparing ########################################### [100%]
Installing...
1:ovs-release warning: /etc/issue created as /etc/issue.rpmnew
########################################### [100%]
Running phase 2 of upgrade... done
Updating fstab and grub.conf with disk uuid information
Files /etc/fstab and /boot/grub/grub.conf updated to use UUID instead of LABEL.
Original files are saved as

/etc/fstab.ovsbak_100918
/boot/grub/grub.conf.ovsbak_100918
Please run the 'up2date -fu' command to continue the upgrade.

#
We will concluse the upgrade process by running the following command, as show in the next example.
# up2date -fu
Fetching Obsoletes list for channel: ovm22_i386_latest...

########################################
Name Version Rel

----------------------------------------------------------
OpenIPMI 2.0.6 11.el5 i386
OpenIPMI-libs 2.0.6 11.el5 i386
SysVinit 2.86 15.el5 i386
acl 2.2.39 3.el5 i386
alsa-lib 1.0.17 1.el5 i386
anaconda 11.1.2.168 1.1.16 i386
anaconda-runtime 11.1.2.168 1.1.16 i386
anacron 2.3 45.0.1.el5 i386
audit-libs 1.7.7 6.el5 i386
authconfig 5.3.21 5.el5 i386
autofs 5.0.1 0.rc2.102.el5_3.1 i386
bash 3.2 24.el5 i386
130 of 243 18.04.2011 16:35

bind-libs 9.3.4 10.P1.el5_3.3 i386

bind-utils 9.3.4 10.P1.el5_3.3 i386
binutils 2.17.50.0.6 9.el5 i386
booty 0.80.6 5.1.5 noarch
bzip2 1.0.3 4.el5_2 i386
bzip2-libs 1.0.3 4.el5_2 i386
chkconfig 1.3.30.1 2 i386
compat-python-virtinst 0.99.0 3.el5.2.12 noarch
coreutils 5.97 19.el5 i386
cracklib 2.8.9 3.3 i386
cracklib-dicts 2.8.9 3.3 i386
crash 4.0 7.2.3.0.1 i386
createrepo 0.4.11 3.el5 noarch
cryptsetup-luks 1.0.3 4.el5 i386
curl 7.15.5 2.1.el5_3.5 i386
dbus 1.1.2 12.el5 i386
dbus-glib 0.73 8.el5 i386
device-mapper 1.02.28 2.el5 i386
device-mapper-multipath 0.4.7 23.el5.4.0.1 i386
dhclient 3.0.5 18.el5 i386
dhcpv6-client 1.0.10 16.el5 i386
diffutils 2.8.1 15.2.3.el5 i386
dmraid 1.0.0.rc13 33.el5 i386
dnsmasq 2.45 1.1.el5_3 i386
dosfstools 2.11 7.el5 i386
e2fsprogs 1.39 20.el5 i386
e2fsprogs-libs 1.39 20.el5 i386
ed 0.2 39.el5_2 i386
elfutils-libelf 0.137 3.el5 i386
elinks 0.11.1 6.el5_4.1 i386
enterprise-linux-ovs 5 0.17 i386
ethtool 6 2.el5 i386
file 4.17 15 i386
filesystem 2.4.0 2.0.1 i386
findutils 4.2.27 5.el5 i386
firstboot 1.4.27.7 1.1.1.el5 i386
firstboot-tui 1.4.27.7 1.1.1.el5 i386
fontconfig 2.4.1 7.el5 i386
ftp 0.17 35.el5 i386
glibc 2.5 34 i686
glibc-common 2.5 34 i386
gnupg 1.4.5 14 i386
grub 0.97 13.2.1.1 i386
gzip 1.3.5 10.el5.0.1 i386
hal 0.5.8.1 38.el5 i386
hwdata 0.213.11 1.1.3.el5 noarch
initscripts 8.45.25 1.1.1.el5 i386
iproute 2.6.18 9.el5 i386
iptables 1.3.5 4.el5 i386
iptables-ipv6 1.3.5 4.el5 i386
iputils 20020927 45.el5 i386
iscsi-initiator-utils 6.2.0.871 0.7.el5 i386
kbd 1.12 21.el5 i386
kernel-ovs 2.6.18 128.2.1.4.9.el5 i686
kpartx 0.4.7 23.el5.4.0.1 i386
ksh 20080202 2.el5 i386
kudzu 1.2.57.1.21 1 i386
libX11 1.0.3 9.el5 i386
libacl 2.2.39 3.el5 i386
libbdevid-python 5.1.19.6 44.0.1 i386
libgcc 4.1.2 44.el5 i386
libgcrypt 1.2.4 1.el5 i386
libpcap 0.9.4 14.el5 i386
libselinux 1.33.4 5.1.el5 i386
libstdc++ 4.1.2 44.el5 i386
libtiff 3.8.2 7.el5_3.4 i386
libuser 0.54.7 2.el5.5 i386
libvirt 0.3.3 14.0.2.el5_3.1 i386
libvirt-python 0.3.3 14.0.2.el5_3.1 i386
libxml2 2.6.26 2.1.2.8.0.2 i386
libxml2-python 2.6.26 2.1.2.8.0.2 i386
lm_sensors 2.10.7 4.el5 i386
logrotate 3.7.4 9 i386
131 of 243 18.04.2011 16:35

logwatch 7.3 6.el5 noarch

lvm2 2.02.40 6.el5 i386
m2crypto 0.16 6.el5.3 i386
make 3.81 3.el5 i386
man-pages 2.39 12.el5 noarch
mcstrans 0.2.11 3.el5 i386
mdadm 2.6.4 1.el5 i386
mkinitrd 5.1.19.6 44.0.1 i386
mkisofs 2.01 10.7.el5 i386
module-init-tools 3.3 0.pre3.1.42.el5 i386
nash 5.1.19.6 44.0.1 i386
neon 0.25.5 10.el5_4.1 i386
net-snmp 5.3.2.2 5.el5_3.2 i386
net-snmp-libs 5.3.2.2 5.el5_3.2 i386
net-tools 1.60 78.el5 i386
newt 0.52.2 12.el5_4.1 i386
nfs-utils 1.0.9 40.el5 i386
nscd 2.5 34 i386
nss_db 2.2 35.3 i386
nss_ldap 253 17.el5 i386
ocfs2-tools 1.4.3 4.el5 i386
openldap 2.3.43 3.el5 i386
openssh 4.3p2 29.el5 i386
openssh-clients 4.3p2 29.el5 i386
openssh-server 4.3p2 29.el5 i386
openssl 0.9.8e 7.el5 i686
oprofile 0.9.3 18.el5 i386
ovs-agent 2.3 19 noarch
ovs-release 2.2 0.17 i386
ovs-utils 1.0 33 i386
p2v-util 0.10 6 i386
pam 0.99.6.2 4.el5 i386
pam_krb5 2.2.14 10 i386
parted 1.8.1 23.el5 i386
patch 2.5.4 29.2.3.el5 i386
pciutils 2.2.3 5 i386
pcre 6.6 2.el5_1.7 i386
perl 5.8.8 18.el5 i386
pexpect 2.3 1.el5 noarch
pm-utils 0.99.3 10.el5 i386
popt 1.10.2.3 9.el5 i386
ppp 2.4.4 2.el5 i386
prelink 0.4.0 2.el5 i386
procmail 3.22 17.1.0.1 i386
procps 3.2.7 11.1.el5 i386
psacct 6.3.2 44.el5 i386
psmisc 22.2 6 i386
pykickstart 0.43.3 1.1.2.el5 noarch
pypxeboot 0.0.2 9 noarch
python 2.4.3 24.el5_3.6 i386
python-pyblock 0.26 3.el5 i386
python-urlgrabber 3.1.0 5.el5 noarch
quota 3.13 1.2.5.el5 i386
rdate 1.4 8.el5 i386
rhnlib 2.2.6 2.el5 noarch
rpm 4.4.2.3 9.el5 i386
rpm-libs 4.4.2.3 9.el5 i386
rpm-python 4.4.2.3 9.el5 i386
rsh 0.17 38.el5 i386
setup 2.5.58 4.1.1.el5 noarch
shadow-utils 4.0.17 14.el5 i386
stunnel 4.15 2.0.1.el5.1 i386
sudo 1.6.9p17 3.el5 i386
sysklogd 1.4.1 44.el5 i386
system-config-date 1.8.12 3.el5.2.1 noarch
system-config-keyboard 1.2.11 1.1.1.el5 noarch
system-config-language 1.1.18 2.1.1.el5 noarch
system-config-network-tui 1.3.99.12 1.0.1.el5 noarch
system-config-rootpassword 1.1.9.1 1.1.1 noarch
system-config-securitylevel 1.6.29.1 2.1.1.1.el5 i386
system-config-securitylevel-tui 1.6.29.1 2.1.1.1.el5 i386
tcp_wrappers 7.6 40.6.el5 i386
tcpdump 3.9.4 14.el5 i386
132 of 243 18.04.2011 16:35

tcsh 6.14 14.el5 i386

tftp 0.42 3.1.0.1 i386
tmpwatch 2.9.7 1.1.el5.2 i386
traceroute 2.0.1 5.el5 i386
tzdata 2009k 1.el5 noarch
usermode 1.88 3.el5.2 i386
util-linux 2.13 0.50.el5 i386
vim-minimal 7.0.109 4.el5_2.4z i386
vixie-cron 4.1 76.el5 i386
vnc 4.1.2 14.el5.1.0.1 i386
words 3.0 9.1 noarch
xen 3.4.0 0.0.23.el5 i386
xen-64 3.4.0 0.0.23.el5 noarch
xen-debugger 3.4.0 0.0.23.el5 noarch
xen-devel 3.4.0 0.0.23.el5 i386
xen-tools 3.4.0 0.0.23.el5 i386
ypbind 1.19 11.el5 i386
yum 3.2.19 18.el5 noarch
yum-metadata-parser 1.1.2 2.el5 i386
zip 2.31 2.el5 i386

########################################
OpenIPMI-2.0.6-11.el5.i386. ########################## Done.
OpenIPMI-libs-2.0.6-11.el5. ########################## Done.
SysVinit-2.86-15.el5.i386.r ########################## Done.
acl-2.2.39-3.el5.i386.rpm: ########################## Done.
alsa-lib-1.0.17-1.el5.i386. ########################## Done.
anaconda-11.1.2.168-1.1.16. ########################## Done.
anaconda-runtime-11.1.2.168 ########################## Done.
anacron-2.3-45.0.1.el5.i386 ########################## Done.
audit-libs-1.7.7-6.el5.i386 ########################## Done.
authconfig-5.3.21-5.el5.i38 ########################## Done.
autofs-5.0.1-0.rc2.102.el5_ ########################## Done.
avahi-0.6.16-1.el5.i386.rpm ########################## Done.
bash-3.2-24.el5.i386.rpm: ########################## Done.
bind-libs-9.3.4-10.P1.el5_3 ########################## Done.
bind-utils-9.3.4-10.P1.el5_ ########################## Done.
binutils-2.17.50.0.6-9.el5. ########################## Done.
booty-0.80.6-5.1.5.noarch.r ########################## Done.
bzip2-1.0.3-4.el5_2.i386.rp ########################## Done.
bzip2-libs-1.0.3-4.el5_2.i3 ########################## Done.
chkconfig-1.3.30.1-2.i386.r ########################## Done.
compat-python-virtinst-0.99 ########################## Done.
coreutils-5.97-19.el5.i386. ########################## Done.
cracklib-2.8.9-3.3.i386.rpm ########################## Done.
cracklib-dicts-2.8.9-3.3.i3 ########################## Done.
crash-4.0-7.2.3.0.1.i386.rp ########################## Done.
createrepo-0.4.11-3.el5.noa ########################## Done.
cryptsetup-luks-1.0.3-4.el5 ########################## Done.
curl-7.15.5-2.1.el5_3.5.i38 ########################## Done.
dbus-1.1.2-12.el5.i386.rpm: ########################## Done.
dbus-glib-0.73-8.el5.i386.r ########################## Done.
dbus-libs-1.1.2-12.el5.i386 ########################## Done.
device-mapper-1.02.28-2.el5 ########################## Done.
device-mapper-event-1.02.28 ########################## Done.
device-mapper-multipath-0.4 ########################## Done.
dhclient-3.0.5-18.el5.i386. ########################## Done.
dhcpv6-client-1.0.10-16.el5 ########################## Done.
diffutils-2.8.1-15.2.3.el5. ########################## Done.
dmraid-1.0.0.rc13-33.el5.i3 ########################## Done.
dnsmasq-2.45-1.1.el5_3.i386 ########################## Done.
dosfstools-2.11-7.el5.i386. ########################## Done.
e2fsprogs-1.39-20.el5.i386. ########################## Done.
e2fsprogs-libs-1.39-20.el5. ########################## Done.
e4fsprogs-1.41.1-2.el5.i386 ########################## Done.
ebtables-2.0.8-2.el5.i386.r ########################## Done.
ed-0.2-39.el5_2.i386.rpm: ########################## Done.
elfutils-libelf-0.137-3.el5 ########################## Done.
elinks-0.11.1-6.el5_4.1.i38 ########################## Done.
enterprise-linux-ovs-5-0.17 ########################## Done.
ethtool-6-2.el5.i386.rpm: ########################## Done.
133 of 243 18.04.2011 16:35

file-4.17-15.i386.rpm: ########################## Done.

filesystem-2.4.0-2.0.1.i386 ########################## Done.
findutils-4.2.27-5.el5.i386 ########################## Done.
fipscheck-1.0.3-1.el5.i386. ########################## Done.
firstboot-1.4.27.7-1.1.1.el ########################## Done.
firstboot-tui-1.4.27.7-1.1. ########################## Done.
fontconfig-2.4.1-7.el5.i386 ########################## Done.
ftp-0.17-35.el5.i386.rpm: ########################## Done.
glibc-2.5-34.i686.rpm: ########################## Done.
glibc-common-2.5-34.i386.rp ########################## Done.
gnupg-1.4.5-14.i386.rpm: ########################## Done.
gnutls-1.4.1-3.el5_3.5.i386 ########################## Done.
grub-0.97-13.2.1.1.i386.rpm ########################## Done.
gzip-1.3.5-10.el5.0.1.i386. ########################## Done.
hal-0.5.8.1-38.el5.i386.rpm ########################## Done.
hwdata-0.213.11-1.1.3.el5.n ########################## Done.
initscripts-8.45.25-1.1.1.e ########################## Done.
iproute-2.6.18-9.el5.i386.r ########################## Done.
iptables-1.3.5-4.el5.i386.r ########################## Done.
iptables-ipv6-1.3.5-4.el5.i ########################## Done.
iputils-20020927-45.el5.i38 ########################## Done.
iscsi-initiator-utils-6.2.0 ########################## Done.
kbd-1.12-21.el5.i386.rpm: ########################## Done.
kernel-ovs-2.6.18-128.2.1.4 ########################## Done.
kpartx-0.4.7-23.el5.4.0.1.i ########################## Done.
ksh-20080202-2.el5.i386.rpm ########################## Done.
kudzu-1.2.57.1.21-1.i386.rp ########################## Done.
libX11-1.0.3-9.el5.i386.rpm ########################## Done.
libacl-2.2.39-3.el5.i386.rp ########################## Done.
libbdevid-python-5.1.19.6-4 ########################## Done.
libgcc-4.1.2-44.el5.i386.rp ########################## Done.
libgcrypt-1.2.4-1.el5.i386. ########################## Done.
libpcap-0.9.4-14.el5.i386.r ########################## Done.
libselinux-1.33.4-5.1.el5.i ########################## Done.
libstdc++-4.1.2-44.el5.i386 ########################## Done.
libtiff-3.8.2-7.el5_3.4.i38 ########################## Done.
libuser-0.54.7-2.el5.5.i386 ########################## Done.
libvirt-0.3.3-14.0.2.el5_3. ########################## Done.
libvirt-python-0.3.3-14.0.2 ########################## Done.
libxml2-2.6.26-2.1.2.8.0.2. ########################## Done.
libxml2-python-2.6.26-2.1.2 ########################## Done.
lm_sensors-2.10.7-4.el5.i38 ########################## Done.
logrotate-3.7.4-9.i386.rpm: ########################## Done.
logwatch-7.3-6.el5.noarch.r ########################## Done.
lvm2-2.02.40-6.el5.i386.rpm ########################## Done.
m2crypto-0.16-6.el5.3.i386. ########################## Done.
make-3.81-3.el5.i386.rpm: ########################## Done.
man-pages-2.39-12.el5.noarc ########################## Done.
mcstrans-0.2.11-3.el5.i386. ########################## Done.
mdadm-2.6.4-1.el5.i386.rpm: ########################## Done.
mesa-libGL-6.5.1-7.7.el5.i3 ########################## Done.
mkinitrd-5.1.19.6-44.0.1.i3 ########################## Done.
mkisofs-2.01-10.7.el5.i386. ########################## Done.
module-init-tools-3.3-0.pre ########################## Done.
multiprocessing-2.6.1.1-1.i ########################## Done.
nash-5.1.19.6-44.0.1.i386.r ########################## Done.
neon-0.25.5-10.el5_4.1.i386 ########################## Done.
net-snmp-5.3.2.2-5.el5_3.2. ########################## Done.
net-snmp-libs-5.3.2.2-5.el5 ########################## Done.
net-tools-1.60-78.el5.i386. ########################## Done.
newt-0.52.2-12.el5_4.1.i386 ########################## Done.
nfs-utils-1.0.9-40.el5.i386 ########################## Done.
nscd-2.5-34.i386.rpm: ########################## Done.
nspr-4.7.3-2.el5.i386.rpm: ########################## Done.
nss-3.12.3.99.3-1.el5_3.2.i ########################## Done.
nss_db-2.2-35.3.i386.rpm: ########################## Done.
nss_ldap-253-17.el5.i386.rp ########################## Done.
ocfs2-tools-1.4.3-4.el5.i38 ########################## Done.
open-ovf-0.1-3.0.7.el5.noar ########################## Done.
openldap-2.3.43-3.el5.i386. ########################## Done.
openssh-4.3p2-29.el5.i386.r ########################## Done.
openssh-clients-4.3p2-29.el ########################## Done.
openssh-server-4.3p2-29.el5 ########################## Done.
134 of 243 18.04.2011 16:35

openssl-0.9.8e-7.el5.i686.r ########################## Done.

oprofile-0.9.3-18.el5.i386. ########################## Done.
ovs-agent-2.3-19.noarch.rpm ########################## Done.
ovs-release-2.2-0.17.i386.r ########################## Done.
ovs-utils-1.0-33.i386.rpm: ########################## Done.
p2v-util-0.10-6.i386.rpm: ########################## Done.
pam-0.99.6.2-4.el5.i386.rpm ########################## Done.
pam_krb5-2.2.14-10.i386.rpm ########################## Done.
parted-1.8.1-23.el5.i386.rp ########################## Done.
patch-2.5.4-29.2.3.el5.i386 ########################## Done.
pciutils-2.2.3-5.i386.rpm: ########################## Done.
pcre-6.6-2.el5_1.7.i386.rpm ########################## Done.
perl-5.8.8-18.el5.i386.rpm: ########################## Done.
pexpect-2.3-1.el5.noarch.rp ########################## Done.
pm-utils-0.99.3-10.el5.i386 ########################## Done.
popt-1.10.2.3-9.el5.i386.rp ########################## Done.
ppp-2.4.4-2.el5.i386.rpm: ########################## Done.
prelink-0.4.0-2.el5.i386.rp ########################## Done.
procmail-3.22-17.1.0.1.i386 ########################## Done.
procps-3.2.7-11.1.el5.i386. ########################## Done.
psacct-6.3.2-44.el5.i386.rp ########################## Done.
psmisc-22.2-6.i386.rpm: ########################## Done.
pykickstart-0.43.3-1.1.2.el ########################## Done.
pypxeboot-0.0.2-9.noarch.rp ########################## Done.
python-2.4.3-24.el5_3.6.i38 ########################## Done.
python-iniparse-0.2.3-4.el5 ########################## Done.
python-pyblock-0.26-3.el5.i ########################## Done.
python-simplejson-2.0.7-2.e ########################## Done.
python-urlgrabber-3.1.0-5.e ########################## Done.
quota-3.13-1.2.5.el5.i386.r ########################## Done.
rdate-1.4-8.el5.i386.rpm: ########################## Done.
redhat-lsb-3.1-12.3.0.1.EL. ########################## Done.
rhnlib-2.2.6-2.el5.noarch.r ########################## Done.
rpm-4.4.2.3-9.el5.i386.rpm: ########################## Done.
rpm-libs-4.4.2.3-9.el5.i386 ########################## Done.
rpm-python-4.4.2.3-9.el5.i3 ########################## Done.
rsh-0.17-38.el5.i386.rpm: ########################## Done.
setup-2.5.58-4.1.1.el5.noar ########################## Done.
shadow-utils-4.0.17-14.el5. ########################## Done.
stunnel-4.15-2.0.1.el5.1.i3 ########################## Done.
sudo-1.6.9p17-3.el5.i386.rp ########################## Done.
sysklogd-1.4.1-44.el5.i386. ########################## Done.
system-config-date-1.8.12-3 ########################## Done.
system-config-keyboard-1.2. ########################## Done.
system-config-language-1.1. ########################## Done.
system-config-network-tui-1 ########################## Done.
system-config-rootpassword- ########################## Done.
system-config-securitylevel ########################## Done.
system-config-securitylevel ########################## Done.
tcp_wrappers-7.6-40.6.el5.i ########################## Done.
tcpdump-3.9.4-14.el5.i386.r ########################## Done.
tcsh-6.14-14.el5.i386.rpm: ########################## Done.
tftp-0.42-3.1.0.1.i386.rpm: ########################## Done.
tmpwatch-2.9.7-1.1.el5.2.i3 ########################## Done.
traceroute-2.0.1-5.el5.i386 ########################## Done.
tzdata-2009k-1.el5.noarch.r ########################## Done.
usermode-1.88-3.el5.2.i386. ########################## Done.
util-linux-2.13-0.50.el5.i3 ########################## Done.
vim-minimal-7.0.109-4.el5_2 ########################## Done.
vixie-cron-4.1-76.el5.i386. ########################## Done.
vnc-4.1.2-14.el5.1.0.1.i386 ########################## Done.
words-3.0-9.1.noarch.rpm: ########################## Done.
xen-3.4.0-0.0.23.el5.i386.r ########################## Done.
xen-64-3.4.0-0.0.23.el5.noa ########################## Done.
xen-debugger-3.4.0-0.0.23.e ########################## Done.
xen-devel-3.4.0-0.0.23.el5. ########################## Done.
xen-tools-3.4.0-0.0.23.el5. ########################## Done.
xml-common-0.6.3-18.0.1.noa ########################## Done.
ypbind-1.19-11.el5.i386.rpm ########################## Done.
yum-3.2.19-18.el5.noarch.rp ########################## Done.
yum-metadata-parser-1.1.2-2 ########################## Done.
zip-2.31-2.el5.i386.rpm: ########################## Done.
dbus-python-0.70-7.el5.i386 ########################## Done.
135 of 243 18.04.2011 16:35

gettext-0.14.6-4.el5.i386.r ########################## Done.

libXxf86vm-1.0.1-3.1.i386.r ########################## Done.
libdaemon-0.10-5.el5.i386.r ########################## Done.
libdrm-2.0.2-1.1.i386.rpm: ########################## Done.
m4-1.4.5-3.el5.1.i386.rpm: ########################## Done.
Preparing ########################################### [100%]
Installing...
1:libgcc ########################################### [100%]
2:tzdata ########################################### [100%]
3:glibc-common ########################################### [100%]
4:glibc ########################################### [100%]
5:bash ########################################### [100%]
6:chkconfig ########################################### [100%]
7:popt ########################################### [100%]
8:audit-libs ########################################### [100%]
9:bzip2-libs ########################################### [100%]
10:libxml2 ########################################### [100%]
11:libstdc++ ########################################### [100%]
12:tcp_wrappers ########################################### [100%]
13:elfutils-libelf ########################################### [100%]
14:perl ########################################### [100%]
15:libX11 ########################################### [100%]
16:libacl ########################################### [100%]
17:binutils ########################################### [100%]
18:nspr ########################################### [100%]
19:nss ########################################### [100%]
20:procps ########################################### [100%]
21:iptables ########################################### [100%]
22:libgcrypt ########################################### [100%]
23:lm_sensors ########################################### [100%]
24:diffutils ########################################### [100%]
25:gzip ########################################### [100%]
26:make ########################################### [100%]
27:iptables-ipv6 ########################################### [100%]
28:iputils ########################################### [100%]
29:iproute ########################################### [100%]
30:dosfstools ########################################### [100%]
31:ethtool ########################################### [100%]
32:file ########################################### [100%]
33:mkisofs ########################################### [100%]
34:tftp ########################################### [100%]
35:setup ########################################### [100%]
36:filesystem ########################################### [100%]
37:enterprise-linux-ovs ########################################### [100%]
38:cracklib-dicts ########################################### [100%]
39:booty ########################################### [100%]
40:gnutls ########################################### [100%]
41:libXxf86vm ########################################### [100%]
42:libtiff ########################################### [100%]
43:pcre ########################################### [100%]
44:ed ########################################### [100%]
45:m4 ########################################### [100%]
46:patch ########################################### [100%]
47:libdaemon ########################################### [100%]
48:libdrm ########################################### [100%]
49:nash ########################################### [100%]
50:grub ########################################### [100%]
51:crash ########################################### [100%]
52:oprofile ########################################### [100%]
53:acl ########################################### [100%]
54:bzip2 ########################################### [100%]
55:anacron ########################################### [100%]
56:ypbind ########################################### [100%]
57:mdadm ########################################### [100%]
58:ovs-release warning: /etc/issue created as /etc/issue.rpmnew
########################################### [100%]
59:procmail ########################################### [100%]
60:ftp ########################################### [100%]
61:rdate ########################################### [100%]
62:rsh ########################################### [100%]
63:traceroute ########################################### [100%]
64:zip ########################################### [100%]
136 of 243 18.04.2011 16:35

65:words ########################################### [100%]

66:man-pages ########################################### [100%]
67:libselinux ########################################### [100%]
68:device-mapper ########################################### [100%]
69:e2fsprogs-libs ########################################### [100%]
70:openssl ########################################### [100%]
71:shadow-utils ########################################### [100%]
72:openldap ########################################### [100%]
73:e2fsprogs ########################################### [100%]
74:findutils ########################################### [100%]
75:coreutils ########################################### [100%]
76:python ########################################### [100%]
77:module-init-tools ########################################### [100%]
78:libxml2-python ########################################### [100%]
79:newt ########################################### [100%]
80:kpartx ########################################### [100%]
81:system-config-securityl########################################### [100%]
82:ovs-utils ########################################### [100%]
83:curl ########################################### [100%]
84:net-snmp-libs ########################################### [100%]
85:net-tools ########################################### [100%]
86:psmisc ########################################### [100%]
87:dmraid ########################################### [100%]
88:hwdata ########################################### [100%]
89:pciutils ########################################### [100%]
90:cracklib ########################################### [100%]
91:pam warning: /etc/pam.d/system-auth created as /etc/pam.d/system-auth.rpmnew
########################################### [100%]
92:util-linux ########################################### [100%]
93:initscripts ########################################### [100%]
94:libuser ########################################### [100%]
95:usermode ########################################### [100%]
96:system-config-date ########################################### [100%]
97:yum-metadata-parser ########################################### [100%]
98:OpenIPMI-libs ########################################### [100%]
99:bind-libs ########################################### [100%]
100:libpcap ########################################### [100%]
101:cryptsetup-luks ########################################### [100%]
102:parted ########################################### [100%]
103:SysVinit ########################################### [100%]
104:logrotate ########################################### [100%]
105:authconfig ########################################### [100%]
106:kbd ########################################### [100%]
107:system-config-language ########################################### [100%]
108:system-config-securityl########################################### [100%]
109:dnsmasq ########################################### [100%]
110:mcstrans ########################################### [100%]
111:openssh ########################################### [100%]
112:sysklogd ########################################### [100%]
113:m2crypto ########################################### [100%]
114:python-urlgrabber ########################################### [100%]
115:compat-python-virtinst ########################################### [100%]
116:pykickstart ########################################### [100%]
117:nscd ########################################### [100%]
118:mesa-libGL ########################################### [100%]
119:vim-minimal ########################################### [100%]
120:p2v-util ########################################### [100%]
121:vixie-cron ########################################### [100%]
122:ebtables ########################################### [100%]
123:pam_krb5 ########################################### [100%]
124:device-mapper-multipath########################################### [100%]
125:multiprocessing ########################################### [100%]
126:python-iniparse ########################################### [100%]
127:python-simplejson ########################################### [100%]
128:gettext ########################################### [100%]
129:redhat-lsb ########################################### [100%]
130:alsa-lib ########################################### [100%]
131:fontconfig ########################################### [100%]
132:xml-common ########################################### [100%]
133:fipscheck ########################################### [100%]
134:nss_ldap ########################################### [100%]
135:neon ########################################### [100%]
136:device-mapper-event ########################################### [100%]
137 of 243 18.04.2011 16:35

137:lvm2 ########################################### [100%]

138:mkinitrd ########################################### [100%]
139:libbdevid-python ########################################### [100%]
140:python-pyblock ########################################### [100%]
141:e4fsprogs ########################################### [100%]
142:nss_db ########################################### [100%]
143:ocfs2-tools warning: /etc/sysconfig/o2cb created as /etc/sysconfig/o2cb.rpmnew
########################################### [100%]
144:pypxeboot ########################################### [100%]
145:sudo ########################################### [100%]
146:openssh-clients ########################################### [100%]
147:openssh-server ########################################### [100%]
148:ppp ########################################### [100%]
149:bind-utils ########################################### [100%]
150:OpenIPMI ########################################### [100%]
151:system-config-rootpassw########################################### [100%]
152:dhclient ########################################### [100%]
153:dhcpv6-client ########################################### [100%]
154:quota ########################################### [100%]
155:vnc ########################################### [100%]
156:nfs-utils ########################################### [100%]
157:prelink ########################################### [100%]
158:tmpwatch ########################################### [100%]
159:gnupg ########################################### [100%]
160:iscsi-initiator-utils ########################################### [100%]
161:pexpect ########################################### [100%]
162:rhnlib ########################################### [100%]
163:autofs ########################################### [100%]
164:ksh ########################################### [100%]
165:logwatch ########################################### [100%]
166:psacct ########################################### [100%]
167:tcsh ########################################### [100%]
168:tcpdump ########################################### [100%]
169:elinks ########################################### [100%]
170:stunnel ########################################### [100%]
171:dbus-libs ########################################### [100%]
172:xen-devel ########################################### [100%]
173:rpm-libs ########################################### [100%]
174:rpm ########################################### [100%]
175:rpm-python ########################################### [100%]
176:dbus-glib ########################################### [100%]
177:xen-tools warning: /etc/xen/xend-config.sxp created as /etc/xen/xend-config.sxp.rpmnew
########################################### [100%]
178:dbus ########################################### [100%]
179:xen ########################################### [100%]
180:createrepo ########################################### [100%]
181:yum ########################################### [100%]
182:dbus-python ########################################### [100%]
183:avahi ########################################### [100%]
184:libvirt ########################################### [100%]
185:libvirt-python ########################################### [100%]
186:open-ovf ########################################### [100%]
187:ovs-agent ########################################### [100%]
188:kernel-ovs ########################################### [100%]
189:xen-64 ########################################### [100%]
190:xen-debugger ########################################### [100%]
191:net-snmp ########################################### [100%]
192:hal ########################################### [100%]
193:kudzu ########################################### [100%]
194:system-config-network-t########################################### [100%]
195:system-config-keyboard ########################################### [100%]
196:pm-utils ########################################### [100%]
197:firstboot-tui ########################################### [100%]
198:firstboot ########################################### [100%]
199:anaconda ########################################### [100%]
200:anaconda-runtime ########################################### [100%]
The following packages were added to your selection to satisfy dependencies:
Name Version Release

--------------------------------------------------------------
avahi 0.6.16 1.el5
dbus-libs 1.1.2 12.el5
device-mapper-event 1.02.28 2.el5
138 of 243 18.04.2011 16:35

e4fsprogs 1.41.1 2.el5

ebtables 2.0.8 2.el5
fipscheck 1.0.3 1.el5
gnutls 1.4.1 3.el5_3.5
mesa-libGL 6.5.1 7.7.el5
multiprocessing 2.6.1.1 1
nspr 4.7.3 2.el5
nss 3.12.3.99.3 1.el5_3.2
open-ovf 0.1 3.0.7.el5
python-iniparse 0.2.3 4.el5
python-simplejson 2.0.7 2.el5
redhat-lsb 3.1 12.3.0.1.EL
xml-common 0.6.3 18.0.1
dbus-python 0.70 7.el5
gettext 0.14.6 4.el5
libXxf86vm 1.0.1 3.1
libdaemon 0.10 5.el5
libdrm 2.0.2 1.1
m4 1.4.5 3.el5.1
#
Reboot the Oracle VM Pool Master Server
Next we will reboot the Oracle VM pool master server as shown in the next example.
# reboot
Broadcast message from root (pts/0) (Sun Oct 18 15:03:22 2009):
The system is going down for reboot NOW!

Reboot the None Master Oracle VM servers
Next we reboot all of the none master Oracle VM servers as shown in the next example.
# reboot
Broadcast message from root (pts/0) (Sun Oct 18 15:03:22 2009):
The system is going down for reboot NOW!

Enter the Server Pool Virtual IP using Oracle VM Manager
If you are using HA, the final step is to access Oracle VM Manager and enter a server pool virtual IP address for your server pool or
pools. The server pool virtual IP is used to provide automatic Oracle VM master server failover to an Oracle VM none master pool
member. The server pool virtual IP must be a vaild IP address on the same network as the pool servers.
Access Oracle VM Manager, click the Server Pools tab then click the Edit button to access the Edit Server Pool page. From the
Edit Server Pool page, enter the server pool virtual IP in the Server Pool Virtual IP text box then click OK as shown in Figure 12.
Watch this space

139 of 243 18.04.2011 16:35

Chapter 6: Oracle VM 2.2 SAN, iSCSI and NFS Back-end

Storage Configurations
This chapter outlines how to configure an Oracle VM 2.2 pool with Fibre Channel and iSCSI SANs and NFS back-end storage. The
chapter also covers guest front-end storage options and configurations. The chapter starts with an overview of the Oracle VM storage
stack, followed by an introduction to Oracle VM back-end storage options, configurations and considerations. Next, we will summarize
storage administration with Oracle VM 2.2 followed by example root and extended storage repository (SR) configurations using Fibre
Channel and iSCSI SANs and NFS storage arrays. The chapter concludes with a review Oracle VM guest front-end storage options
and configurations.
Table of Contents
The Oracle VM Storage Stack
OCFS2 Cluster File System
OCFS2 User-Space Management Utilities and Commands
Oracle VM Storage Repositories
Storage Administration
The Storage Array Layer
…Swap
…Backup and Restoration
…SAN and iSCSI Multi-pathing
…Sparse Files and Unwritten Extents
The Server Layer
SAN Storage Repository Configuration
…SAN Back-end Storage Prerequisites
…SAN Oracle VM Server Prerequisites
…Adding an Extended SAN Storage Repository
iSCSI Storage Repository Configuration
…iSCSI Back-end Storage Prerequisites
…iSCSI Oracle VM Server Prerequisites
…Adding an Extended iSCSI Storage Repository
NFS Storage Repository Configuration
…NFS Back-end Storage Prerequisites
…NFS Oracle VM Server Prerequisites
…Adding an Extended NFS Storage Repository
Oracle VM Guest Front-end Storage
…File-backed block device
…Physical backed block device
Appendix A Example multipath.conf Files
The Oracle VM Storage Stack

An Oracle VM storage solution consists of three distinct layers. Each layer has its own unique requirements, configurations,
dependencies and features. The first layer is the storage array, which is referred to as back-end storage. Oracle VM supports local
storage, Fibre Channel and iSCSI SANs and NFS back-end storage. The second layer is the server layer, which consists of the Oracle
VM server storage configurations and the virtual machine file system, i.e. the Oracle Cluster File System 2 (OCFS2) or NFS. Oracle
VM supports a wide variety of configurations for Fibre Channel and iSCSI SANs and NFS storage arrays. The third layer is the guest
front-end storage, which consists of multiple guest storage and driver options.
Note: Oracle VM supports both local and shared back-end storage. Local storage refers to a file system that can only be accessed by a
single Oracle VM server. This chapter covers shared back-end storage supporting a clustered multi server pool environment, not local
storage.
Figure 1 shows a high-level overview of the three layers of the storage stack with a virtual machine running on an Oracle VM server,
connected to a storage array. At the bottom of the stack is the storage array. The storage array layer is where the physical disks are
managed and presented to the Oracle VM pool members as logical disks. Above the storage array is the server layer. The server layer
is where the storage configurations and the OCSF2 or NFS virtual machine file system (the cluster stack) are managed. At the top of
the stack is the virtual machine layer. The virtual machine layer is where virtual machine storage is presented to the virtual machine
by the Oracle VM server.
Figure 1 shows the Oracle VM storage stack.
140 of 243 18.04.2011 16:35

When designing an Oracle VM virtual environment, one of the most important considerations is the back-end storage. There are many
back-end storage options ranging from local storage, Fibre Channel and iSCSI SANs and NFS. Each back-end storage option has its
own capacity, performance and availability features. The back-end storage is where you store and run the virtual machines. Oracle VM
supports the Oracle Cluster File System v2 (OCSF2) and NFS on the back-end storage to store and run the virtual machines. The
OCFS2 cluster file system and NFS both have their own unique management, performance and availability features.
The next section will review the OCSF2 cluster file system, NFS and the cluster stack. The goal of the OCSF2 section is to provide an
overview of the architecture,configurations, dependencies and features of the virtual machine file system, at the server layer of the
storage stack. Understanding the architecture, configurations, dependencies and features at the server layer will allow you to design
a manageable back-end storage solution for Oracle VM.
Note: For the remainder of this chapter, the terms “pool” and “cluster” should be considered to be interchangeable.
OCFS2 Cluster File System

The Oracle Cluster File System 2 (OCFS2) is a general-purpose journaling file system developed by Oracle. Oracle released OCFS2
under the GNU General Public License (GPL), version 2. The OCSF2 source code and its tool set are part of the mainline Linux 2.6
kernel and above. The OCSF2 source code and its tool set can be downloaded from kernel.org and from the Unbreakable Linux
Network.
Note: OCFS2 is not integrated or supported with any volume manager (LVM) to manage the back-end block storage. Fibre Channel
and iSCSI partitions must be provisioned at static sizes, i.e. partition sizes can not change once a partition is formatted with OCFS2.
Many customers try to use LVM to manage the back-end block storage for OCFS2. LVM is not cluster aware, so changes made to the
back-end block storage by LVM will not be propagated to the OCFS2 file system. The Oracle VM pool members would continue to
write to the old volume layout, and corruption will occur.
OCFS2 has two components, a kernel component and a user-space component. The kernel component consists of the file system and
the cluster stack. The user-space component consists of the utilities to manage the file system and the cluster stack.
A slightly modified version of OCSF2 (o2dlm) is bundled with Oracle VM. The OCFS2 file system and cluster stack are installed and
configured as part of an Oracle VM server installation. The o2cb service manages the cluster stack and the ocfs2 service manages the
OCSF2 file system. The o2cb cluster service is a set of modules and in-memory file systems that manage the ocfs2 file system service.
Once a server pool is created using Oracle VM Manager, two cluster configuration files are shared across the server pool that
maintain the cluster layout and cluster timeouts configurations. The /etc/ocfs2/cluster.conf file maintains the cluster layout and the
/etc/sysconfig/o2cb file maintains the cluster timeouts. Both configuration files are read by the user-space utility configfs. configfs
communicates the list of nodes in the /etc/ocfs2/cluster.conf file to the in-kernel node manager, along with the resource used for the
heartbeat to the in-kernel heartbeat thread.
The ovs-agent, which is also installed and configured by default, is responsible for propagating the /etc/ocfs2/cluster.conf file to all of
the pool members. The ovs-agent is an Oracle VM server service that is used for centralized pool management, orchestrated by Oracle
VM Manager or the Oracle VM Management Pack. Each time an ovs-agent starts and stops, it updates the pool status, which is
managed by the master pool agent. The master pool agent updates the pool membership status and then propagates an up to date
/etc/ocfs2/cluster.conf file to all of the pool’s ovs-agents.
An Oracle VM server must be online to be in an OCFS2 cluster. Once the cluster is on-line, each pool member starts a process, o2net.
The o2net process creates TCP/IP intra-cluster node communication channels on port 7777 and sends regular keepalive packages to
each node in the cluster to validate if the nodes are alive. The intra-cluster node communication uses the Oracle VM management
network. The Oracle VM management network is selected during the Oracle VM server installation. If a pool member falls of the
network and the keepalive connection becomes silent, the server will self-fence. Fencing forcefully removes dead servers from a pool
to ensure that active servers are not obstructed from accessing fenced servers cluster resources.
Along with the keepalive packages that check for node connectivity, the cluster stack also employs a disk heartbeat check. o2hb is the
process that is responsible for the disk heartbeat component of cluster stack that actively monitors the status of all pool members.
The heartbeat system uses a file on the OCSF2 file system, that each pool member periodically writes a block to, along with a time
stamp. The time stamps are read by each pool member and are used to check if a pool member is alive or dead. If a pool member’s
block stops getting updated, the server is considered dead. When a server dies, the server gets fenced. Fencing forcefully removes
dead pool member from the pool to ensure that active pool members are not obstructed from accessing fenced pool members
resources.
141 of 243 18.04.2011 16:35

Another important OCSF2 component is the distributed lock manager. The distributed lock manager (o2dlm) tracks all the locks in the
cluster, including lock ownership and lock status. Cluster locking is added at the lowest level, in the xend code. The locking method is
defined in the xend-config.sxp file, (xend-domains-lock-path /opt/ovs-agent-2.3/utils/dlm.py) All access methods must take a lock, for .
.example, Oracle VM Manager, xm and the XenAPIdlm is also used for Oracle VM HA, which relies on the cluster stack to validate
pool member status for HA purposes. For example, as pool members boot, reboot and restart, pool membership status will change
across the pool.
There is also a virtual filesystem interface (dlmfs) that allows user space processes to access the in-kernel distributed lock manager.
dlmfs communicates locking and unlocking for pool wide locks on resources to the in-kernel distributed lock manager. The in-kernel
distributed lock manager keeps track of all locks and their owners and status. The o2cb init script mounts the virtual filesystem under
/dlm on each Oracle VM server.
To provide OCFS2 functionality with an NFS storage repository, Oracle VM uses a hidden OCFS2 file-backed block device that
facilitates the use of the OCFS2 distributed lock manager (DLM) with NFS. The ability to use the OCFS2 distributed lock manager
with OCFS2 and NFS allows Oracle VM to monitor both OCFS2 and NFS storage repositories with the same interface.
Table 1 shows the OCSF2 cluster service stack.

Service Description
In Kernel Node The in kernel node manager tracks

Manager (NM) all of the pool members in the
/etc/ocfs2/cluster.conf file.
Network and The network and storage heartbeat

Storage Heartbeat dispatches up/down notifications
(HB) when pool members join or leave the
cluster.
TCP/IP The TCP/IP protocols handle the

communication between pool
members.
Distributed Lock DLM tracks locks in the pool,

Manager (DLM) including lock ownership and lock
status.
configfs configfs communicates the list of

pool members to the in-kernel node
manager. configsf also
communicates the heartbeat
resource to the in-kernel heartbeat
thread. Configfs mounts under
/sys/kernel/config
dlmfs dlmfs communicates locking and

unlocking for pool wide locks on
resources to the in-kernel
distributed lock manager. The
in-kernel distributed lock manager
keeps track of all locks and their
owners and status. The dlmfs user
space virtual filesystem interface
mounts under /dlmfs
Now that we have reviewed the components of the OCFS2 file system and cluster stack, let’s see how OCFS2 works together with
Oracle VM.
When an Oracle VM 2.2 server boots, the o2cb and ocfs2 services are started which bring up the OCFS2 clusterstack. Once the
OCFS2 clusterstack is online, the ovs-agent informs the pool master that the node is online. Next, the pool master updates the
nodemap file with the node’s online status. Next, the ovs-agent queries the pool master and pulls down an up-to-date /etc/ocfs2
/cluster.conf configuration. Next, the ovs-agent mounts the root and any extended repositories and checks that /OVS is symlinked
correctly.
When a pool member stops, starts or dies, the pool master attempts to take an EX lock for the dead pool member’s resources. The
master agent then updates the nodemap file to monitor the aliveness of all active pool members. Whenever the pool membership
status changes, the mater agent will recreate the cluster.conf file and propagate the changes to all of the pool members.
The next section will review the OCFS2 user-space management utilities and commands.
OCFS2 User-Space Management Utilities and Commands

OCFS2 has a full suite of utilities to manage the OCFS2 file system and the cluster stack.
Table 2 lists the OCFS2 file system utilities that are available in dom0.
OCFS2 Description
142 of 243 18.04.2011 16:35

Utility
mkfs.ocfs2 mkfs.ocfs2 is used to format an OCFS2 file system on a

device. mkfs.ocfs2 requires the O2CB cluster service to be
up.
tune.ocfs2 tune.ocfs2 is used to manage OCFS2 file system

parameters, including the volume label, number of node
slots and journal size for all node slots.
mounted.ocfs2 mounted.ocfs2 detects and lists all OCFS2 volumes on an

Oracle VM server.
fsck.ocfs2 fsck.ocfs2 checks and repairs the OCFS2 file system.
debugfs.ocfs2 debugfs.ocfs2 is used to query the state of the OCFS file

system for debugging.
Table 3 lists the commands to manage the o2cb services (the clusterstack).
Command Description
/etc/init.d/o2cb Reports if the o2cb services are loaded and mounted

status
/etc/init.d/o2cb Loads the O2CB modules and in-memory file systems

load
/etc/init.d/o2cb Onlines the cluster named ocfs2. The default name for
online ocfs2 Oracle VM OCFS2 cluster is ocfs2. The cluster name is
defined in the cluster.conf file. At least one pool
member must be active for the cluster to be online.
/etc/init.d/o2cb Offlines the cluster named ocfs2. The default name for
offline ocfs2 Oracle VM OCFS2 cluster is ocfs2. The cluster name is
defined in the cluster.conf file.
/etc/init.d/o2cb Unloads the O2CB modules and in-memory file systems

unload
/etc/init.d/o2cb Starts the cluster named ocfs2 by loading o2cb and

start ocfs2 onlining the cluster. The default name for Oracle VM
OCFS2 cluster is ocfs2. The cluster name is defined in
the cluster.conf file. At least one pool member must be
active for the cluster to be online.
Next, we will review Oracle VM local storage, Fibre Channel and iSCSI SANs and NFS storage repositories.
Oracle VM Storage Repositories

A default Oracle VM 2.2 server installation creates a “local” OCFS2 virtual machine file system that is mounted under /var/ovs
/mount/UUID and linked to /OVS. Using a local storage repository restricts pool membership to “one” Oracle VM server without Live
Migration or HA functionality. To increase the capacity of an Oracle VM pool past one Oracle VM server, the addition of a shared
back-end storage repository is required.
An Oracle VM storage repository can consist of “one” large repository, commonly referred to as “a root repository” or a root
repository with multiple extended sub repositories. Oracle VM 2.x does not have volume management, so adding storage to a root
repository volume will not grow the root repository. The only option to grow an Oracle VM 2.x storage repository is to add sub
repositories beneath the root repository. A best practice is to provision one or more larger repositories to avoid the management
overhead of numerous sub repositories.
Tip: In general, you should consider provisioning at least 30% to 50% more storage for your Oracle VM storage repositories than the
expected size.
Configuring an Oracle VM pool’s storage repository is a multi step process. Once the back-end storage is provisioned, the pool master
must be connected to the storage from dom0. Next, all of the Oracle VM servers that will be added to the pool should be connected to
the storage, again from dom0. Finally, all of the Oracle VM servers should be added to the pool using Oracle VM Manager or the
Oracle VM Management Pack. Once the pool has multiple servers, virtual machines can start on and migrate to any server in the pool.
To add storage to an Oracle VM storage repository, the first step is to provision the storage. Next, connect the pool master, followed
by each pool member to the storage using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new) followed by the -i (initialize,
aka mount) switches, to add and then mount the sub storage repository. Finally the new mount point in /var/ovs/mount/UUID needs to
be linked to /OVS/UUID, by typing “ln -nsf /var/ovs/mount/<UUID>/OVS”, again from dom0. The end result is a root repository with an
“extended” sub repository mounted under /var/ovs/mount/UUID, linked to /OVS/UUID. The Oracle VM agents will automatically place
resources such as virtual machines, templates, or ISO files on the root or sub repository with available space.
Figure 2 shows a root storage repository.
143 of 243 18.04.2011 16:35

Figure 3 shows a root storage repository and an extended sub repository.
Oracle VM 2.2 uses the /opt/ovs-agent-2.3/utils/repos.py script to configure storage repositories and a local Berkley DB to save the
storage repository configurations. The Oracle VM agent is also responsible for mounting and linking storage repositories when an
Oracle VM server boots or restarts. For example, you will not see entries in /etc/fstab for any Oracle VM storage repositories. Oracle
VM storage repository configurations are saved in a local Berkley DB or in a shared Berkley DB in the root storage repository.
Oracle VM root and extended storage repositories all share the same directory structure. Oracle VM’s OCFS2 file system,
clusterstack, repos.py script, Oracle VM agent, Oracle VM Manager as well as the Oracle VM Management Pack are wired to use the
default storage repository directory structure.
The following example shows the Oracle VM storage repository directory structure including a brief explanation of each directory.
/OVS (Root directory)

| B47E850ABA50460882B30645CF051619 (UUID of an extended file system)
| iso_pool (ISO files storage, requires VT chip extensions)
| lost+found (The lost and found directory)
| publish_pool (Public virtual machine storage)
| running_pool (Published virtual machine storage)
| seed_pool (Virtual Machine template storage)
| sharedDisk (Shared virtual disk storage)
The next example shows the storage repository directory structure of an extended storage repository.
/ B47E850ABA50460882B30645CF051619 (UUID of an extended file system)

| iso_pool (ISO files storage, requires VT chip extensions)
| lost+found (The lost and found directory)
| publish_pool (Public virtual machine storage)
| running_pool (Published virtual machine storage)
| seed_pool (Virtual Machine template storage)
| sharedDisk (Shared virtual disk storage)
Storage Administration
Now that we know all about the OCFS2 file system, clusterstack and the storage repository directory structure, we will to turn our
attention to Oracle VM storage administration. As discussed in the Oracle VM Storage Stacksection, there are three distinct layers of
an Oracle VM storage solution. The first layer is the storage array, which is referred to as back-end storage. The second layer is the
server layer, which consists of the Oracle VM server storage configurations and the virtual machine file system. The third layer is the
144 of 243 18.04.2011 16:35

guest front-end storage, which consists of multiple guest storage and driver options.
The following sections will review storage administration at each layer of the storage stack.
The Storage Array Layer

Oracle VM storage administration is done at the storage array layer. Oracle VM Manager or the Oracle VM Management pack is
responsible for pool creation, not for storage repository management. For example, storage repository provisioning, storage
repository snapshotting, storage repository replication, storage repository monitoring as well as storage repository backup and
restoration are preformed at the storage array layer.
Storage array configurations and storage best practices are vendor specific and out of the scope of this document. Please consult your
storage administrators and storage vendor and application owners to help develop a storage solution to meet your business
requirements. This section will review Oracle VM specific storage array considerations.
Administrators do not always have the luxury to design the best storage solution for their environment. Ultimately management
makes the decisions and administrators make the best out of what equipment they get. Ideally, we would like to design a storage
solution that allowed us to provision tiered storage for different workloads. Each workload, for example, RAC, Fusion Middleware, and
E-Business Suite has different requirements, so depending on the workload, back-end disk configurations and guest front-end disk
configurations will affect the performance of the workloads. The only way to validate the best configuration for a workload is to
benchmark the workload using a variety of back-end and front-end configurations. Once you know which configurations provide the
best performance for a given workload then it is time to provision and configure the back-end and front-end storage accordingly.
Swap
Swap is another storage array layer component that requires careful consideration. The best practice for guests is to add RAM to a
guest to tune the database or application workload to minimize swapping altogether. If some swapping is necessary, placing the
guests swap files on the Oracle VM server’s local disk will offer better peformnace than hosting a guests swap file on a SAN. Paging
over a SAN in parallel with swap traffic from other guests can easily contribute to an I/O bottleneck. Swap traffic from other guests is
especially bad when a common set of physical LUNs is provisioned as swap space for many guests. If several guests load up and start
swapping heavily, all the guests on that storage will grind to a halt waiting for the saturated LUNs to respond.
Please note that placing a guests swap file on an Oracle VM server’s local disk will eliminate the ability to use Live Migration.
Backup and Restoration

Identify a backup and restoration strategy for the guests. If the storage array does not offer a suitable guest backup and restoration
solution at the storage array layer consider using an OS agent based backup and restoration solution.
SAN and iSCSI Multi-pathing

Configure the Fibre Channel and iSCSI multi-pathing using dm-multipath. Installing 3 rd party SAN connectivity software in dom0 is
not supported by Oracle.
Sparse Files and Unwritten Extents

Guest virtual disks stored on Oracle VM 2.2 and above OCFS2 file systems use sparse files and unwritten extents by default. When
using sparse files and unwritten extents, guest virtual disk files grows proportionally to the number of writes to the disk by the guest,
so that large portions of the unused disk do not consume space.
The advantage of using sparse files is that storage is allocated only when needed which reduces the time it takes to create sparse files
along with saving disk space.
The disadvantage of using sparse files is that the file system free space reports may be misleading. For example, since storage is
allocated only when needed, the file system free space reports may not be accurate since large portions of unused disk, i.e. the sparse
zero sections have not yet been written to disk.
Tip: Some application do not support copying sparse files and may copy the entire uncompressed size of the file including the sparse
sections.
The Server Layer

Configuring an Oracle VM pool’s storage repository is a multi step process. Once the back-end storage is provisioned, the pool master
must be connected to the storage from dom0. Oracle VM supports SAN, iSCSI and NFS back-end storage. Once the pool master is
connected to the storage and an HA pool is created in Oracle VM Manager, each pool member should be configured to access the
storage, and then added to the pool using Oracle VM Manager.
Next, we walk through example root and extended storage repository (SR) configurations using Fibre Channel and iSCSI SANs and
NFS storage arrays.
SAN Storage Repository Configuration

In this section we will walk through the steps to configure Oracle VM servers using a Fibre Channel SAN storage array. All of the
steps will be executed on each Oracle VM server from dom0 as root. Once all of the Oracle VM servers are configured, an HA enabled
pool will be created in Oracle VM Manager with the pool master server. Next, all of the other configured Oracle VM server will be
added to the HA enabled pool.
Tip: An HA enabled pool will automatically add (repos.py -n and -r) and mount (repos.py -i) root and extended storage repositories for
all pool members.
145 of 243 18.04.2011 16:35

OCFS2 is not integrated or supported with any volume manager (LVM) to manage the back-end block storage. Fibre Channel and
iSCSI partitions must be provisioned at static sizes, i.e. partition sizes can not change once a partition is formatted with OCFS2. The
challenge with supporting volume management with OCFS2 is that the volume manager needs to be cluster-aware and integrated
with the OCFS2 cluster stack. To date there are no supported volume management solutions for OCFS2. For example, many
customers use LVM to manage the back-end block storage for OCFS2. LVM is not cluster aware, so changes made to the back-end
block storage by LVM will not be propagated to the OCFS2 file system. The Oracle VM pool members would continue to write to the
old volume layout, and corruption will occur.
SAN connectivity is configured using fiber channel HBAs with dm-multipath in dom0 to allow the Oracle VM server to access a Logical
Unit (LU) using multiple paths. Oracle VM also supports boot from SAN. The "linux mpath" install option is used to boot an Oracle VM
server from a SAN. By using the "linux mpath" install option, the installer will see the multipath devices and allow you to create the
boot/root partitions, along with the master boot record (MBR) on the SAN. Please note that this document will not cover boot from
SAN.
To connect an Oracle VM server to a Fibre Channel storage array, each Oracle VM server’s HBAs must be zoned and masked to the
storage. Once the HBAs are zoned and masked, the next step is to configure dm-multipath to detect the LUNs which are recognized as
multipath devices. Once the multipath devices are detected, we need to format the devices on the “pool master” using the mkfs.ocfs2
utility. Next, use the repos.py script to configure the storage repository. Finally, create a pool using Oracle VM Manager or the Oracle
VM Management Pack by selecting the pool master server. Once the pool is created add all the other Oracle VM servers to the pool.
SAN Back-end Storage Prerequisites:
Create the LUN(s)

The HBAs must be zoned and masked to the storage.
SAN Oracle VM Server Prerequisites:

1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the storage configurations will
not be hampered by unpatched bugs.
2. Select an Oracle VM server that will be used as the Oracle VM pool master. After the Oracle VM pool master and all the other
Oracle VM pool members meet the prerequisites outlined in the following steps, access Oracle VM Manager and create an HA
enabled pool using the Oracle VM pool master server.
Note: An HA enabled pool automatically mounts and links root and extended storage repositories for each Oracle VM pool member
that is added to a pool.
3. Create a multipath.conf file for the storage array. Please refer to Appendix A for examples multipath.conf files.
server myntp1.com
server myntp2.com
# ntpstat

at stratum 4 time correct to within 54 ms
146 of 243 18.04.2011 16:35

# host server2
server2 has address
192.168.4.6
# host 192.168.4.6
name pointer
server2

localhost
servername
# iptables -L

ACCEPT)


COMMIT
8. If an Oracle VM server was originally installed with a local ocfs2 storage repository, it is necessary to remove and unmount the
local ocfs2 storage repository before adding the Oracle VM server to a pool. To determine if an Oracle VM server is using a local
storage repository type "/opt/ovs-agent-2.3/utils/repos.py -l" to list all configured storage repositories. If a storage repository is listed,
type "/opt/ovs-agent-2.3/utils/repos.py -d UUID" to remove the local repository from the Oracle VM server.
Next, check if the local storage repository is still mounted under /var/ovs/mount/UUID. Type “mount |grep mount”, as shown in the
next example to list the mounts.
# mount |grep mount

/dev/sda3 on /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D type ocfs2 (rw,heartbeat=none)
The above example shows that a storage repository /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D is still mounted under
/dev/sda3. Next, unmount the OCFS2 repository by typing “umount /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D” as
# umount /var/ovs/mount
147 of 243 18.04.2011 16:35

/62C757BA5E174DF7B5AB01BBAE0F765D
Tip: A default Oracle VM server installation dedicates the majority of the local disk to the OVS partition and create a small root
partition. If your Oracle VM server was installed with the default OVS partition with a small root partition, consider rebuilding the
server to create a disk layout that allocates the disk space to the root partition.
Type “df -h” to list the size of an Oracle VM servers' partitions.
Another consideration for small roots that include /var is the potential for large saves from the xendomains service. If an Oracle VM
server crashes, the xendomains service will save the state (the memory foot print of each guest) of all running guests in the /var/lib
/xen/save directory, which could fill up a small root partition. If xendomains functionality is not needed disable it. The next example
shows how to disable or edit the location of the saved xendomains files.
1. Edit /etc/sysconfig/xendomains
2. find the section:
## Type: string
## Default: /var/lib/xen/save
#
# Directory to save running domains to when the system (dom0) is
# shut down. Will also be used to restore domains from if # XENDOMAINS_RESTORE
# is set (see below). Leave empty to disable domain saving on shutdown
# (e.g. because you rather shut domains down).
# If domain saving does succeed, SHUTDOWN will not be executed.
#
XENDOMAINS_SAVE=/var/lib/xen/save
3. Clear the XENDOMAINS_SAVE path to disable saves. Or point the XENDOMAINS_SAVE path to a partition with available space.
4. Rest the xendomains services by typing “service xendomains restart” to enable a new configuration.
9. If an Oracle VM server have previously been added to an Oracle VM server pool, the Oracle VM server's cluster configurations will
need to be cleaned before added to a new Oracle VM server pool. To clean an Oracle VM server's cluster configurations it is necessary
to a) empty the /etc/ocfs/cluster.conf file b) delete and recreate the local BerkleyDB and c) run the cleanup.py script to stop o2cb
heartbeat, offline o2cb, remove o2cb configuration file, umount ovs-agent storage repositories and to cleanup ovs-agent local
database.

/cluster.conf
local BerkleyDB.
# /opt/ovs-agent-2.3/utils/cleanup.py
ovs-agent.

*) offline o2cb
repositories

Cleanup done.
Step 1: The first step is to validate that the HBAs are listed in the /sys/class/fc_host directory. The goal of this step is to record the
host adapter ID number(s) and to troubleshoot any SAN connectivity issues. You can skip this step if you’re able to view the HBAs
listed in the /sys/class/fc_host directory.
Tip: If there are no host adapters listed in the /sys/class/fc_host directory, check if the HBAs are properly zoned and masked.
As shown in the next example, type “ll /sys/class/fc_host” to list the host adapters.
# ll /sys/class/fc_host
total 0
drwxr-xr-x 3 root root 0 Oct 11 08:24 host6
148 of 243 18.04.2011 16:35

drwxr-xr-x 3 root root 0 Jul 5 09:28 host7
The output from “ll /sys/class/fc_host” shows that there are two host adapters, host6 and host7.
Once you’re able to list the host adapters in the /sys/class/fc_host directory, cat each host adapter’s “port_name” file to get the host
adapter number ID number.
#cat /sys/class/fc_host/host6/port_name
0x10000000c0ffee7e
The above example shows that 0x10000000c0ffee7e is the host adapter number ID number for host6.
Next, cat the host7/port_name file to get the host adapter number ID number.
cat /sys/class/fc_host/host7/port_name
0x10000000c0ffee7f
The above example shows that 0x10000000c0ffee7f is the host adapter number ID number for host7.
If you need to rescan the bus, echo the “/sys” filesystem as shown in the next examples.
#echo “- - -” > /sys/class/scsi_host/hostH/scan
For example.
#echo “- - -” > /sys/class/scsi_host/host6/scan

#echo “- - -” > /sys/class/scsi_host/host7/scan
We have successfully discovered the host adapter ID numbers from each Oracle VM server.
Step 2: Next, validate that multipath daemon is properly configured on each Oracle VM server. From dom0, type “service multipathd
status”, as shown in the following example.
#service multipathd status

multipathd (pid 10333) is running...
If your multipath daemon is running, please skip to Step 3.
If your system’s multipath daemon is stopped, use chkconfig to configure the multipath daemon, as shown in the below example.
Next, type “chkconfig --list multipathd” to view the multipath daemon configuration, as shown in the next example.
#chkconfig --list multipathd

multipathd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
The output of “chkconfig --list multipathd” shows that the multipath daemon is not configured to run at any system run level i.e. run
level 0 through 6.
Next, type “chkconfig multipathd on” in order to automatically start the multipath daemon at run level 3, 4 and 5, as shown in the
next example.
#chkconfig multipathd on
Next, validate the multipathd startup configuration by typing “chkconfig --list multipathd” as shown in the next example.
# chkconfig --list multipathd

multipathd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
The output of “chkconfig --list multipathd” validates that the multipath daemon is configured to run at run level 2, 3, 4, and 5.
Finally, start the multipath daemon by typing “service multipathd start”, as shown in the following example.
# service multipathd start

Starting multipathd daemon: [ OK ]
We have successfully configured and started the multipath daemon on each Oracle VM server.
149 of 243 18.04.2011 16:35

Step 3: Next, we will configure dm-multipath on each Oracle VM server by replacing or modifying the default /etc/multipath.conf file
with a multipath.conf file crafted for your storage solution. multipath.conf file settings can be vendor specific, please check with your
storage vendor for a multipath.conf file for Oracle VM or RHEL 5U3. If you already have a working multipath.conf file please skip to
Step 4.
Each Oracle VM server has an example multipath.conf file located at /etc/multipath.conf. The example multipath.conf file should be
modified or replaced with a vendor specific multipath.conf file to support your storage array. A multipath.conf file is divided into four
sections. List 4 shows the four sections of a multipath.conf file.
1. blacklist
2. defaults
3. multipaths
4. devices
Next, we will review the four sections of a multipath.conf file.
Blacklist
The blacklist section lists devices that are to be excluded from multipath control. For example, if the server boots from a local disks
i.e. sda, sdb, hda, hdb, etc…then we need to include those disks in the blacklist. The example multipath.conf file has two blacklist
entries.
As shown below, the first blacklist entry is uncommented and will blacklist all devices.
blacklist {
devnode "*"
}
If you are going to test the example multipath.conf file, comment the blacklist entry to allow devices to be managed by dm-multipath.
The next example shows the blacklist entries commented.
#blacklist {
# devnode "*"
#}
The second blacklist entry is commented and shows how to blacklist WWIDs, ram, raw, loop, fd, md, dm-, sr, scd, st, and hd devices.
The next example shows the second black list entry.
#blacklist {
# wwid 26353900f02796769
# devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
# devnode "^hd[a-z]"
#}
The first entry wwid 26353900f02796769 is an example that shows how to blacklist WWID. If you need to blacklist WWIDs, add an
entry for each WWID, for example.
wwid "3600508b40008dc480000500000670000"
wwid "3600508b40008dc480000500000640000"
wwid "3600508b40008dc480000500000610000"
The second devnode line devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" blacklists ram, raw, loop, fd, md, dm-, sr, scd, st,
and hd devices.
The third devnode entry devnode "^hd[a-z]" will blacklist hda disks.
To test the default blacklist entry, first comment out the blacklist entry that blacklist all devices. Next, remove the wwid line, then
uncomment the blacklist, devnode and the } sections, as shown below.
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
}
You will need to restart the multipath daemon to test any new settings. The next example shows how to restart the multipath daemon.
# service multipathd restart

Stopping multipathd daemon: [ OK ]
150 of 243 18.04.2011 16:35

If your Oracle VM servers use a serial or SCSI controller for local disks then the interface names will be similar to sda, sdb, etc. To
exclude the local disks i.e. sda and sdb you need to add a devnode entry for the devices. The example devnode entry below will
blacklist the sda and the sdb device.
blacklist {
devnode "^hd[a-z]"
devnode "^sd[a-b]$"
}
Defaults
The defaults section allows you to configure default settings for dm-multipath that “may” be supported by your storage array. If you
determined that your storage array does not use the settings in the defaults section, use the devices section of the multipath.conf file.
The defaults section settings are overwritten when the devices and multipaths sections are used. The default “defaults” setting in the
example multipath.conf uses the “user_friendly_names yes” entry. The “user_friendly_names yes” entry will allow you to use an alias,
instead of WWID (World Wide Identifier) names.
Multipath devices can be identified by a WWID names or by an alias. A WWID is a unique identifier for the multipath device that does
not change. Device names such as /dev/sdx and /dev/dm-x can change on reboot, so defining multipath devices by their ID is
preferred. The multipath device names in the /dev/mapper directory references LUN IDs that do not change and are user friendly, i.e.
mpath0, mpath1, etc…
Multipaths
The multipaths section is where you map devices to a user friendly name. Each multipath entry will specify the UUID or wwid and the
alias of a LUN along with path_checker variables, which will regularly check the path. The settings in the multipaths section overwrite
the settings specified in the defaults and devices sections.
Devices
The devices section is used to define vendor specific settings. Consult your storage vendor for the entries for your storage array. If you
are using multiple SAN storage systems, several device entries are necessary.
After changing settings in a multipath.conf file, administrators must restart the dm multipath daemon by typing service multipathd
restart, as shown in the next example.
# service multipathd restart

Stopping multipathd daemon: [ OK ]
To generate detailed return messages, administrators can type “multipath -ll”. The “multipath -ll” command will list all LUNs by
WWID with their multipath device names and the individual paths used to create the multipath.
The next example shows Oracle VM’s default multipath.conf file.
# This is a basic configuration file with some examples, for device mapper
# multipath.
# For a complete list of the default configuration values, see
# /usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.defaults
# For a list of configuration options with descriptions, see
# /usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.annotated
# Blacklist all devices by default. Remove this to enable multipathing

# on the default devices.
blacklist {
devnode "*"
}
## By default, devices with vendor = "IBM" and product = "S/390.*" are

## blacklisted. To enable multipathing on these devices, uncomment the
## following lines.
#blacklist_exceptions {
# device {
# vendor "IBM"
# product "S/390.*"
# }
#}
## Use user friendly names, instead of using WWIDs as names.

defaults {
user_friendly_names yes
151 of 243 18.04.2011 16:35

}
##
## Here is an example of how to configure some standard options.
##
#
#defaults {
# udev_dir /dev
# polling_interval 10
# selector "round-robin 0"
# path_grouping_policy multibus
# getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
# prio_callout /bin/true
# path_checker readsector0
# rr_min_io 100
# max_fds 8192
# rr_weight priorities
# failback immediate
# no_path_retry fail
# user_friendly_names yes
#}
##
## The wwid line in the following blacklist section is shown as an example
## of how to blacklist devices by wwid. The 2 devnode lines are the
## compiled in default blacklist. If you want to blacklist entire types
## of devices, such as all scsi devices, you should use a devnode line.
## However, if you want to blacklist specific devices, you should use
## a wwid line. Since there is no guarantee that a specific device will
## not change names on reboot (from /dev/sda to /dev/sdb for example)
## devnode lines are not recommended for blacklisting specific devices.
##
#blacklist {
# wwid 26353900f02796769
# devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
# devnode "^hd[a-z]"
#}
#multipaths {
# multipath {
# wwid 3600508b4000156d700012000000b0000
# alias yellow
# path_selector "round-robin 0"
# failback manual
# no_path_retry 5
# }
# multipath {
# wwid 1DEC_____321816758474
# alias red
# }
#}
#devices {
# device {
# vendor "COMPAQ "
# product "HSV110 (C)COMPAQ"
# getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
# path_selector "round-robin 0"
# hardware_handler "0"
# failback 15
# no_path_retry queue
# }
# device {
# vendor "COMPAQ "
# product "MSA1000 "
# }
#}
Note: If you do not have a working multipath.conf file please reference Appendix A for example multipath.conf files.
152 of 243 18.04.2011 16:35

Once you have a working multipath.conf file and have restarted the multipath daemon you can list the mapped devices in the
/dev/mapper directory. Next we will access dom0 as root and type “ll /dev/mapper” to view the mapped devices, as shown in the
following example.
# ll /dev/mapper
total 0
crw------- 1 root root 10, 62 Jan 23 16:44 control
brw-rw---- 1 root disk 253, 0 Jan 23 17:01 mpath0
The "mpath0" entry validates that the mapped device is available from dom0.
We can also list the mapped devices with their major and minor numbers by typing “dmsetup ls”. The minor numbers corresponds
with the dm device name. In the following example the minor number of 0 corresponds to the multipath device /dev/dm-0.
#dmsetup ls
mpath0 (253, 0)
We can also validate the mapped devices with the corresponding multipath devices by listing the /dev/mpath/ directory as shown in
the next example.
# ll /dev/mpath/
total 0
lrwxrwxrwx 1 root root 7 Jan 23 17:01 mpath0 -> ../dm-0
To list all the storage devices and the available paths type “multipath –l”, as shown in the next example.
# multipath -l
mpath0 (2000b080000002369) dm-0 Pillar,Axiom 600
[size=603G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
\_ 7:0:0:0 sdb 8:16 [active][undef]
\_ 7:0:1:0 sdc 8:32 [active][undef]
\_ 8:0:0:0 sdd 8:48 [active][undef]
The "multipath -l" command queries sysfs and the device mapper only, it does not invoke path checkers. The "multipath -ll" gets
information from all relevant sources, including path checkers.
In Step 3 we reviewed the syntax of a multipath.conf file and showed how to restart the multipath daemon to view the mapped
devices, and the storage devices, along with the available paths.
Step 4: Next we will create an OCFS2 storage repository on a LUN or LUNs from “one” Oracle VM server. We will format the OCSF
partition from the pool master.
The next example shows the syntax to format an OCFS2 volume.
#mkfs.ocfs2 -L mylabel -Tdatafiles -N8 <device>
In the above example, the mkfs.ocfs2 utility is used to format the device. The “-L” parameter is optional and can be used to add a
descriptive label to the OCFS2 volume. The “-Tdatafiles” parameter makes mkfs.ocfs2 choose the optimal filesystem parameters for
the device. The -N parameter selects the number of slots. The number of slots determines the number of pool members that can
concurrently mount the OCFS2 volume. The OCFS2 file system can support up to 255 nodes. For example, if your Oracle VM server
pool will have 20 pool members, select -N20. The slot number can later be increased or decreased using the tunefs.ocfs2 utility.
Next, from dom0, format an OCFS2 volume by typing “mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/mapper/mpath0”, as shown in the
next example.
Substitute root-sr with your desired label name and /dev/mapper/mpath0 with the proper device path for your environment.
# mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/mapper

/mpath0
mkfs.ocfs2 1.4.3
Cluster stack: classic o2cb
Filesystem Type of datafiles
Filesystem label=root-sr
Block size=4096 (bits=12)
Cluster size=1048576 (bits=20)
Volume size=497142464512 (474112 clusters)
(121372672 blocks)
15 cluster groups (tail covers 22528 clusters, rest cover
153 of 243 18.04.2011 16:35

32256 clusters)
Journal size=33554432
Initial number of node slots: 16
Creating bitmaps: done
Initializing superblock: done
Writing system files: done
Writing superblock: done
Writing backup superblock: 5 block(s)
After the OCFS2 volume has been formatted you can use the mounted.ocfs2 utility to detect and list the OCFS2 volume.
To detect and list the OCFS2 volume from the above example, type “mounted.ocfs2 –d” from dom0, as shown in the next example.
# mounted.ocfs2 -d
Device FS UUID Label
/dev/mapper/mpath0 ocfs2 35c601e9-b0da-4950-9a90-ec0193baa205 root-sr
The above mounted.ocfs2 -d example lists the device name, the file system type, the UUID and the label.
You can also use the full (-f) mode to lists the status of each OCFS2 volume.
# mounted.ocfs2 -f
Device FS Nodes
/dev/mapper/mpath0 ocfs2 Not mounted
The above mounted.ocfs2 -f example lists the device name, the file system type, and the status of the node.
Note: Sparse files and unwritten extents are activated by default when using Oracle VM 2.2’ mkfs.ocfs2 utility. If your system was
upgraded from 2.1 to 2.2, it’s necessary to enable sparse files and unwritten extents using the following procedure.
# umount <device>
# tunefs.ocfs2 --fs-features=sparse,unwritten <device>
To validate the enabled OCSF2 features, type “tunefs.ocfs2 -Q "%M %H %O\n" <device>” as shown in the next example.
# tunefs.ocfs2 -Q "%M %H %O\n" <device>

backup-super strict-journal-super sparse
inline-data unwritten
We have successfully formatted the /dev/mpath0 device with OCSF2 using the mkfs.ocfs2 utility as well as reviewed mounted.ocfs2 –d,
mounted.ocfs2 -f and how to list the OCFS2 features using tunefs.ocfs2.
Step 5: Next, on the pool master configure the root repository using the repos script. After the pool master is configured, configure
all other pool members.
1. From dom0 type "/opt/ovs-agent-2.3/utils/repos.py -l" to list any configured storage repositories, as shown in the next example.
/repos.py -l
#
Typing "/opt/ovs-agent-2.3/utils/repos.py -l" should result with an empty entry. If a storage repository is listed, type "/opt/ovs-agent-
2.3/utils/repos.py -d UUID" to remove the repository. Next, unmout the storage repository in /var/ovs/mount/UUID by typing “umount
/var/ovs/mount/UUID”.
2. Next, type "/opt/ovs-agent-2.3/utils/repos.py -n /dev/mpath0" to add the new device to the list of managed devices, as shown in
the next example. Substitute /dev/sdb for the correct device path for your environment.
# /opt/ovs-agent-2.3/utils/repos.py -n /dev/mapper/mpath0
[ NEW ] 002463a4-8998-4423-a797-8a1544739409 => /dev/mapper/mpath0
3. Next, type "/opt/ovs-agent-2.3/utils/repos.py -r UUID" to tag the storage repository as the root storage repository, as shown in
the next example.
# /opt/ovs-agent-2.3/utils/repos.py -r 002463a4-8998-4423-
a797-8a1544739409
[ R ] 002463a4-8998-4423-a797-8a1544739409 => /dev/mapper
/mpath0
154 of 243 18.04.2011 16:35

Note: The UUID will be listed in step 2 or you can list the UUID by typing repos.py -l.
4. Next, "only" on the pool master type /opt/ovs-agent-2.3/utils/repos.py -i to mount the root storage repository, as shown in the
next example. This step only needs to be performed on the pool master.
# /opt/ovs-agent-2.3/utils/repos.py -i
Note: When repos.py -i is run, the new storage repository will be mounted under /var/ovs/mount/UUID, although the new storage
repository will not be linked to /OVS. The Oracle VM agent is responsible for mounting and linking storage repositories for pool
members.
Next, validate that the storage repository has been mounted by typing “mounted.ocfs2 -f“, as shown in the next example.
# mounted.ocfs2 -f
Device FS Nodes
/dev//mapper/mpath0 ocfs2 ovs1.sf.itnc.com
You can also validate the ocfs2 mounts by typing mount|grep ocfs2, as shown in the next example.
# mount|grep ocfs2
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw)
/dev/mapper/mpath0 on /var/ovs/mount
/A85D7145957842F988293FDA43F8754D type ocfs2
(rw,_netdev,heartbeat=local)
Typing df –h would also validate that the root storage repository is mounted, as shown in the next example.
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 451G 961M 426G 1% /
/dev/sda1 99M 46M 48M 49% /boot
tmpfs 285M 0 285M 0% /dev/shm
/dev/mapper/mpath0 463G 541M 463G 1% /var/ovs/mount/A33BF3D2E09B45F3931F830CB1A404AA
5. Next, create the pool in Oracle VM Manager. When creating the pool select the Oracle VM server that was used to format the
OCFS2 file system as the pool master.
6. Next, add all of the other configured Oracle VM servers to the pool.
Adding an Extended SAN Storage Repository

To add storage to an Oracle VM storage repository, the first step is to provision the storage. The HBAs must be zoned and masked to
the storage to be able to use the LUNs. Next, connect the pool master and format the storage using the steps outlined in Step 4.
Subsequently, connect the other pool member to the storage using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new)
followed by the -i (initialize, aka mount) switches, to add and then mount the sub storage repository. Finally, the new mount point in
/var/ovs/mount/UUID needs to be linked to /OVS/UUID, by typing “ln -nsf /var/ovs/mount/<UUID>/ /OVS”, again from dom0. The end
result is a root repository with an “extended” sub repository mounted under /var/ovs/mount/UUID which is linked to /OVS/UUID.
Once a pool is configured, the Oracle VM agent will automatically place resources such as virtual machines, templates, or ISO files on
the storage repository with available space. The Oracle VM agent is also responsible for mounting and linking storage repositories.
iSCSI Storage Repository Configuration

This section will review how to configure a root and an extended iSCSI storage repository with Oracle VM 2.2. Network interface
bonding as well as dm-multipath may be used with iSCSI storage to provide multiple path support with Oracle VM.
iSCSI Back-end Storage Prerequisites:
Create the LUN(s)

Create the masking rules
iSCSI Oracle VM Server Prerequisites:

1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the storage configurations will not be hampered by
unpatched bugs.
2. Select an Oracle VM server that will be used as the Oracle VM pool master. After the Oracle VM pool master and all the other Oracle VM pool
members meet the prerequisites outlined in the following steps, access Oracle VM Manager and create an HA enabled pool using the Oracle VM pool
master server.
Note: An HA enabled pool automatically mounts and links root and extended storage repositories for each Oracle VM pool member that is added to a
pool.
155 of 243 18.04.2011 16:35

First, open the “/etc/ntp.conf” file by typing “vi /etc/ntp.conf” and validate that at least two available NTP servers entries are listed. The next example
shows two bold NTP server entries in an ntp.conf file.
server myntp1.com
server myntp2.com
Next, type "ntpstat" on each Oracle VM server to validate the NTP configuration. The next example shows the output from typing the ntpstat command
on an Oracle VM server that has its time synchronized to an NTP server with the IP address of 192.168.4.251.
# ntpstat
synchronized to NTP server (192.168.4.251) at

stratum 4 time correct to within 54 ms polling
server every 1024 s
Finally, validate that the time, date and time zone on each Oracle VM server as well as on the Oracle VM Manager host is synchronized by typing the
"date" command.
First, open the “/etc/resolv.conf” file by typing “vi /etc/resolv.conf” and validate that two available DNS servers are listed. The next example shows two
DNS servers listed in a resolve.conf file.
Next, validate the forward and reverse lookups for each Oracle VM pool member and the Oracle VM Manager host using the “host” command. For
example, to validate server2's forward lookup from server1 type “host server2” as shown in the next example.
# host server2
server2 has address
192.168.4.6
# host 192.168.4.6
6.4.168.192.in-addr.arpa domain name
pointer
server2
5. The Oracle VM server’s host name in the /etc/hosts file must be associated with the Oracle VM server's public IP address. If an Oracle VM pool
member's host name is associated with 127.0.0.1, the cluster.conf file will be malformed and the Oracle VM pool will not be operational. The next
example shows the improper syntax from an Oracle VM server's hosts file entry.

127.0.0.1 localhost.localdomain localhost

6. ocfs2 network connectivity between all Oracle VM server pool members must be operational before creating a multiple server pool. Check the ocfs2
network connectivity between all Oracle VM pool members by typing "nc -zv <myoraclevmserver1> 7777". For example, if you have two Oracle VM
servers named ovs1 and ovs2, from ovs1 type "nc -zv ovs2 7777". Typing "nc -zv ovs2 7777" from ovs1 should return "succeeded!". If you receive a
"failed: Connection refused" message between any Oracle VM servers, something (firewall, switch, router, cable, etc..) is restricting communication
between the hosts.
The iptables firewall on an Oracle VM server may be blocking the ocfs2 connectivity. If iptables is disabled and allowing all connections, the output from
typing “iptables -L” will look like the next example.
# iptables -L
156 of 243 18.04.2011 16:35


Chain FORWARD (policy ACCEPT)


If typing “iptables -L” lists firewall rules, you can a) disable iptables by typing "service iptables stop && chkconfig iptables off" or b) add the following
bold iptables rule to the /etc/sysconfig/iptables file prior to the last line on all Oracle VM pool members.

COMMIT
7. If an Oracle VM server was originally installed with a local ocfs2 storage repository, it is necessary to remove and unmount the local ocfs2 storage
repository before adding the Oracle VM server to a pool. To determine if an Oracle VM server is using a local storage repository type "/opt/ovs-agent-
2.3/utils/repos.py -l" to list all configured storage repositories. If a storage repository is listed, type "/opt/ovs-agent-2.3/utils/repos.py -d UUID" to remove
the local repository from the Oracle VM server.
Next, check if the local storage repository is still mounted under /var/ovs/mount/UUID. Type “mount |grep mount”, as shown in the next example to list
the mounts.
# mount |grep mount

The above example shows that a storage repository /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D is still mounted under /dev/sda3. Next,
unmount the OCFS2 repository by typing “umount /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D” as shown in the next example.
Tip: A default Oracle VM server installation dedicates the majority of the local disk to the OVS partition and create a small root partition. If your Oracle
VM server was installed with the default OVS partition with a small root partition, consider rebuilding the server to create a disk layout that allocates the
disk space to the root partition.
Another consideration for small roots that include /var is the potential for large saves from the xendomains service. If an Oracle VM server crashes, the
xendomains service will save the state (the memory foot print of each guest) of all running guests in the /var/lib/xen/save directory, which could fill up a
small root partition. If xendomains functionality is not needed disable it. The next example shows how to disable or edit the location of the saved
xendomains files.
## Type: string
#
#
database.

/cluster.conf
157 of 243 18.04.2011 16:35

local BerkleyDB.
ovs-agent.

*) offline o2cb
repositories

Cleanup done.
Step 1: The first step is to validate that the iscsi service is running on each Oracle VM server. Access dom0 as root and check the status of the iscsi
service by typing “service iscsi status”, as shown in the next example.
# service iscsi status

iscsid (pid 2314 2313) is running...
If the iscsi service is not running, start the iscsi service by typing “service iscsi start” as shown in the next example.
# service iscsi start
We have successfully validated that the iscsi service is running on each Oracle VM pool member.
Step 2: Next, on each Oracle VM server discover the iSCSI LUNs using the iscsiadm utility. Once the iSCSI LUNs have been
discovered, if necessary, remove any entries that will not be used. Next, we will verify that the unused LUNs are removed.
List 1 shows the procedure to discover, remove and validate iSCSI LUNs.
1. First, from dom0 type “iscsiadm -m discovery -t sendtargets -p iSCSI-Target-IPADDRESS”, to discover the entries from your
iSCSI target. Substitute “iSCSI-Target-IPADDRESS” with the IP address or FQDN of your iSCSI target.
2. Second, remove any unused entries by typing “iscsiadm -m node -p iSCSI Qualified Name -o delete”, for example, iscsiadm -m
node -p 192.168.4.10:3260,1 -T iqn.2006-01.com.openfiler:tsn.a83c0838952c -o delete.
3. Finally, validate that only the desired LUNs are discovered by typing “iscsiadm -m node”.
As shown in the next example, the output from “iscsiadm -m discovery -t sendtargets -p 192.168.4.10” lists two entries. The first entry,
192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.a83c0838952c will be removed. The second entry, 192.168.4.10:3260,1
iqn.2006-01.com.openfiler:tsn.db29e77712c0 will become the root repository.
# iscsiadm -m discovery -t sendtargets -p 192.168.4.10

192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.a83c0838952c
192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.db29e77712c0
Note: Discovered LUNs will appear in /proc/partitions only after restarting the iscsi service.
In general, if your Oracle VM server lists entries that you will not use, you will need to remove all of the entries.
To remove an unused entry, for example type “iqn.2006-01.com.openfiler:tsn.a83c0838952c” type “iscsiadm -m node -p
192.168.4.10:3260,1 -T iqn.2006-01.com.openfiler:tsn.a83c0838952c -o delete
”, as shown in the next example.
#iscsiadm -m node -p 192.168.4.10:3260,1 -T

iqn.2006-01.com.openfiler:tsn.a83c0838952c -o delete
The next example shows how to verify that the unused entry has been removed by typing “iscsiadm -m node".
# iscsiadm -m node
192.168.4.10:3260,1
iqn.2006-01.com.openfiler:tsn.aa231ffd6ef2
As shown in the above example only one entry is listed.
158 of 243 18.04.2011 16:35

In Step 2 we reviewed how to discover, remove and validate iSCSI LUNs.
Step 3: Next, on each Oracle VM server list /proc/partitions to review each Oracle VM server’s devices. After we review the
devices listed in /proc/partitions, restart the iscsi service to mount the discovered iSCSI LUN. After the iSCSI service is restarted, the
iSCSI LUN will be listed in /proc/partitions as a new device, i.e. sdb.
Before we restart the iscsi service, review the devices listed on each Oracle VM server by typing “cat /proc/partitions”, as shown in
the next example.
# cat /proc/partitions
major
minor #blocks name
8 0 488386584 sda
8 1 104391 sda1
8 2 487227352
sda2
8 3 1052257
sda3
Note the sda, sda1, sda2 and the sda3 devices.
Next, type “service iscsi restart” to restart the iscsi service, which also mounts the discoved LUN.
# service iscsi restart

Stopping iSCSI daemon:
iscsid dead but pid file exists [ OK ]
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
[ OK ]
Setting up iSCSI targets: Logging in to [iface: default, target:
iqn.2006-01.com.openfiler:tsn.db29e77712c0, portal:
192.168.4.10,3260]
Login to [iface: default, target:
iqn.2006-01.com.openfiler:tsn.db29e77712c0, portal:
192.168.4.10,3260]: successful
[ OK ]
#
After the iscsi service is restarted, any discovered iSCSI LUNs will be listed in /proc/partitions as a new device, as shown in the next
example.
# cat /proc/partitions
major
minor #blocks name
8 0 488386584 sda
8 1 104391 sda1
8 2 487227352
sda2
8 3 1052257
sda3
8 16 485490688
sdb
Note that the LUN i.e. the sdb device is listed in /proc/partitions. Now the new device can be partitioned using the mkfs.ocfs2 utility.
In Step 3 we reviewed /proc/partitions on each Oracle VM server. Next, we restarted the iscsi service which mounted the discovered
iSCSI LUN. After the iSCSI LUN was mounted, we validated that the new device was listed in /proc/partitions.
Step 4: Next, “only on the pool master” you have to format an OCFS2 volume on the new device (the iSCSI LUN), using the
mkfs.ocfs2 utility. The OCSF2 volume should be formatted “only” from one Oracle VM server, i.e. the pool master server.
Note: If you already have created a server pool, format the OCSF2 volume “only” on the server pool master server. If you have not
created a server pool in Oracle VM manager, use the Oracle VM server that you will select as the pool master to format the OCFS2
volume.
The next example shows the syntax to format an OCFS2 volume.
#mkfs.ocfs2 -L mylabel -Tdatafiles -N8

/dev/sdx
159 of 243 18.04.2011 16:35

In the above example, the mkfs.ocfs2 utility is used to format the device. The “-L” parameter is optional and can be used to add a
descriptive label to the OCFS2 volume. The “-Tdatafiles” parameter makes mkfs.ocfs2 chose the optimal filesystem parameters for the
device. The -N parameter selects the number of slots. The number of slots determines the number of pool members that can
concurrently mount the OCFS2 volume. The OCFS2 file system can support up to 255 nodes. For example, if your Oracle VM server
pool will have 20 pool members, select -N20. The slot number can later be increased or decreased using the tunefs.ocfs2 utility. Next,
from dom0, format an OCFS2 volume by typing “mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/sdb”, as shown in the next example.
Substitute root-sr with your desired label name and /dev/sdb with the proper device path for your enviroment.
# mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/sdb

mkfs.ocfs2 1.4.3
Cluster stack: classic o2cb
Filesystem Type of datafiles
Filesystem label=root-sr
Block size=4096 (bits=12)
Cluster size=1048576 (bits=20)
Volume size=497142464512 (474112 clusters)
(121372672 blocks)
15 cluster groups (tail covers 22528 clusters, rest cover
32256 clusters)
Journal size=33554432
Initial number of node slots: 16
Creating bitmaps: done
Initializing superblock: done
Writing system files: done
Writing superblock: done
Writing backup superblock: 5 block(s)
After the OCFS2 volume has been formatted you can use the mounted.ocfs2 utility to detect and list the OCFS2 volume.
To detect and list the OCFS2 volume from the above example, type “mounted.ocfs2 –d” from dom0, as shown in the next example.
# mounted.ocfs2 -d
Device FS UUID
Label
/dev/sdb ocfs2 35c601e9-b0da-4950-9a90-
ec0193baa205 root-sr
The above mounted.ocfs2 -d example lists the device name, the file system type, the UUID and the label.
You can also use the full (-f) mode to lists the status of each OCFS2 volume.
# mounted.ocfs2 -f
Device FS Nodes
/dev/sdb ocfs2 Not
mounted
The above mounted.ocfs2 -f example lists the device name, the file system type, and the status of the node.
Note: Sparse files and unwritten extents are activated by default when using Oracle VM 2.2’ mkfs.ocfs2 utility. If your system was
upgraded from 2.1 to 2.2, it’s necessary to enable sparse files and unwritten extents using the following procedure.
# umount <device>
# tunefs.ocfs2 --fs-features=sparse,unwritten <device>
To validate the enabled OCSF2 features, type “tunefs.ocfs2 -Q "%M %H %O\n" <device>” as shown in the next example.
# tunefs.ocfs2 -Q "%M %H %O\n" <device>

backup-super strict-journal-super sparse
inline-data unwritten
We have successfully formatted the /dev/sdb volume with OCSF2 using the mkfs.ocfs2 utility as well as reviewed mounted.ocfs2 –d,
mounted.ocfs2 -f and how to list the OCFS2 features using tunefs.ocfs2.
Step 5: Next, on the pool master configure the root repository using the repos script. After the pool master is configured, configure
all other pool members.
1. From dom0 type "/opt/ovs-agent-2.3/utils/repos.py -l" in order to list any configured storage repositories, as shown in the next
example.
160 of 243 18.04.2011 16:35

/repos.py -l
#
2. Next, type "/opt/ovs-agent-2.3/utils/repos.py -n /dev/sdb" to add the new device to the list of managed devices, as shown in the
next example. Substitute /dev/sdb for the correct device path for your environment.
# /opt/ovs-agent-2.3/utils/repos.py -n /dev/sdb
[ NEW ] 002463a4-8998-4423-a797-8a1544739409
=> /dev/sdb
the next example.
# /opt/ovs-agent-2.3/utils/repos.py -r 002463a4-8998-4423-
a797-8a1544739409
[ R ] 002463a4-8998-4423-a797-8a1544739409 => /dev/sdb
4. Next, "only" on the pool master type /opt/ovs-agent-2.3/utils/repos.py -i to mount the root storage repository, as shown in the
next example. This step only needs to be performed on the pool master.
repository will not be linked to /OVS.
Next, validate that the storage repository has been mounted by typing “mounted.ocfs2 -f“, as shown in the next example.
# mounted.ocfs2 -f
Device FS Nodes
/dev/sdb
ocfs2 ovs2.sf.itnc.com
You can also validate the ocfs2 mounts by typing mount|grep ocfs2, as shown in the next example.
# mount|grep ocfs2
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw)
/dev/sdb on /var/ovs/mount/A85D7145957842F988293FDA43F8754D
type ocfs2 (rw,_netdev,heartbeat=local)
Typing df –h would also validate that the root storage repository is mounted, as shown in the next example.
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 451G 961M 426G 1% /
/dev/sda1 99M 46M 48M 49% /boot
tmpfs 285M 0 285M 0% /dev/shm
/dev/sdb 463G 541M 463G 1% /var/ovs/mount
/A33BF3D2E09B45F3931F830CB1A404AA
Adding an iSCSI Extended Storage Repository

To add storage to an Oracle VM storage repository, the first step is to provision the storage. The storage will need to be zoned and
masked to be able to use the LUNs.
Next, discover the new device by typing “iscsiadm -m node -T target --rescan”.
Note: If you restart the iscsi service to detect the new LUN, the iscsi service will log out of the existing storage repository, discover
the new LUN and reboot the server. To avoid a reboot of your Oracle VM server use the rescan option with the iscsiadm utility, i.e.
161 of 243 18.04.2011 16:35

“iscsiadm -m node -T target --rescan”.
Next, connect the pool master and format the storage using the steps outlined in Step 4. Then, connect the other pool member to the
storage using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new) followed by the -i (initialize, aka mount) switches, to add
and then mount the sub storage repository. Finally, the new mount point in /var/ovs/mount/UUID needs to be linked to /OVS/UUID, by
typing “ln -nsf /var/ovs/mount/<UUID>/ /OVS”, again from dom0. The end result is a root repository with an “extended” sub
repository mounted under /var/ovs/mount/UUID which is linked to /OVS/UUID.
Once a pool is configured, the Oracle VM agent will automatically place resources such as virtual machines, templates, or ISO files on
the storage repository with available space. The Oracle VM agent is also responsible for mounting and linking storage repositories.
NFS Storage Repository Configuration

This section will review how to configure a root and an extended NFS storage repository with Oracle VM 2.2.
NFS Back-end Storage Prerequisites:
Create the LUN(s)

Create the masking rules
The NFS share must have the “no_root_squash” option enabled.
NFS Oracle VM Server Prerequisites:
1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the storage configurations will
not be hampered by unpatched bugs.
2. Select an Oracle VM server that will be used as the Oracle VM pool master. After the Oracle VM pool master and all the other
Oracle VM pool members meet the prerequisites outlined in the following steps, access Oracle VM Manager and create an HA
enabled pool using the Oracle VM pool master server.
Note: An HA enabled pool automatically mounts and links root and extended storage repositories for each Oracle VM pool member
that is added to a pool.
server myntp1.com
server myntp2.com
# ntpstat

at stratum 4 time correct to within 54 ms
# host server2
server2 has address
192.168.4.6
162 of 243 18.04.2011 16:35

# host 192.168.4.6
name pointer
server2

localhost
servername
# iptables -L

ACCEPT)


COMMIT
7. If an Oracle VM server was originally installed with a local ocfs2 storage repository, it is necessary to remove and unmount the
local ocfs2 storage repository before adding the Oracle VM server to a pool. To determine if an Oracle VM server is using a local
storage repository type "/opt/ovs-agent-2.3/utils/repos.py -l" to list all configured storage repositories. If a storage repository is listed,
type "/opt/ovs-agent-2.3/utils/repos.py -d UUID" to remove the local repository from the Oracle VM server.
Next, check if the local storage repository is still mounted under /var/ovs/mount/UUID. Type “mount |grep mount”, as shown in the
next example to list the mounts.
# mount |grep mount

The above example shows that a storage repository /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D is still mounted under
/dev/sda3. Next, unmount the OCFS2 repository by typing “umount /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D” as
Tip: A default Oracle VM server installation dedicates the majority of the local disk to the OVS partition and create a small root
partition. If your Oracle VM server was installed with the default OVS partition with a small root partition, consider rebuilding the
server to create a disk layout that allocates the disk space to the root partition.
163 of 243 18.04.2011 16:35

Another consideration for small roots that include /var is the potential for large saves from the xendomains service. If an Oracle VM
server crashes, the xendomains service will save the state (the memory foot print of each guest) of all running guests in the /var/lib
/xen/save directory, which could fill up a small root partition. If xendomains functionality is not needed disable it. The next example
shows how to disable or edit the location of the saved xendomains files.
## Type: string
#
#
database.

/cluster.conf
local BerkleyDB.
ovs-agent.

*) offline o2cb
repositories

Cleanup done.
Step 1: On the pool master configure the root repository using the repos script. After the pool master is configured, configure all other pool members.
1. From dom0 type "/opt/ovs-agent-2.3/utils/repos.py -l" to list any configured storage repositories, as shown in the next example.
/repos.py -l
#
2. Next, type "/opt/ovs-agent-2.3/utils/repos.py -n nfsserver:/mnt/vol1/ovs-root/" to add the new share, as shown in the next
example. Substitute nfsserver with the FQDN or IP address of your filer and substitute :/path/to/ovs-root-share with the path to your
NFS share.
164 of 243 18.04.2011 16:35

# /opt/ovs-agent-2.3/utils/repos.py -n 192.168.4.10:/mnt/vg-931
/nfs-root/nfs-sr/
[ NEW ] ef47b8a9-620f-4ed1-aac1-7ac10f4f7fcf =>
192.168.4.10:/mnt/vg-931/nfs-root/nfs-sr/
the next example.
# /opt/ovs-agent-2.3/utils/repos.py -r ef47b8a9-620f-
4ed1-aac1-7ac10f4f7fcf
[ R ] ef47b8a9-620f-4ed1-aac1-7ac10f4f7fcf =>
192.168.4.10:/mnt/vg-931/nfs-root/nfs-sr/
4. Next, type /opt/ovs-agent-2.3/utils/repos.py -i to mount the root storage repository, as shown in the next example. This step only
needs to be performed on the pool master. When Oracle VM pool members are added to a pool the agent will mount and link the root
storage repository.
repository will not be linked to /OVS. Once the pool is created the Oracle VM agent will auto mount and link the storage repository.
Adding an Extended NFS Storage Repository

To add storage to an Oracle VM storage repository, the first step is to provision the storage. The storage needs to be zoned and
masked before it can be added as an extended storage repository. Next, connect the pool master and all the pool members to the
share using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new) as outlined in Step 2. Next, type /opt/ovs-agent-2.3/utils
/repos.py -i mount the sub storage repository. Finally the new mount point in /var/ovs/mount/UUID needs to be linked to /OVS/UUID,
by typing “ln -nsf /var/ovs/mount/<UUID>/ /OVS”, again from dom0. The end result is a root repository with an “extended” sub
repository mounted under /var/ovs/mount/UUID which is linked to /OVS/UUID.
Oracle VM Guest Front-end Storage

This section will review the virtual machine layer of the storage stack. The virtual machine layer is where the storage is presented to
virtual machines as either a flat file, as a LUN, or as a combination of flat files and LUNs. The virtual machine storage layer is referred
to as the guest front-end storage.
The section starts with a review of file-backed block devices and the file-backed block device driver options. The section concludes
with a review of physical backed block devices.
File-backed block device

A file-backed block device uses a flat file in the storage repository as the guest’s primary storage. By default, Oracle VM Manager and
the Oracle VM Management Pack create a file named System.img for each guest. For example, a guest named racnode1 would have a
directory named xxx_racnode1 in the in the /OVS/*_pool/ xxx_racnode1/ directory that contains the System.img file.
By default, Oracle VM Manager and the Oracle VM Management Pack configure guest storage as file-backed block device using the
fast-loopback driver in dom0. You can validate that a guest is using a file-backed block device by looking at the 'disk =' directive in a
guest’s vm.cfg file. Each guest has a vm.cfg file in the /OVS/*_pool/vmname/ directory. A 'file:' reference indicates the use of the
fast-loopback driver.
File-backed block devices can use one of two drivers a) the default fast-loopback driver or b) the blktap driver. In certain
circumstances, the blktap driver may provide better performance than the fast-loopback driver. Oracle VM Manager and the Oracle
VM Management Pack do not support editing the file-backed block device driver settings. To test the blktap driver you must edit the
'file' directive by hand in the desired guest’s vm.cfg file from ‘file’ to 'tap:aio:'.
The next example shows a vm.cfg file from an 11g Oracle VM template that is configured with two virtual disks using file-backed block
devices with the fast-loopback driver. The first of two virtual disks is defined in the 'disk =' directive contains the OS and the second
virtual disk defined in the 'disk =' directive is an ASM disk.
'file:/OVS/running_pool/v52x8611g1/oracle11g_x86_asm.img,xvdb,w',
]
memory = '2048'
name = 'v52x8611g1'
165 of 243 18.04.2011 16:35

uuid = 'f8725f79-c6c8-26d4-a51e-1f32cf010c84'
vcpus = 2
The next example shows the same vm.cfg file from the above 11g Oracle VM template that is configured using file-backed block
devices with the blktap driver. The first of two virtual disks is defined in the 'disk =' directive contains the OS and the second virtual
disk defined in the 'disk =' directive is an ASM disk.
disk = ['tap:aio:/OVS/running_pool/v52x8611g1/System.img,xvda,w',
'tap:aio:/OVS/running_pool/v52x8611g1/oracle11g_x86_asm.img,xvdb,w',
]
memory = '2048'
name = 'v52x8611g1'
vcpus = 2
You can quickly benchmark guest front-end performance by typing “hdparm -tT <device>” within the guest. Replace <device> with
the device you would like to benchmark. Type df to list the devices.
For example, if a guest is using the default file-backed block device with the fast-loopback driver, as root from the guest console type
“hdparm -tT <device>” to gather cached reads and buffered disk reads statistics, as shown in the next example.
# hdparm -tT /dev/xvda2
/dev/xvda2:
Timing cached reads: 26968 MB in 1.99 seconds = 13536.94 MB/sec
Timing buffered disk reads: 148 MB in 3.03 seconds = 48.87 MB/sec
Record the data from “hdparm -tT <device>” and power off the guest. Once the guest is powered of edit the guest’s vm.cfg file and
replace the file directive with tap:aio. Power on the guest and run the same “hdparm -tT <device>” command to gather the cached
reads and the buffered disk reads statistics using the with the blktap driver.
Physical backed block device

The second guest storage option is a physical backed block device. A physical backed block device offers the lowest overhead and best
performance of the two Oracle VM guest storage options. In most cases, a physical backed block device will be the best option for
hight I/O workloads.
For example, Oracle’s certified Oracle VM RAC configuration uses physical backed block devices to provide the best performance for
RAC. To use a physical backed block device, you export a physical block device e.g. a LUN from dom0 to the guest, as a virtual block
device.
As of this writing, Oracle VM Manager or the Oracle VM Management Pack cannot manage physical backed block devices. To use
physical backed block devices with Oracle VM, you need to edit the guest’s vm.cfg file manually to use a physical backed block device.
Note: Oracle VM 2.2 Manager can manage physical multipath devices as Shared Disks.
The next example shows a vm.cfg file that uses physical backed block devices. The first of two disks is defined in the 'disk =' directive,
and contains the OS, the second disk defined in the 'disk =' directive is an ASM disk.
disk = ['phy:/dev/sdu,xvda,w!’, 'phy:/dev/sdv,xvda,w!’
]
memory = '2048'
name = 'v52x8611g1'
vcpus = 2
The next example shows a vm.cfg file from a guest that uses a file backed block device for the OS and eight physical backed block
devices.
166 of 243 18.04.2011 16:35

disk = ['file:/var/ovs/mount/
4C42F6B3FCB841499D595C0CC36D7695/
running_pool/266_lax005112pvm07/System.img,xvda,w',
'phy:/dev/sda,/dev/xvda3,w!',
'phy:/dev/sdb,/dev/xvda4,w!',
'phy:/dev/sdc,/dev/xvda5,w!',
'phy:/dev/sdd,/dev/xvda6,w!',
'phy:/dev/sde,/dev/xvda7,w!',
'phy:/dev/sdf,/dev/xvda8,w!',
'phy:/dev/sdg,/dev/xvda9,w!',
'phy:/dev/sdh,/dev/xvda10,w!',
]
keymap = 'en-us'
memory = '4096'
name = 'oel5u5pv07'
uuid = '4fa8e2f0-4514-5169-467a-7fd64fe62147'
vcpus = 2
vif =
['bridge=xenbr0,mac=00:16:3E:69:E4:04,type=netfront',
'bridge=xenbr1,mac=00:16:3E:61:DE:5E,type=netfront',
]
Configuring a physical backed block device is a manual multi step process.
List 5 shows the steps to configure a physical backed block device.
1. The first step is to create a guest using Oracle VM Manager. Please note, that after the guest is created, the default file backed
block device, i.e. the System.img file can be used or deleted and replaced with a physical backed block device.
2. Provision a disk for the guest, i.e. one or more LUNs. The Oracle VM servers must be zoned and masked to be able to access the
storage.
3. Configure the storage in each dom0. For example, if the guest will run on 4 servers within a pool, the storage must be configured
in dom0 on all four servers.
4. Once the LUN is presented in dom0, export the LUN to the guest by editing the vm.cfg file using a physical backed block device
as show in the above examples.
Appendix A Example multipath.conf Files

Appendix A lists five multipath.conf files from production Oracle VM systems. Appendix A starts with two example multipath.conf files
for EMC CLARiiON and SYMMETRIX, followed with multipath.conf example files for Pillar Data Axiom 600, HP EVA SAN, IBM 2145
SAN and a 3PAR 224MU6 Storage Array.
1. EMC CLARiiON and SYMMETRIX

Example 1 – Quick and Simple
#vi /etc/multipath.conf
devnode_blacklist {
devnode "^hd[a-z]"
}
defaults {
}
2. EMC CLARiiON and SYMMETRIX

Example 2 – Verbose
## This is the /etc/multipath.conf file recommended for
## EMC storage devices.
##
## OS : RHEL5
## Arrays : CLARiiON and SYMMETRIX
## Use user friendly names, instead of using WWIDs as names.
defaults {
}
## The blacklist is the enumeration of all devices that are to be
167 of 243 18.04.2011 16:35

## excluded from multipath control

devnode_blacklist {
## Replace the wwid with the output of the command
## 'scsi_id -g -u -s /block/internal scsi disk name'
## Enumerate the wwid for all internal scsi disks.
## Optionally, the wwid of VCM database may also be listed here.
# wwid 20010b9fd080b7321
devnode "sd[a]$"
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)0-9*"
devnode "^hda-z"
devnode "^cciss!c0-9d0-9*"
}
devices {
## Device attributes for EMC SYMMETRIX
device {
vendor "EMC "
product "SYMMETRIX"
path_grouping_policy multibus
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
path_selector "round-robin 0"
features "0"
hardware_handler "0"
failback immediate
}
## Device attributes for EMC CLARiiON
device {
vendor "DGC"
product "*"
path_grouping_policy group_by_prio
prio_callout "/sbin/mpath_prio_emc /dev/%n"
hardware_handler "1 emc"
features "1 queue_if_no_path"
no_path_retry 300
path_checker emc_clariion
failback immediate
}
}
3. Pillar Data Axiom 600
First add the following lines to /etc/modprobe.conf
alias qla2100 qla2xxx

blacklist {
devnode "^hd[a-z]"
devnode "^cciss!c[0-9]d[0-9]*"
wwid
}
devices {
device {
vendor "Pillar"
product "Axiom 600"
prio_callout "/sbin/mpath_prio_alua %n"
features "0"
rr_weight priorities
rr_min_io 1000
path_checker tur
168 of 243 18.04.2011 16:35

}
device {
vendor "Pillar"
product "Axiom 500"
prio_callout "/sbin/mpath_prio_alua_pillar %n"
features "0"
rr_min_io 1000
path_checker tur
}
4. HP EVA SAN
# vi /etc/multipath.conf
defaults {
}
blacklist {
devnode "^(hd|xvd)[a-z]*"
wwid "*"
}
# Make sure our multipath devices are enabled.
blacklist_exceptions {
wwid "3600508b40008dc480000500000550000"
wwid "3600508b40008dc4800005000005b0000"
wwid "3600508b40008dc4800005000005e0000"
wwid "3600508b40008dc480000500000670000"
wwid "3600508b40008dc480000500000640000"
wwid "3600508b40008dc480000500000610000"
}
multipath {
wwid 3600508b40008dc4800005000005b0000
alias mpath1
}
multipath {
wwid 3600508b40008dc4800005000005e0000
alias mpath2
}
multipath {
wwid 3600508b40008dc480000500000610000
alias mpath3
}
multipath {
wwid 3600508b40008dc480000500000640000
}
multipath {
wwid 3600508b40008dc480000500000670000
alias mpath5
}
5. IBM 2145 SAN
# vi /etc/multipath.conf
devnode_blacklist {
# wwid 26353900f02796769
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*|sda"
devnode "^hd[a-z]"
}
multipaths {
multipath {
169 of 243 18.04.2011 16:35

wwid 3600507680194011ef000000000000b46
alias oraclevm-lun0
path_grouping_policy failover
path_checker readsector0
path_selector "round-robin 0"
failback 0
# no_path_retry 5
}
multipath {
wwid 3600507680194011ef000000000000dbd
alias oraclevm-lun1
}
multipath {
wwid 3600507680194011ef000000000000dbe
alias oraclevm-lun2
}
multipath {
wwid 3600507680194011ef000000000000dc4
alias oraclevm-lun3
}
multipath {
wwid 3600507680194011ef000000000000dbf
alias oraclevm-lun4
}
multipath {
wwid 3600507680194011ef000000000000dc0
alias oraclevm-lun5
}
multipath {
wwid 3600507680194011ef000000000000eaf
alias oraclevm-lun6
}
multipath {
wwid 3600507680194011ef000000000000eb0
alias oraclevm-lun7
}
multipath {
wwid 3600507680194011ef000000000000eb1
alias oraclevm-lun8
}
multipath {
wwid 3600507680194011ef000000000000eb2
alias oraclevm-lun9
}
devices {
# IBM 2145
device {
vendor "IBM"
product "2145"
prio_callout "/sbin/mpath_prio_alua /dev/%n"
}
}
6. 3PAR 224MU6 Storage Array
# Default Blacklist - Ignore Internal Devices

blacklist {
devnode
"^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^cciss!c[0-9]d[0-9]*"
}
defaults {
udev_dir /dev
polling_interval 10
selector "round-robin 0"
getuid_callout "/sbin/scsi_id -g -u -s
/block/%n"
170 of 243 18.04.2011 16:35

prio_callout /bin/true
rr_min_io 100
failback immediate
}
devices {
device {
vendor "3PARdata"
product "VV"
path_checker tur
no_path_retry 60
}
}
Watch this space

Chapter 7: Oracle VM Networking from A to 802.1Q

Oracle VM Networking from A to 802.1Q
Oracle VM Networking Introduction

Beware of Baseboard Management Controllers and Broadcom NetXtreme II NICs
Oracle VM Guest Networking
Oracle VM Server Networking Configuration Examples
The Default ethX/xenbX Configuration
802.3AD NIC Bonding with Xen Bridges
802.3AD NIC Bonding with Xen Bridges using 802.1Q—VLANning
802.3AD NIC Bonding with a Single Xen Bridge Configuration
802.3AD NIC Bonding with Xen bridges and 802.1Q Configuration
Oracle VM Networking Introduction

Oracle VM ships with a flexible networking configuration that can be used as-is or modified to meet your business requirements.
Oracle VM network configuration management is not supported by Oracle VM Manager or the Oracle VM Management Pack; it must
therefore be performed by hand in dom0. Both 802.3AD NIC bonding and 802.1Q VLANning are supported, although they must also
be configured by hand in dom0.
The default Oracle VM server networking configuration routes all dom0 and guest traffic through a Xen bridge. A Xen bridge operates
at layer 2 of the OSI model, effectively acting as a layer 2 (L2) switch passing packets to the egress port; it relies on the TCP protocol
for rate control and packet loss.
Table 1 shows the OSI model. Note that Xen bridges operate in layer 2.
Layer Description
7 Application
Layer
6 Presentation
Layer
5 Session Layer
4 Transport
Layer
171 of 243 18.04.2011 16:35

3 Network
Layer
2 Data Link
Layer
LLC sublayer
MAC sublayer
1 Physical Layer
Oracle VM's default network configuration pairs each network interface (NIC) with a Xen bridge. The Xen bridges are created by
default by the Xen SysV startup script (/etc/init.d/xend). An Oracle VM server with two NICs will have two Xen bridges, eth0/xenbr0
and eth1/xenbr1. The first Xen bridge, eth0/xenbr0, is configured with an IP address on xenbr0 and is dedicated to Oracle VM
management and HA traffic. The second Xen bridge will not have an IP address assigned; it effectively acts as a layer-two switch for
guest traffic. Any Xen bridge, including the ent0/xenbr0 pair, can be used for guest traffic. Guest virtual NICs can be used with any
Xen bridges by editing the guest's vm.cfg file or by editing the guest's network properties using Oracle VM Manager or the Oracle VM
Management Pack.
Tip: In an HA-enabled pool, the loss of network connectivity for the Oracle VM management interface causes an HA event. When an
VM pool member.
It's your prerogative how to physically wire your Oracle VM server's NICs. For example, you could wire each NIC into a managed
switch using VLANs to segregate the traffic from each Xen bridge. Another option is to wire your NICs into an unmanaged switch on a
flat network, using the Xen bridges to distribute traffic across the switch.
Figure 1 shows the default Oracle VM Xen bridge configuration: an Oracle VM server with two NICs wired into a single switch,
hosting three guests.
As shown in Figure 1, the default Oracle VM server network configuration will pair eth0 with xenbr0 and use the eth0/xenbr0 pair for
Oracle VM agent, OCFS2 cluster heartbeat, and Live Migration traffic, and pair eth1 with xenbr1. If an Oracle VM server had four or
more NICs, the default configuration would pair eth0 with xenbr0, used for Oracle VM management traffic, and then create additional
Xen bridges, that is, eth1/xenbr1, eth2/xenbr2, eth3/xenbr3, and so on. Any Xen bridge can be used for guest traffic, even
eth0/xenbr0.
In Figure 1, packets that arrive at the physical NIC (eth0) are handled by dom0’s Ethernet driver and appear on xenbr0. Xen bridges
distribute packets like a layer-two switch for dom0 and guests. Xen bridges route guest packets based on the guest's MAC address.
The next example shows the output from “ifconfig” and “brctl show” from an Oracle VM server with two NICs without any guests.
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:30:48:7F:35:0A
UP BROADCAST RUNNING MULTICAST MTU:1500
Metric:1
RX packets:17459615 errors:0 dropped:0 overruns:0
frame:0
TX packets:15661870 errors:0 dropped:0 overruns:0
carrier:0
collisions:0 txqueuelen:100
RX bytes:3807333630 (3.5 GiB) TX bytes:3092948689 (2.8
GiB)
eth1 Link encap:Ethernet HWaddr 00:30:48:7F:35:0B

172 of 243 18.04.2011 16:35

Metric:1
RX packets:93673 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:9902147 (9.4 MiB) TX bytes:0 (0.0 b)
lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
carrier:0
RX bytes:52010419 (49.6 MiB) TX bytes:52010419 (49.6
MiB)
xenbr0 Link encap:Ethernet HWaddr 00:30:48:7F:35:0A

Metric:1
RX packets:17379990 errors:0 dropped:0 overruns:0
frame:0
carrier:0
RX bytes:3559117450 (3.3 GiB) TX bytes:3092938757 (2.8
GiB)
xenbr1 Link encap:Ethernet HWaddr 00:30:48:7F:35:0B

Metric:1
# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.0030487f350a no eth0
xenbr1 8000.0030487f350b no eth1
The output from “ifconfig” in the above example shows, eth0, eth1, lo, xenbr0, and xenbr1. Next, the output from “brctl show” shows
the Xen bridge configurations, that is, xenbr0 is connected to eth0 and xenbr1 is connected to eth1.
The Oracle VM server's dom0 interfaces are managed using native Linux network scripts, for example, /etc/sysconfig/network and
/etc/sysconfig/network-scripts/ifcfg-*. After an Oracle VM Server 2.2 boots, the Xen /etc/xen/xend-config.sxp script (network-script
network-bridge) entry is referenced and creates an independent Xen bridge for each physical NIC. For example, typing "brctl addif
xenbr0 eth0" creates the eth0/xenbr0 pair and typing "brcrtl addif xenbr0 eth1" creates the eth1/xenbr1 pair.
dom0's management interface is selected during the Oracle VM server installation and can be edited after the installation in the
/etc/ovs-config file. From an HA perspective, the NIC used for Oracle VM management traffic is the only network interface that is
monitored for HA events. If HA is a requirement for both the Oracle VM hosts and guests, it’s important to design your Oracle VM
servers with a single bond used by all Xen bridges for dom0 and guests.
The next example shows Oracle VM's default /etc/sysconfig/network-scripts/ifcfg* scripts from an Oracle VM server with two NICs.
# cat /etc/sysconfig/network-scripts/ifcfg*
# Intel Corporation 80003ES2LAN Gigabit Ethernet
Controller (Copper)
DEVICE=eth0
BOOTPROTO=static
BROADCAST=192.168.4.255
HWADDR=00:30:48:7F:44:6E
IPADDR=192.168.4.7
NETMASK=255.255.255.0
NETWORK=192.168.4.0
ONBOOT=yes
# Intel Corporation 80003ES2LAN Gigabit Ethernet
Controller (Copper)
DEVICE=eth1
BOOTPROTO=static
HWADDR=00:30:48:7F:44:6F
ONBOOT=yes
DEVICE=lo
173 of 243 18.04.2011 16:35

IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
# If you're having problems with gated making
127.0.0.0/8 a martian,
# you can change this to something else
(255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
Note: Oracle VM 2.2 does not use the default Xen.org pseudo devices in dom0, for example, no vif or peth devices are used.
Beware of Baseboard Management Controllers and Broadcom NetXtreme II NICs

Beware of Baseboard Management Controllers that virtualize ethernet ports! Baseboard Management Controllers that virtualize
ethernet ports may not work well with dom0's bonded interfaces and might result in broadcast storms on the connected switches. A
broadcast storm could cause a switch to disable the ports that detect the broadcast storm. HA enabled Oracle VM servers connected
to a switch port that gets disabled would promptly fence from the cluster, reboot and the HA enabled guests would restart on
available pool members.
Bonded Oracle VM server's that use the Broadcom NetXtreme II bnx2x driver on hardware with Baseboard Management Controllers
are subject to broadcast storms. The solution to eliminate the broadcast storms is to a) change the network interface driver from the
bnx2x driver to the tg3 driver or b) disable the Baseboard Management Controller's management firmware and/or c) both a and b.
Typing “dmesg|grep Broad” from dom0 will validate if an Oracle VM server is using Broadcom NetXtreme II network interfaces. The
next example shows the result from typing “dmesg|grep Broad” on an Oracle VM server using Broadcom NetXtreme II network
interfaces.
# dmesg |grep Broad
Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver bnx2x 1.52.12 ($DateTime: 2009/12/17 12:14:50 $)
eth0: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem fb000000, IRQ 22, node addr d8d385d91f88
eth1: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem fa000000, IRQ 23, node addr d8d385d91f8c
eth2: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f9000000, IRQ 23, node addr d8d385d91f89
eth3: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f8000000, IRQ 24, node addr d8d385d91f8d
eth4: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f7000000, IRQ 24, node addr d8d385d91f8a
eth5: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f6000000, IRQ 25, node addr d8d385d91f8e
eth6: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f5000000, IRQ 25, node addr d8d385d91f8b
eth7: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f4000000, IRQ 22, node addr d8d385d91f8f
Broadcom NetXtreme II CNIC Driver cnic v1.9.13b (Dec 16, 2009)
Broadcom NetXtreme II iSCSI Driver bnx2i v1.8.12f (Jan 19, 2010)
The next example shows an Oracle VM server using the bnx2x network interface driver for eth0 and eth1.
# cat /etc/modprobe.conf
alias eth0 bnx2x
alias eth1 bnx2x
alias scsi_hostadapter lpfc
alias scsi_hostadapter1 usb-storage
alias bond0 bonding options bonding mode=1 miimon=100 downdelay=200 updelay=200 max_bonds=1
To change the network interface driver from bnx2x to tg3, it is necessary to edit dom0's modprob.conf file as shown in the next
example.
# cat /etc/modprobe.conf
alias eth0 tg3
alias eth1 tg3
alias scsi_hostadapter lpfc
alias scsi_hostadapter1 usb-storage
alias bond0 bonding options bonding mode=1 miimon=100 downdelay=200 updelay=200 max_bonds=1
After changing the network interface driver in the modprobe.conf file, restart the network service by typing “service network restart”.
Typing “service network restart” will unload the bnx2x driver and load the tg3 driver.
Broadcom has a DOS utility named xdiag that can be used to disable the Baseboard Management Controller's management firmware
(MF) code. If typing "xdiag -ver" shows MF active" on a controller, it is necessary to type "xdiag -c <controller#> -mfw 0" to disable
the controler. xdiag can be downloaded as a stand alone application or as a bootable ISO.
The next example shows how to disable a Baseboard Management Controller's management firmware using a bootable ISO that
contains xdiag.
1- Download and burn this image:

http://www.broadcom.com/support/ethernet_nic/driver-sla.php?driver=NX2-diag
2- Boot the Oracle VM server using the bootable cdrom
3- At the boot prompt press enter to continue
4- Select 1 Install to hardisk using Freedos Setup
5- Press enter to select English
6- Run freedos from the ISO image
7- Disable the management firmware on each network interface as shown in the following example:
174 of 243 18.04.2011 16:35

>cd nx2
>uxdiag -c 1 -mfw 0
>uxdiag -c 2 -mfw 0
The above example disabled the management firmware on controllers 1 and 2.
Oracle VM Guest Networking

Oracle VM networking can be broken down into two parts. The first part of the networking configuration consists of the Oracle VM
servers and the second part of the networking configuration is for the guests. As explained above, Oracle VM server networking is
configured in dom0 using native Linux networking scripts along with the default Xen SysV startup script (/etc/init.d/xend). Guest
networking is configured in each guest's vm.cfg file and within the each guest’s operating system.
Guest networking is configured in each guest's vm.cfg file in the “vif” and “vif_other_config” directives. Each guest will have a unique
vm.cfg file, located in the storage repository, for example, the /var/ovs/mount/<UUID>/running_pool/<VIRTUAL MACHINE NAME>
directory. The next example shows a vm.cfg file from the Oracle VM 11g template. The “vif” and “vif_other_config” directives are
highlighted.
disk = ['file:/OVS/running_pool/v52x8611g1
/System.img,xvda,w',
'file:/OVS/running_pool/v52x8611g1
/oracle11g_x86_asm.img,xvdb,w',
]
memory = '2048'
name = 'v52x8611g1'
vcpus = 2
vif =
['bridge=xenbr0,mac=00:16:3E:56:20:63,type=netfront']
In the above example, the settings in the “vif” directive show that:
1. the guest is using xenbr0 “bridge=xenbr0”

2. the guest's MAC address “mac=00:16:3E:29:D1:49”
3. dom0's front end driver used for the guest “type=netfront”
The “vif_other_config” directive is empty, indicating that there is not an Oracle VM Manager network policy. A guest with the proper
“vif” and OS networking setup will appear on the network from the assigned Xen bridge in the guest's vm.cfg file.
Tip: To generate a unique MAC address for a guest, type the following text while logged in as root:
PYTHONPATH=/opt/ovs-agent-2.2 python -c "from OVSCommons import randomMAC; print randomMAC()"
The next example shows the output from “ifconfig” and “brctl show” from an Oracle VM server with two NICs hosting three guests;
each guest has a single NIC.
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:30:48:7F:45:AE
Metric:1
MiB)
eth1 Link encap:Ethernet HWaddr 00:30:48:7F:45:AF

Metric:1
carrier:0
RX bytes:986681511 (940.9 MiB) TX bytes:166187546
(158.4 MiB)

inet addr:127.0.0.1 Mask:255.0.0.0
175 of 243 18.04.2011 16:35


carrier:0
(417.1 MiB)
vif2.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF

Metric:1
carrier:0
(136.1 MiB)

Metric:1
RX bytes:6934997 (6.6 MiB) TX bytes:10148669 (9.6 MiB)

Metric:1
MiB)
xenbr0 Link encap:Ethernet HWaddr 00:30:48:7F:45:AE

Metric:1
MiB)
xenbr1 Link encap:Ethernet HWaddr 00:30:48:7F:45:AF

Metric:1
# brctl show
xenbr0 8000.0030487f45ae no vif2.0
eth0
xenbr1 8000.0030487f45af no vif4.0
vif3.0
eth1
The output from “ifconfig” in the above example shows eth0, eth1, lo, vif2.0, vif3.0, vif4.0, xenbr0, and xenbr1. vif2.0, vif3.0, vif4.0 are
the guest virtual NICs. Next, the output from “brctl show” shows the bridge configurations, that is, xenbr0 is connected to vif2.0 and
eth0, and xenbr1 is connected to vif3.0, vif4.0, and eth1.
Oracle VM Server Networking Configuration Examples

In this section we will examine the following three Oracle VM networking configurations:
The default ethX/xenbX configuration

802.3AD NIC bonding with Xen bridges
802.3AD NIC bonding with Xen bridges using 802.1Q - VLANning
The Default ethX/xenbX Configuration
Figure 1 shows the default Oracle VM Xen bridge configuration, with an Oracle VM server with two NICs wired into a single
176 of 243 18.04.2011 16:35

unmanaged switch. The Oracle VM server is hosting three guests that use xennbr1; xenbr0 is dedicated to management traffic.
In Figure 1, the Oracle VM management traffic and HA monitoring are isolated to the eth0/xenbr0 interface and the loss of network
connectivity for eth0/xenbr0 will cause an HA event. If the eth1/xenbr1 interface looses connectivity and all the guests are dropped
from the network, an HA event will not occur.
Figure 2 shows the default Oracle VM Xen bridge configuration with a two-server Oracle VM pool. Each Oracle VM server has two
NICs wired in to a single unmanaged switch. The Oracle VM server is hosting three guests that use xenbr1; xenbr0 is dedicated to
management traffic.
In Figure 2, the Oracle VM management traffic and HA monitoring are isolated to eth0/xenbr0 and the loss of network connectivity for
eth0/xenbr0 will cause an HA event. If the eth1/xenbr1 interface loses connectivity and all the guests are dropped from the network
an HA event will not occur.
802.3AD NIC Bonding with Xen Bridges
Figure 3 shows an 802.3AD NIC bonding configuration with one Xen bridge on bond1.
177 of 243 18.04.2011 16:35

The Oracle VM server in Figure 3 has four NICs configured with two bonds, bond0 and bond1. Each NIC from each bond is wired into
a separate switch for high availability. The Oracle VM server is hosting three guests using xenbr1; bond0 and eth0:0 are dedicated to
management traffic.
Tip: If a bond is dedicated for dom0 management traffic it is not necessary to configure a Xen bridge for dom0; a Xen bridge is only a
requirement for guests to access the network.
In Figure 3, the Oracle VM management traffic and HA monitoring are isolated to bond0 and the eth0:0 interface and the loss of
network connectivity for bond0 or the eth0:0 interface would cause an HA event. If the bond1/xenbr1 interface loses connectivity and
all the guests are dropped from the network, an HA event will not occur.
Figure 4 shows an 802.3AD NIC bonding configurations with one Xen bridge with a two-server Oracle VM pool.
Each Oracle VM server in Figure 4 has four NICs configured with two bonds, bond0 and bond1. Each NIC from each bond is wired
into a separate switch for high availability. Each Oracle VM server is hosting three guests that use xenbr1; bond0 and eth0:0 are
dedicated to management traffic.
In Figure 4 the Oracle VM management traffic and HA monitoring are isolated to bond0 and the eth0:0 interface and the loss of
network connectivity for bond0 or the eth0:0 interface would cause an HA event. If the bond1/xenbr1 interface loses connectivity and
all the guests are dropped from the network, an HA event will not occur.
802.3AD NIC Bonding with Xen Bridges and 802.1Q - VLANning

Figure 5 shows an 802.3AD NIC bonding configuration that uses Xen bridges with 802.1Q VLANning.
The Oracle VM server in Figure 5 has four NICs configured with one bond, bond0. Each NIC from each bond is wired into a separate
switch for high availability. The Oracle VM server is hosting three guests that use vlan2, vlan3, and vlan4; vlan1 is dedicated to
management traffic.
Note: In Figure 5 vlan1, vlan2, vlan3 and vlan4 are Xen bridges. Xen bridges can have discriptive names, i.e. vlanX, xenbrX, etc...
In Figure 5, the Oracle VM management traffic and HA monitoring are isolated to the bond0:1 interface over vlan1. If dom0 loses
network connectivity due to bond0, bond0:1 or vlan1 an HA event would occur causing the Oracle VM server to fence from the pool
and reboot, all HA-enabled guests would restart on a live Oracle VM pool member.
Figure 6 shows an 802.3AD NIC bonding configurations with a two-server Oracle VM pool that uses Xen bridges with 802.1Q VLAN
and VLAN tagging.
178 of 243 18.04.2011 16:35

Each Oracle VM server in Figure 5 has four NICs configured with one bond, bond0. Each NIC from each bond is wired into a separate
switch for high availability. Each Oracle VM server is hosting three guests that use vlan2, vlan3, and vlan4, respectively; vlan1 is
dedicated to management traffic. The loss of network connectivity with the bond0 interface would cause an HA event that fences the
Oracle VM server and reboots it, and restart all HA-enabled guests on a live Oracle VM pool member.
802.3AD NIC Bonding with a Single Xen Bridge Configuration

This section will explain how to configure NIC bonding with a single Xen bridge on an Oracle VM server with two NICs. The example
configuration has a single Xen bridge used for dom0 and the guests. In this example, the Oracle VM server and the guests are on the
same network and use the same Xen bridge. Each Oracle VM server in a pool should have the identical networking configuration.
Figure 7 shows an example configuration with NIC bonding using a single Xen bridge.
The example configuration will create a bond with two NICs, one interface and one Xen bridge, used by dom0 and guests. After
configuring the bonding interface, dom0 will have the following interfaces:
eth0: the first physical NIC

eth1: the second physical NIC
bond0: The bonding device bond0
xenbr0: Xen bridge with an IP address for dom0 and the guests
Note: This example configuration provides HA for both dom0 and the guests if the Oracle VM server loses network connectivity.
Step 1: Disable the default xend network configuration. From dom0 type the following commands while logged in as root to disable
the xend network configuration:
# cd /etc/xen/scripts/
# ./network-bridges
stop
Step 2: Create a file in the /etc/xen/scripts directory named “network-bridge-ovs” that contains the following lines:
#!/bin/sh
/bin/true
For example, from dom0 while logged in as root type “vi /etc/xen/scripts/network-bridge-ovs” to create a file named network-
bridge-ovs, then enter the text in the next example:
179 of 243 18.04.2011 16:35

#!/bin/sh
/bin/true
Step 3: Make the etc/xen/scripts/network-bridge-ovs file executable by typing the text in the next example.
# chmod 755 /etc/xen/scripts/network-

bridge-ovs
Step 4: Edit the /etc/xen/xend-config.sxp file and change the following line from “network-script network-bridges” to “network-script
network-bridge-ovs”.
Step 5: Edit /etc/modprobe.conf and load and configure the bonding modules. For example:
alias eth0 <DriverName>

alias bond0 bonding
options bonding mode=1 miimon=100 downdelay=200 updelay=200
Change the <DriverName> to the correct driver for your NICs, for example, tg3, e1000, or the like.
We have done extensive testing with the various bonding modes and suggest mode 1 for all Oracle VM configurations. For example,
the existing 2.6.18 kernel has known issues with mode 4 and 6.
Note: Please review Bonding with LACP does not work if your testing mode 4. Consider testing the "xmit_hash_policy=layer2+3"
setting in the above modprobe.conf file. Adding the "xmit_hash_policy=layer2+3" setting in the modprobe.conf file may help spread
the load between the NICs in the bond and provide better throughput.
To list the NIC drivers, try one of the following commands from dom0 while logged in as root:
# dmesg | grep 'Ethernet driver'

# grep 'Ethernet driver' /var/log/dmesg
# grep ethX /etc/modules.conf
Step 6: Configure the network devices and the Xen bridge.
To configure the eth0 and eth1 interfaces, edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file as follows:
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
HWADDR=<YOUR MAC ADDRESS>
For /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
The HWADDR parameter is the pointer to the NIC's MAC address. The SLAVE parameter defines the network card as a slave of a
bond-device, in this case bond0. The MASTER parameter points to the actual bonding device that this network interface will be part
of.
Next, create the bonding device bond0 by creating a file named /etc/sysconfig/network-scripts/ifcfg-bond0, containing the following
lines:
DEVICE=bond0
ONBOOT=yes
BRIDGE=xenbr0
USERCTL=no
The next example shows how to create and save a file named /etc/sysconfig/network-scripts/ifcfg-bond0 with the bond device
parameters.
# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
180 of 243 18.04.2011 16:35

BRIDGE=xenbr0
USERCTL=no
:wq!
The BRIDGE parameter specifies that the bond interface does not have an IP address, but will be connected to bridge xenbr0.
Next, create the Xen bridge by creating a file named /etc/sysconfig/network-scripts/ifcfg-xenbr0 containing the following lines:
DEVICE=xenbr0
ONBOOT=yes
STP=off
USERCTL=no
IPADDR=<YOUR IP ADDRESS>
NETMASK=<YOUR NETMASK>
The next example shows how to create and save a file named ifcfg-xenbr0 that contains the Xen bridge parameters.
# vi /etc/sysconfig/network-scripts/ifcfg-xenbr0
DEVICE=xenbr0
ONBOOT=yes
STP=off
USERCTL=no
IPADDR=<YOUR IP ADDRESS>
NETMASK=<YOUR NETMASK>
:wq!
An IP address is assigned to xenbr0 using the IPADDR and NETMASK lines. The NETWORK and BROADCAST parameters are
deprecated and are automatically calculated with ipcalc.
Step 5: Restart the network and xend services from dom0 by typing “service network restart” followed by “service xend restart,” as
shown in the next example:
# service network restart

# service xend restart
Once xend and the network service have been restarted, the bond with two NICs with one Xen bridge will be ready for use. In Oracle
VM Manager each guest should be configured to use xenbr0.
To validate the configurations, from dom0 type “ifconfig” to list the interfaces and the Xen bridges, as shown in the next example.
bond0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91

UP BROADCAST RUNNING MASTER MULTICAST MTU:1500
Metric:1
MiB)
eth0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91

UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500
Metric:1

BROADCAST MULTICAST MTU:1500 Metric:1
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

inet addr:127.0.0.1 Mask:255.0.0.0
xenbr0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91

181 of 243 18.04.2011 16:35

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

As shown in the next example, type “brctl show” to confirm that the desired interface is using the appropriate Xen bridge.
# brctl show
xenbr0 8000.0015172d8a91 no bond0
The output for “brctl show” lists the bridge name, bridge ID, spanning-tree protocol (STP) configuration, and the configured
interfaces.
The next example shows a vm.cfg file from a guest that is configured to use xenbr0.
disk = ['file:/OVS/running_pool/v52x8611g1
/System.img,xvda,w',
'file:/OVS/running_pool/v52x8611g1
/oracle11g_x86_asm.img,xvdb,w',
]
memory = '2048'
name = 'v52x8611g1'
vcpus = 2
vif =
['bridge=xenbr0,mac=00:16:3E:56:20:63,type=netfront']
Tip: All Oracle VM servers using bonded interfaces should modify two default O2CB heartbeat configurations to allow a bond fail-over
event to occur without the Oracle VM server fencing from the pool. Edit the /etc/sysconfig/o2cb file using your favorite text editor and
change "O2CB_HEARTBEAT_THRESHOLD=" to "O2CB_HEARTBEAT_THRESHOLD= 61" and "O2CB_IDLE_TIMEOUT_MS=" to
"O2CB_IDLE_TIMEOUT_MS= 60000". The next example shows the suggested modifications made to a /etc/sysconfig/o2cb file.
# vi /etc/sysconfig/o2cb
#
# This is a configuration file for automatic startup of the O2CB
# driver. It is generated by running /etc/init.d/o2cb configure.
# Please use that method to modify this file
#
# O2CB_ENABELED: 'true' means to load the driver on boot.
O2CB_ENABLED=true
# O2CB_BOOTCLUSTER: If not empty, the name of a cluster to start.
O2CB_BOOTCLUSTER=rmvsx
# O2CB_HEARTBEAT_THRESHOLD: Iterations before a node is considered dead.
O2CB_HEARTBEAT_THRESHOLD= 61
# O2CB_IDLE_TIMEOUT_MS: Time in ms before a network connection is considered dead.
O2CB_IDLE_TIMEOUT_MS= 60000
# O2CB_KEEPALIVE_DELAY_MS: Max time in ms before a keepalive packet is sent
O2CB_KEEPALIVE_DELAY_MS=
802.3AD NIC Bonding and Xen Bridges with 802.1Q Configuration

This section will examine how to configure NIC bonding with VLANning using multiple Xen bridge on an Oracle VM server with two
NICs. To be able to use 802.1Q VLANning, your switches will need to support VLAN trunking and be correctly configured. The
example configuration uses a single bond with multiple Xen bridges for dom0 and guests. Each Oracle VM server in a pool should
have the identical networking configuration.
Figure 8 shows an example 802.3AD NIC bonding and Xen bridges with 802.1Q VLANning.
182 of 243 18.04.2011 16:35

The example configuration will have an 802.3AD bond with two NICs with one interface using four Xen bridges. After the bonding
interface is configured, dom0 will have the following interfaces:
eth0: the first physical NIC

eth1: the second physical NIC
bond0: bonding interface for eth0 and eth1
bond0:1 with vlan1 for dom0
bond0:2 with vlan2
bond0:3 with vlan3
bond0:4 with vlan4
The example configuration offers HA for both dom0 and the guest Xen bridges if the Oracle VM server loses network connectivity.
Note: Please note that the :XX part of the interface name, that is, bond0:1, bond0:2, bond0:3, bond0:4, is referencing VLAN 1, 2, 3,
and 4, respectively.
Step 1: Disable the default xend network configuration. From dom0 type the following commands while logged in as root to disable
the xend network configuration:
# cd /etc/xen/scripts/
# ./network-bridges
stop
Step 2: Create a file in the /etc/xen/scripts directory named “network-bridge-ovs” that contains the lines:
#!/bin/sh
/bin/true
For example, from dom0 while logged in as root type “vi /etc/xen/scripts/network-bridge-ovs” to create a file named network-
bridge-ovs, then enter the text in the next example.
#vi /etc/xen/scripts/network-
bridge-ovs
#!/bin/sh
/bin/true
:wq!
Step 3: Make the etc/xen/scripts/network-bridge-ovs file executable by typing the text in the next example.
# chmod 755 /etc/xen/scripts/network-

bridge-ovs
Step 4: Edit the /etc/xen/xend-config.sxp file and change the line“network-script network-bridges” to “network-script network-
bridge-ovs”.
Step 5: Edit /etc/modprobe.conf and load and configure the bonding modules. For example:

alias bond0 bonding
options bonding mode=1 miimon=100 downdelay=200 updelay=200
Change the <DriverName> to the correct driver for your NICs, for example, tg3, e1000, or the like.
183 of 243 18.04.2011 16:35

Note: Please review Bonding with LACP does not work and consider testing the "xmit_hash_policy=layer2+3" setting in the above modprobe.conf file.
Adding the "xmit_hash_policy=layer2+3" setting in the modprobe.conf file may help spread the load between the NICs in the bond and provide better
throughput.
To list the NIC drivers, use one of the following commands from dom0 while logged in as root.
# dmesg | grep 'Ethernet driver'

# grep 'Ethernet driver' /var/log/dmesg
# grep ethX /etc/modules.conf
Step 6: Configure the network devices and the Xen bridge.
To configure the eth0 and eth1 interfaces, edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file as follows:
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
For /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
The HWADDR parameter is the pointer to the NIC's MAC address. The SLAVE parameter defines the network card as a slave of a
bond-device, in this case, bond0. The MASTER parameter points to the actual bonding device that this network interface will be part
of.
Next, create the bonding device bond0 by creating a file named /etc/sysconfig/network-scripts/ifcfg-bond0 containing the following
lines:
DEVICE=bond0
ONBOOT=yes
USERCTL=no
The next example shows how to create and save a file named /etc/sysconfig/network-scripts/ifcfg-bond0 with the bond-device
parameters.
# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
USERCTL=no
:wq!
Next, create the interfaces by making a file for each interface named /etc/sysconfig/network-scripts/ifcfg-<INTERFACE NAME>. The
following examples show four interface files.
Note: Any descriptive name can be used for the interface and Xen bridge. You can create as many interfaces and Xen bridges as
needed for your environment.
The next example is the file for the bond0:1 interface that uses vlan1.
DEVICE=bond0:1
ONBOOT=yes
BOOTPROTO=none
VLAN=yes
TYPE=Ethernet
BRIDGE=vlan1
DEVICE=bond0:2
ONBOOT=yes
BOOTPROTO=none
VLAN=yes
TYPE=Ethernet
BRIDGE=vlan2
184 of 243 18.04.2011 16:35

DEVICE=bond0:3
ONBOOT=yes
BOOTPROTO=none
VLAN=yes
TYPE=Ethernet
BRIDGE=vlan3
DEVICE=bond0:4
ONBOOT=yes
BOOTPROTO=none
VLAN=yes
TYPE=Ethernet
BRIDGE=vlan4
Next, create the Xen bridges by making a file for each Xen bridge, named /etc/sysconfig/network-scripts/ifcfg-<XEN BRIDGE NAME>.
The following examples show four Xen bridge files.
The next example is the file for the vlan1 that is used by bond0:1.
DEVICE=vlan1
BOOTPROTO=static
ONBOOT=yes
IPADDR=<YOUR IP ADDRESS FOR DOM0>
NETMASK=<YOUR NETMASK FOR DOM0>
An IP address is assigned to xenbr11 using the IPADDR and NETMASK sections. The NETWORK and BROADCAST parameters are
deprecated and are automatically calculated with ipcalc.
DEVICE=vlan2
BOOTPROTO=none
ONBOOT=yes
Type=Bridge
DEVICE=vlan3
BOOTPROTO=none
ONBOOT=yes
Type=Bridge
DEVICE=vlan4
BOOTPROTO=none
ONBOOT=yes
Type=Bridge
The next example shows how to create and save a file named ifcfg-vlan2 that contains the Xen bridge parameters.
# vi /etc/sysconfig/network-scripts/ifcfg-vlan2
DEVICE=vlan2
BOOTPROTO=none
ONBOOT=yes
Type=Bridge
:wq!
Step 5: Restart the network and xend services from dom0 by typing “service network restart” followed by “service xend restart” as
# service network restart

# service xend restart
Once xend and the network service have been restarted, the new configuration will be ready for use. In Oracle VM Manager, each
guest can be configured to use any of the Xen bridges.
To validate the configurations, from dom0 type “ifconfig” to list the interfaces and the Xen bridges, as shown in the next example.
185 of 243 18.04.2011 16:35

bond0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91

Metric:1
MiB)
bond0:1 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91

Metric:1

Metric:1

Metric:1
RX bytes:9773662 (9.3 MiB) TX bytes:53429 (52.1 KiB)

Metric:1
RX bytes:9773662 (9.3 MiB) TX bytes:53429 (52.1 KiB)

UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500
Metric:1

BROADCAST MULTICAST MTU:1500 Metric:1
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

inet addr:127.0.0.1 Mask:255.0.0.0
vlan2 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91


186 of 243 18.04.2011 16:35




As shown in the next example, type “brctl show” to confirm that the desired interfaces are using the correct Xen bridges.
# brctl show
vlan1 8000.0015172d8a91 no bond0:1
vlan2 8000.0015172d8a91 no bond0:2
vlan3 8000.0015172d8a91 no bond0:3
vlan4 8000.0015172d8a91 no bond0:4
The output for “brctl show” lists the bridge name, bridge ID, spanning-tree protocol (STP) configuration and the configured
interfaces.
Tip: All Oracle VM servers using bonded interfaces should modify two default O2CB heartbeat configurations to allow a bond fail-over
event to occur without the Oracle VM server fencing from the pool. Edit the /etc/sysconfig/o2cb file using your favorite text editor and
change "O2CB_HEARTBEAT_THRESHOLD=" to "O2CB_HEARTBEAT_THRESHOLD= 61" and "O2CB_IDLE_TIMEOUT_MS=" to
"O2CB_IDLE_TIMEOUT_MS= 60000". The next example shows the suggested modifications made to a /etc/sysconfig/o2cb file.
# vi /etc/sysconfig/o2cb
#
# This is a configuration file for automatic startup of the O2CB
# driver. It is generated by running /etc/init.d/o2cb configure.
# Please use that method to modify this file
#
# O2CB_ENABELED: 'true' means to load the driver on boot.
O2CB_ENABLED=true
# O2CB_BOOTCLUSTER: If not empty, the name of a cluster to start.
O2CB_BOOTCLUSTER=rmvsx
# O2CB_HEARTBEAT_THRESHOLD: Iterations before a node is considered dead.
O2CB_HEARTBEAT_THRESHOLD= 61
# O2CB_IDLE_TIMEOUT_MS: Time in ms before a network connection is considered dead.
O2CB_IDLE_TIMEOUT_MS= 60000
# O2CB_KEEPALIVE_DELAY_MS: Max time in ms before a keepalive packet is sent
O2CB_KEEPALIVE_DELAY_MS=
Watch this space
Chapter 8: Virtualizing Oracle Database 10g/11g

Oracle Database 10g/11g Virtualization
187 of 243 18.04.2011 16:35

This chapter will review how to rapidly deploy a virtualized Oracle 10g/11g Database using Oracle VM Templates with Oracle VM Manager.
Note: As of this writing, the Oracle 10g/11g Database Oracle VM Templates will not boot on an Oracle VM server with more than 60 G or RAM. Oracle VM templates
using Oracle Linux 5.3 and below ship with a known kernel bug that will not allow the Oracle VM template to boot on an Oracle VM server with more than 60 G or
RAM. If your Oracle VM server has more than 60 G of RAM and an Oracle VM template will not boot, edit the grub.conf file on the Oracle VM server to add the
memory parameter to kernel line that loads the Xen hypervisor, i.e.
kernel /xen-64bit.gz dom0_mem=1024M mem=50G
After adding the mem=50G entry, reboot the Oracle VM server. With the addition of the mem=50G entry, the Oracle VM template will be able to boot on the Oracle
VM server. Once the Oracle VM template is running, patch the kernel to resolve the kernel bug, and then remove the the mem=50G entry on the Oracle VM server.
Oracle VM Templates Overview

Oracle VM Templates are self-contained and pre-configured virtual machines with key Oracle technologies that can be downloaded and rapidly deployed in to
production or test environments. Each Oracle VM Template is packaged using Oracle best practices, which eliminates operating system and application installation
and configuration costs, reduces risk and dramatically shortens deployment time. Oracle VM Templates are available from the Linux Oracle eDelivery portal. There is
no other vendor in the virtualization market that can offer this level of in-depth knowledge, integration, management and pre-packaging of applications with a
production ready operating system.
The use of Oracle VM Templates for the deployment of applications in Oracle VM guest virtual machines eliminates the need for a user to install and configure the
operating system or applications. The virtual machines created using Oracle VM Templates can be started either from the Oracle VM Manager or the Oracle VM
Management Pack (an Oracle Enterprise Manager plug-in).
Oracle VM Templates include a free download and free trial license with the option to purchase a product license. Oracle VM Templates do not have time limits or
feature limitations, e.g. Oracle VM Templates are full featured and do not have expiration dates. Oracle VM Templates can be quickly transitioned from evaluation
into production by purchasing Oracle technology licenses.
List 1 shows Oracle’s current Oracle VM templates collection:
Oracle Virtualization and Management

Oracle VM Manager 2.2 and 2.1.5
Oracle VM Template Builder 2.1
Oracle VM Server 2.2.1 and 2.1.5 SDK
Sun Ray Software 5
Oracle Secure Global Desktop
Oracle Applications
E-Business Suite 12.X Sparse Middle Tier
JD Edwards EnterpriseOne 9.0 Update 1 with ESUs and JD Edwards EnterpriseOne Tools 8.98 Update 3
JD Edwards EnterpriseOne 9.0 Update 1 and JD Edwards EnterpriseOneTools 8.98 Update 2
PeopleSoft ELM 9.1 Bundle #2 with PeopleTools 8.50.09
PeopleSoft FSCM 9.1 Bundle #4 (includes Maintenance Pack 2) with PeopleTools 8.50.10
PeopleSoft CRM 9.1 Bundle #2 with PeopleTools 8.50.09
PeopleSoft Portal Solutions 9.1 and PeopleTools 8.50.09
PeopleSoft HCM 9.1 and PeopleTools 8.50.02
Siebel CRM SIA 8.1.1
Siebel CRM SIA 8.0
Oracle Middleware
Oracle WebLogic Server on JRockit Virtual Edition 11g R1 (10.3.2)
Oracle WebLogic Server 10g Release 3
Oracle Business Intelligence Enterprise Edition 10.1.3.4
Oracle Application Server 10g Release 3 WebCenter
Oracle Identity Management 10g Release 2
Oracle Fusion Middleware Service Oriented Architecture (SOA) 10.1.3.4 and 10.1.3.3
Oracle Database & Oracle Real Application Clusters (RAC)

Oracle Database 11g and Oracle Database 10g Release 2
Oracle Operating Systems

Oracle Linux 4 and 5
188 of 243 18.04.2011 16:35

Oracle Solaris 10
3rd Party Software

Zeus Traffic Manager
Watch this space

How to Download Oracle VM Templates

This chapter covers how to download Oracle VM templates.
Note: The Oracle VM Templates are only available from the Linux Oracle eDelivery site at http://edelivery.oracle.com/linux. The
Oracle VM templates are not available from the root http://edelivery.oracle.com/ eDelivery URL.
How to Download Oracle VM Templates

a) Enter http://edelivery.oracle.com/linux in to a Web browser. Click the Continue button to proceed to the Registration form.
Figure 1
b) Complete the Registration form, accept the license agreement and the export agreement. Next, click the Continue button to
proceed to the Search page.
Figure 2
189 of 243 18.04.2011 16:35

c) From the Media Pack Search page you can select the a) Oracle VM Templates b) Enterprise Linux c) Oracle VM search
options.
Figure 3
Select the Oracle VM Templates search option from the Select a Product Pack drop down list. Next, select ether x86 32 bit or the
x86 64 option from the Platform drop down list. Click the Go button to proceed to the Oracle VM Templates Media Pack
Search page.
Figure 4
190 of 243 18.04.2011 16:35

d) From the Media Pack Search page, locate and click on the desired Oracle VM Template link.
Figure 5
191 of 243 18.04.2011 16:35

d) Review the Oracle VM Templates on the desired <Oracle VM Template Name> Media Pack Download pages and select and
download the zip files.
Figure 6 shows the Oracle VM Template for Oracle Database Media Pack Download page. To download the Oracle Database
Media Pack, select and download both files.
192 of 243 18.04.2011 16:35

e) Next, click the Readme button and review the Readme for the Oracle VM Template template.
f) Once the Oracle VM template files have been downloaded, copy the zip files to your Oracle VM server's /OVS/running_pool directory
or the /OVS/seed_pool directory. Next, unzip and untar the files in the directory.
Note: Use the running_pool directory if you would like to deploy the Oracle VM template as a running virtual machine. Use the
seed_pool directory if you would like to deploy the Oracle VM template as a reusable template that can be re-deployed an unlimited
number of times.
Watch this space
How to Import the Oracle Database 11g Template with

Oracle VM Manager
This chapter will review how to import the Oracle Database 11g template using Oracle VM Manager.
a) Access the Oracle VM Manager GUI by typing the Oracle VM Manager URL in your browser. Enter your username and password
in to the Username and Password text boxes. Then, press the Login button to authenticate to access the Oracle VM Manager GUI.
Figure 1
b) From the Virtual Machines home page click the Resources tab. From the Resources tab click the Virtual Machine Image link. From the Virtual
Machine Images page click the Import Button.
Note: The example assumes that the Oracle Database 11g template has been unzipped and untarred in the /OVS/running_pool directory. If the Oracle
database 11g template was unzipped and untarred in the /OVS/seed_pool directory, click the Virtual Machine Templates tab to import the template.
Figure 2
193 of 243 18.04.2011 16:35

d) From the Source page select the Select from Server Pool (Discover and register) radio button and click Next to proceed.
Note: The Select from Server Pool (Discover and register) requires that the Oracle Database 11g template has been staged, i.e. unzipped
and untarred in the /OVS/running_pool directory.
Figure 3
d) From the General Information page complete the form for the Oracle VM 11g Template's properties. Once all the data is complete click Next to
proceed.
Server Pool Name:

Required Sharing:
Required Virtual Machine Image Name:
Enable High Availability
Required Operating System: Oracle Enterprise Linux 5 (either x86 or x86_64)
Virtual Machine System Username:
Virtual Machine System Password:
Required Console Password:
Required Confirm Console Password:
Description:
Figure 4
e) Click the Confirm button to proceed.

Figure 5
194 of 243 18.04.2011 16:35

f) From the Virtual Machine Images page click the Refresh button. The status will change from Importing to Pending. Next, click
the Approve button.
Figure 6
g) From the View Imported Virtual Machine page click the Approve button.
Figure 7
h) After the Virtual Machine Image is approved click the Virtual Machines tab. From the Virtual Machines page you will see the Oracle Database 11g
template in the list of Virtual Machines in the Powered Off status.
Figure 8
195 of 243 18.04.2011 16:35

Watch this space

How to Start, Configure and Deploy the Oracle Database 11g

Template using Oracle VM Manager
This chapter show how to start, configure and how to deploy the Oracle Database 11g template.
The Virtual Machines page is the Oracle VM virtual infrastructure dashboard.

The dashboard offers visibility in to the virtual machine's status and properties, i.e. memory allocation, status, owner, group name, server name, server
pool name. For example, click the the virtual machine's +Show option to list the virtual machine properties dashboard.
The Select and feature allow you to Power On, access the virtual machine's Console, Power Off, and Configure a virtual machine's properties, i.e.
resource allocations and policies.
The More Actions drop down list offers the following virtual machine administrative features: Deploy, Live Migration, Clone, Save as Template, Pause,
Unpause, Suspend, Resume, Delete and Reset.
a) From the Virtual Machine page start your virtual machine by pressing the Power On button. The virtual machine status will change from Powered
Off to Initializing to Running.
Figure 1
196 of 243 18.04.2011 16:35

b) Once the status is Running click the Console button to access the virtual machine's console.
Note: If you receive any Java Warnings, click the Always trust content from the publisher check box and click Run.
c) Enter the VNC Authentication Password in the Password text box and click the OK button to access the virtual machine's console.
Figure 2
d) When a virtual machine created from Oracle Database template boots up for the first time, the boot up process gathers database configuration
information from the user and configures the database automatically, please follow the prompts in the Oracle VM Manager virtual machine VNC console
to complete the process.
1) System configuration
User will be asked if a static IP address should be configured. Answer no “y” to use DHCP.
Configuring network for Oracle Database.
Current network is using DHCP. Using static IP address is recommended.
Use DHCP? y/n [n]
Configuring static IP.
Starting network...
2) Oracle database configuration (ACCEPT THE DEFAULTS, JUST PRESS ENTER)

Starting Oracle database reconfiguration.
Do you want to relink binaries? (y/n) [n]
Specify the HTTP port that will be used for Oracle Application Express [8080]:
Specify a port that will be used for the database listener [1521]:
Specify a password to be used for database accounts. Note that the same
password will be used for SYS, SYSTEM and ADMIN for APEX. Oracle recommends
the use of different passwords for each database account. This can be done
after initial configuration:
Do you want Oracle Database to be started on boot (y/n) [y]:
Running orainstRoot.sh and root.sh...
Changing permissions of /u01/app/oraInventory to 770.
Changing groupname of /u01/app/oraInventory to dba.
The execution of the script is complete
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
197 of 243 18.04.2011 16:35

ORACLE_HOME= /u01/app/oracle/product/11.1.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
Starting CSS.
Starting Oracle Net Listener...Done
ASM instance started
ORACLE instance started.
Configuration Completed Successfully.
To access the Oracle Application Express go to
"http://myhost.mydomain.com:8080/apex"
d) The Oracle Database 11g Template is configured, on the network and 100% operational. The virtual machine is accessible via ssh using the IP address
select during the configuration process with the default user name root and the default root password ovsroot. The virtual machine's console can be
accessed using the console password that was selected during the configuration process using the console button in Oracle VM Manager.
Notes:
1. The system image contains a minimal install of Oracle Linux. Only basic RPM packages are installed.
1. The default root password is 'ovsroot'.
2. A database instance 'orcl' is created in the templates.
1. The database user is named “oracle” and belongs to groups 'orainstall' and 'dba'. The password is 'oracle'.
2. The oracle-validated package is installed. This package verifies and sets system parameters based on Oracle validated configuration
recommendations for Oracle Enterprise Linux.
3. The database storage is managed by Automatic Storage Management (ASM). Two ASM disks 'VOL1' and 'VOL2' are created, and the default
ASM group name is 'DATA'.
Watch this space

Chapter 9: Virtualizing Oracle Enterprise Manager 10g

The Oracle Enterprise Manager 10g Oracle VM template allows customers to quickly deploy Oracle Enterprise Manager 10g/11g
using Oracle VM Manager.
The Oracle Enterprise Manager 10g template is comprised of a virtual machine assembly with two pre-configured virtual machines.
Both virtual machines use Oracle Enterprise Linux 4 update 5. The first virtual machine hosts the Oracle Management Service (OMS)
and an optional YUM repository. The second virtual machine hosts the Oracle Management Repository (OMR). Each virtual machine
was created with Oracle best practices, which reduce deployment risk and dramatically shortens deployment timelines. Oracle VM
templates are freely available from http://edelivery.oracle.com/linux.
The Oracle Enterprise Manager 10g Oracle VM template is downloaded as two sets of compressed files (14.81 GB total)
from http://edelivery.oracle.com/linux. The first set of four files (5.69 GB) is named “Oracle VM Enterprise Manager Grid Control
Template for Oracle VM” and the second set of five files (9.12 GB) is named “Yum Repository of Oracle Enterprise Linux 4 and 5 for
use with Enterprise Manager Grid Control Template”. The second set of files for the Yum repository is optional.
Figure 1 shows how the Oracle Enterprise Manager 10g Oracle VM template is deployed in to an Oracle VM server farm.
198 of 243 18.04.2011 16:35

System Requirements
A minimum of one physical servers is required for the Oracle VM Enterprise Manager Grid Control Template.
Oracle VM Server:
Oracle VM Server requires one bare-metal server capable of running at least three virtual machines simultaneously. The
recommended minimum hardware requirements for an Oracle VM Server are:
4 CPUs
4 GB RAM
80 GB disk space without the Yum repository
100 GB disk space with the Yum repository
The Oracle Management Repository Virtual Machine:

2 Virtual CPUs
2 GB RAM
5.2 GB disk space in /OVS/seed_pool/ (virtual machine template storage)
33 GB disk space in /OVS/running_pool/ (virtual machine storage)
The Oracle Management Service and the Yum repository Virtual Machine:
2 Virtual CPUs (virtual machine)
2 GB RAM (virtual machine)
16 GB disk space in /OVS/seed_pool/ (virtual machine template storage)
72 GB disk space in /OVS/running_pool/ (virtual machine storage)
Oracle Enterprise Manager 10g Oracle VM Template Installation Steps

There are a total of eight steps to deploy the Oracle Enterprise Manager 10g Oracle VM template. Table 1 lists each step with a brief explanation including an
estimated time to completion.
Step Description Estimated

Time to
Completion
1. Download Download the Oracle VM 2 to x hours

the Template. Enterprise Manager Grid depending
Control Template on your
from http://edelivery.oracle.com Internet
/linuxand place the zip files in connection
the /OVS/seed_pool/ directory. speed.
2. Unzip and Uncompress the zip files in the Up to 30

concatenate /OVS/seed_pool/ directory, then minutes
the file sets concatenate each set of files depending
and add the and add the yum.img file to the on system
yum.img file OMS vm.cfg file. resources.
to the OMS
vm.cfg file.
199 of 243 18.04.2011 16:35

3. Create the With Oracle VM Manager Up to 15

OMR and create the OMR and OMS minutes.
OMS virtual virtual machine templates.
machine
templates.
4. Create the With Oracle VM Manager Up to 6

OMR and create the OMR and OMS hours
OMS virtual virtual machines with the OMR depending
machines. and OMS templates. on system
resources.
5. Start and With Oracle VM Manager start Up to 1 hour

configure the the OMR virtual machine, for each
OMR and access the console via Oracle virtual
OMS virtual VM Manager to enter the setup machine.
machines. options. After the OMR virtual
machine is configured, start the
OMS virtual machine to enter
the setup options.
Note: The OMR virtual

machine must be started first.
6. Install OMA Login to the OMS virtual Up to 15

on the OMS machine as the root and minutes
virtual execute the “/installagent.sh” depending
machine. script to install the Oracle on system
Management Agent (OMA) resources.
version 10.2.0.4 x86.
7. Log into Login to the Enterprise 1 Minute.

the Enterprise Manager Grid Control portal
Manager Grid using the sysman account and
Control the password selected in step
portal. 5.
8. Change the With Oracle VM Manager use Up to 6

ownership of the Deploy action to change the hours
the OMR and ownership of the OMR and depending
OMS virtual OMS virtual machines from the on system
machines. My Workspace private group to resources.
your desired group.
Note: Using the Deploy action
will create a copy of each
virtual machine, which doubles
the storage requirements.
Watch this space

Step 1: Download the Template

Step 1 will require a visit to http://edelivery.oracle.com/linux to register, search for and to download the Oracle VM Enterprise
Manager Grid Control Template.
1. Point a web browser to http://edelivery.oracle.com/linux as show in Figure 2 click the Continue button to proceed.
200 of 243 18.04.2011 16:35

2. From the Registration page enter a Full Name, Company Name, Email Address, select a Country from the drop down box,
accept the Agreement Terms by selecting the Agreement Terms check box, accept the Export Restrictions by selecting the
Export Restrictions check box and click Continue to proceed.
Figure 3 shows the Registration page.
3. From the Media Pack Search page, select Oracle VM Templates from the Select a Product Pack drop down menu. Then select
x86 32 bit from the Platform drop down menu and click the Go button to proceed as shown in Figure 4.
201 of 243 18.04.2011 16:35

4. As shown in Figure 5, from the Media Pack Search results page select the Oracle VM Enterprise Manager Grid Control
Template for Oracle VM radio button and click the Continue button to proceed to the Oracle VM Grid Control Template 1.0
Media Pack for x86 (32 bit) download page.
5. From the Oracle VM Grid Control Template 1.0 Media Pack for x86 (32 bit) download page click the Download button next
to each part of compressed files (14.81 GB total) to start the download.
Figure 6 shows the Oracle VM Grid Control Template 1.0 Media Pack for x86 (32 bit) download page.
202 of 243 18.04.2011 16:35

Watch this space

Step 2: Unzip and Concatenate Both Sets of Files

In Step 1 we downloaded two sets of files, the first set of files has both the OMR and OMS file set and the second file set is the Yum
repository.
List 1 shows the compressed 5.69 GB OMR/OMS file set.
V13510-01_1of4.zip
V13510-01_2of4.zip
V13510-01_3of4.zip
V13510-01_4of4.zip
List 2 shows the compressed 9.12 GB Yum repository file set.
V13511-01_1of5.zip
V13511-01_2of5.zip
V13511-01_3of5.zip
V13511-01_4of5.zip
V13511-01_5of5.zip
203 of 243 18.04.2011 16:35

In Step 2 we will run through the procedure to unzip and concatenate both files sets in the /OVS/seed_pool directory as well as to add
the yum.img file in the OMS vm.cfg file.
Note: The procedure is identical with a local Oracle VM OVS repository or a centralized Oracle VM OVS repository.
1. Place the zip template files in the /OVS/seed_pool directory as shown in the example:
/OVS/seed_pool/V13510-01_1of4.zip
2. From the Oracle VM Server console login as root and unzip the files using the unzip command as shown in the following example.
# unzip 'V*.zip'
Archive: V13510-01_1of4.zip
inflating: GC_Container.tar.gz.1of4
inflating: yum.tar.gz.1of5
Unzipping all of the file sets will generate the following tar files:
/OVS/seed_pool/GC_Container.tar.gz.1of4
/OVS/seed_pool/yum.tar.gz.1of5
3. Next we will concatenate the OMR/OMS file set. The next example shows how to concatenate the OMR/OMS file set.
# cat GC_Container.tar.gz.1of4 GC_Container.tar.gz.2of4 GC_Container.tar.gz.3of4 GC_Container.tar.gz.4of4 | tar xzvf -

GC_DB_TEMPLATE/
GC_DB_TEMPLATE/vm.cfg
GC_DB_TEMPLATE/README
GC_DB_TEMPLATE/system.img
GC_DB_TEMPLATE/db.img
GC_TEMPLATE/
GC_TEMPLATE/vm.cfg
GC_TEMPLATE/README
GC_TEMPLATE/gc.img
GC_TEMPLATE/system.img
README
The GC_Container contains the Oracle VM images for:
Oracle Enterprise 4 Update 5 for x86

Oracle Enterprise Manager Grid Control Release 10.2.0.4 for Linux x86
Oracle Enterprise Manager Grid Control Release 10.2.0.4 database repository for Linux x86
4. Next we will concatenate the Yum repository file set. The next example shows how to concatenate the Yum repository file set.
204 of 243 18.04.2011 16:35

# cat yum.tar.gz.1of5 yum.tar.gz.2of5 yum.tar.gz.3of5 yum.tar.gz.4of5 yum.tar.gz.5of5 | tar xzvf -

README
yum.img
The yum.img file contains a Yum repository populated with the full Linux distribution RPMs for:
Oracle Enterprise Linux Release 4 Update 6 for x86

Oracle Enterprise Linux Release 4 Update 6 for x86_64
Oracle Enterprise Linux Release 5 Update 1 for x86
Oracle Enterprise Linux Release 5 Update 1 for x86_64
5. The final step is to add the yum.img virtual disk to the OMS virtual machine. We will modify the /OVS/seed_pool/GC_TEMPLATE
/vm.cfg file to include the Yum repository virtual disk. The next example shows the default unmodified OMS vm.cfg file.
# OracleVM XEN config file

name = "EMGC"
memory = "2048"
disk = [ 'file:/OVS/seed_pool/GC_TEMPLATE/system.img,hda,w', 'file:/OVS/seed_pool/GC_TEMPLATE/gc.img,hdb,w']
vif = [ '', ]
bootloader="/usr/bin/pygrub"
vcpus=2
vfb = [ 'type=vnc,vncunused=1,vnclisten=0.0.0.0' ]
Use your favorite text editor for example vi to add “'file:/OVS/seed_pool/GC_

TEMPLATE/yum.img,hdd,w'” to the end of the disk attribute in the vm.cfg file as shown in the following example.
# OracleVM XEN config file

name = "EMGC"
memory = "2048"
disk = [ 'file:/OVS/seed_pool/GC_TEMPLATE/system.img,hda,w',
'file:/OVS/seed_pool/GC_TEMPLATE/gc.img,hdb,w', 'file:/OVS
/seed_pool/GC_
TEMPLATE/yum.img,hdd,w']
vif = [ '', ]
bootloader="/usr/bin/pygrub"
vcpus=2
vfb = [ 'type=vnc,vncunused=1,vnclisten=0.0.0.0' ]
Watch this space

Step 3: Create the OMR and OMS Virtual Machine Templates

In Step 2 we walked through the procedure to unzip and concatenate both files sets and we added an entry for the yum.img file in the
OMS vm.cfg file. In Step 3 we will use Oracle VM Manager to create an OMR and OMS virtual machine template.
1. Login to Oracle VM Manager. Enter the desired user name and password and click the Login button to proceed as shown in
Figure 7.
205 of 243 18.04.2011 16:35

2. From the Virtual Machine home page click the Resources tab to access the Virtual Machine Template page. From the Virtual
Machine Template page click the Import button to start the importation process.
Figure 8 shows the Virtual Machine Template page.
3. From the Source page select the Select from Server Pool (Discover and register) radio button and click Next as shown in
Figure 9.
206 of 243 18.04.2011 16:35

4. From the General Information page we need to enter the following data:
a. Select the desired server pool from the Server Pool Name drop down list.
b. Select the CG_DB_Template from the Virtual Machine Template Name drop down list.
c. Enable High Availability check box if desired, HA will need to be setup in advance to enable HA.
d. Select Oracle Enterprise Linux 4 from the Operating System drop down list.
e. Enter root in the Virtual Machine User Name text box.
f. Enter ovsroot in the Virtual Machine Password text box.
g. Enter a Description for the virtual machine (optional)
h. Click Next to proceed to the Confirmation page.
Figure 10 shows the General Information page.
5. From the Confirm Information page click the Confirm button to proceed.
Figure 11 shows the Confirm Information page.
6. We have successfully created the CG_DB_Template virtual machine template, which is in a pending status. Templates that are in the
pending status are not available to Oracle VM Manager users until they have been approved. Next we will approve the
CG_DB_Template by selecting the CG_DB_Template and clicking the Approve button as shown in Figure 12.
207 of 243 18.04.2011 16:35

7. From the Virtual Machine Templates page click the Approve button to approve the template as shown in Figure 13.
8. The CG_DB_Template has been successfully approved and its Status has changed to Active as shown in Figure 14.
208 of 243 18.04.2011 16:35

Next we will repeat the proceeding procedure for the GC_TEMPLATE.
1. From the Virtual Machine home page click the Resources Tab to access the Virtual Machine Template page. From the Virtual
Machine Template page click the Import button to start the importation process.
Figure 15 shows the Virtual Machine Template page.
2. From the Source page select the Select from Server Pool (Discover and register) radio button and click Next as shown in
Figure 16 to proceed.
209 of 243 18.04.2011 16:35

3. From the General Information page we need to enter the following data:
a. Select the desired server pool from the Server Pool Name drop down list.
b. Select the CG_Template from the Virtual Machine Template Name drop down list.
c. Select the Enable High Availability check box if desired. HA will need to be setup in advance to enable HA.
d. Select Oracle Enterprise Linux 4 from the Operating System drop down list.
e. Enter root in the Virtual Machine User Name text box.
f. Enter ovsroot in the Virtual Machine Password text box.
g. Enter a Description for the virtual machine (optional).
h. Click Next to proceed to the Confirmation page.
Figure 17 shows the General Information page.
4. From the Confirm Information page click the Confirm button to proceed.
Figure 18 shows the Confirm Information page.
210 of 243 18.04.2011 16:35

5. We have successfully created the CG_DB_Template virtual machine template, which is in the pending status. Templates that are in
the pending status are not available to Oracle VM Manager users until they have been approved. Next we will approve the
CG_DB_Template by selecting the CG_DB_Template radio button and clicking the Approve button as shown in Figure 19.
6. From the View Virtual Machine Template page click the Approve button to approve the template as shown in Figure 20.
211 of 243 18.04.2011 16:35

7. The CG_Template has been successfully approved and is now in the Active State as shown in Figure 21.
Watch this space

Step 4: Create the OMR and OMS Virtual Machines
212 of 243 18.04.2011 16:35


In Step 3 we used Oracle VM Manager to create an OMR and OMS template. Next in Step 4 we will use Oracle VM Manager to create
the OMR and OMS virtual machines.
1. From the Virtual Machines page click one of the two Create Virtual Machine buttons to proceed.
Figure 22 shows the Virtual Machines page with two Create Virtual Machine buttons.
2. From the Creation Method page select the Create virtual machine based on virtual machine template radio button, then
click Next to proceed as shown in Figure 23.
3. From the Server Pool page select the desired Server Pool radio button and accept the default Preferred Server setting. Click
Next to proceed as shown in Figure 24.
213 of 243 18.04.2011 16:35

4. From the Source page as shown in Figure 25 select the CG_DB_TEMPLATE radio button and click Next to proceed.
5. From the Virtual Machine Information page enter the following data:
a. Virtual Machine Name
b. Console Password
c. Confirm Console Password
d. Select the Enable High Availability check box if desired. HA will need to be configured in advance to enable HA.
e. Select the desired Network Interface Card and Bridge.
i. The Network Interface Card can be deleted by selecting the Delete radio button.
ii. The Add Row radio button allows us to create up to three Network Interface Cards
iii. The Bridge drop down list will allow us to select any configured bridges.
f. Click Next to proceed.
Figure 26 shows the Virtual Machine Information page.
6. As shown in Figure 27, from the Confirm Information page click Confirm to proceed.
214 of 243 18.04.2011 16:35

7. The web browser will automatically redirect to the Virtual Machine page. From the Virtual Machine page validate the status of
the new virtual machine, as Creating. When the creation process is completed the status will change to Powered Off.
Figure 28 shows the new virtual machine status as Creating.
8. From the Virtual Machines page click one of the two Create Virtual Machine buttons to proceed.
Figure 29 shows the Virtual Machines page with two Create Virtual Machine buttons.
215 of 243 18.04.2011 16:35

9. From the Creation Method page select the Create virtual machine based on virtual machine template radio button, then
click Next to proceed as shown in Figure 30.
10. From the Server Pool page select the desired Server Pool radio button and accept the default Auto Preferred Server option.
Click Next to proceed as shown in Figure 31.
11. From the Source page as shown in Figure 32, select the CG_TEMPLATE radio button and click Next to proceed.
216 of 243 18.04.2011 16:35

12. From the Virtual Machine Information page enter the following data:
a. Virtual Machine Name
b. Console Password
c. Confirm Console Password
d. Select the Enable High Availability check box if desired. HA will need to be configured in advance to enable HA.
e. Select the desired Network Interface Card and Bridge.
i. The Network Interface Card can be deleted by selecting the Delete radio button.
ii. The Add Row radio button allows us to create up to three Network Interface Cards
iii. The Bridge drop down list will allow us to select any configured bridges.
f. Click Next to proceed.
Figure 33 shows the Virtual Machine Information page.
13. As shown in Figure 34, from the Confirm Information page click the Confirm button to proceed.
217 of 243 18.04.2011 16:35

14. The web browser will automatically redirect to the Virtual Machine page. From the Virtual Machine page validate the status of
the new virtual machine, as Creating. When the creation process is completed the status will change to Powered Off.
Figure 35 shows the new virtual machine status as Creating.
15. When the creation process is completed the status will change to Powered Off as shown in Figure 36.
218 of 243 18.04.2011 16:35

Watch this space

Step 5: Start the OMR and OMS Virtual Machines and Enter
Setup Values

In Step 4 we used Oracle VM Manager to create the OMR and OMS virtual machines from the templates we created in Step 3. In Step
5 we will used Oracle VM Manager to start the virtual machine that was created with the GC_DB_TEMPLATE, access the virtual
machine’s console and enter values for a number of setup options. Once we have successfully entered the setup values for the OMR
virtual machine we will start the OMS virtual machine (created from the GC_TEMPLATE template), we will access the virtual
machine’s console and will enter values for a number of setup options.
Note: Its necessary to start and configure the OMR virtual machine first followed by the OMS virtual machine.
List 3 and 4 show the default user names and password for the OMR and OMS virtual machines and the virtual machine console
password.
List 3 shows the default user name and passwords for the virtual machines.
User Name: root

Password: ovsroot
User Name: oracle

Password: oracle
List 4 shows the default password used to access the virtual machine console from Oracle VM Manager.
Password: oracle
Prerequisites
Before the OMR and OMS virtual machines are started ensure that your DNS server has entries for the OMR and OMS virtual
machines and that you have selected and documented the following setup options:
Select a static IP address

Select the network mask
Select a hostname (FQDN)
Select the default gateway
Select the DNS server IP address
Select a single password for the SYS, SYSTEM, SYSMAN and DBSNMP database accounts
*The password must be at least 6 characters in length and contain at least one numeral character.
Setup Tips:
After the virtual machine is started, if you do not enter a password for the database accounts, the setup will fail and you will need
to restart the virtual machine.
After the virtual machine is started, if the setup hangs towards at the end of the setup process, allow up to 20 minutes for the
process to automatically complete. Once the setup is completed you will be presented with the logon screen.
After the virtual machine is started, if the setup is not completed, the next time the virtual machine starts the setup process will
start over from the beginning.
The following section will run through the steps to start and enter the setup data for the OMR virtual machine.
1. Login to Oracle VM Manager and select the OMR virtual machine and click the Power On button as shown in Figure 37. The
Status will change from Powered Off to Initializing.
219 of 243 18.04.2011 16:35

2. Next click the Refresh button until the status changes to Running. Once the status is Running click the Console button to access
the OMR virtual machine console as shown in Figure 38.
3. Enter the default password oracle and click the OK button to proceed.
Figure 39 shows the OMR virtual machine console where the setup options are entered.
220 of 243 18.04.2011 16:35

4. Enter the IP address and press Enter.

5. Enter the netmask address and press Enter.
6. Enter the hostname (FQDN) and press Enter.
7. Enter the default gateway and press Enter.
8. Enter the DNS server IP address and press Enter.
9. Enter the full pathname of the local bin directory: [/usr/local/bin]:
If you want to use the default directory, leave this field empty and press Enter.
10. Enter a password for database accounts SYS, SYSTEM, SYSMAN and DBSNMP.
The password must be at least 6 characters in length that contains at least one numeral character. Enter the password and press
Enter.
11. Once presented with the following banner press any key on the keyboard to complete the setup process.
======================================================================
----------------------------------------------------------------------
The following section will run through the steps to start and enter the setup data for the OMS virtual machine.
1. Login to Oracle VM Manager and select the OMS virtual machine and click the Power On button as shown in Figure 40. The
Status will change from Powered Off to Initializing.
2. Next click the Refresh button until the status changes to Running. Once the status is Running click the Console button to access
the OMS virtual machine console as shown in Figure 41.
221 of 243 18.04.2011 16:35

Enter the default password oracle and click the OK button to proceed.
Figure 42 shows the OMS virtual machine console where the setup options are entered.
3. Enter the IP address and press Enter.

4. Enter the netmask address and press Enter.
5. Enter the hostname (FQDN) and press Enter.
6. Enter the default gateway and press Enter.
7. Enter the DNS server IP address and press Enter.
Enter the pathname of the local bin directory. If you want to use the default directory, leave this field empty and press Enter.
9. Enter repository database hostname and press Enter.
10. Enter the password you created for the OMR repository SYSMAN user and press Enter.
11. Enter new password for OMA Secure Registration and press Enter.
13. If any of the dbhome, oraenv, coraenv files already exist, you are prompted to overwrite them. At each prompt, enter y and press
Enter to overwrite the file.
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
222 of 243 18.04.2011 16:35

The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)

[n]: y
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
======================================================================
Watch this space
Step 6: Installing the OMA Client on the OMR Virtual

Machine

In Step 5 we used Oracle VM Manager to start both the OMR and OMS virtual machines, access the virtual machine’s console and
enter values for a number of setup options. Next in Step 6 we will install OMA using the installagent.sh script.
The following procedure will walk through the installation of the Oracle Management Agent (OMA) on the OMR virtual machine.
1. Log into the OMR virtual machine via the Oracle VM Manager console or via an SSH client as root using the default ovsroot
password.
2. Run the OMA installation script named installagent.sh which is located in the /u01/scripts directory, e.g.
#cd /u01/scripts
#./installagent.sh
3. Enter the Agent Registration password and press Enter.
4. Enter the OMR SYSMAN password and press Enter.
6. If any of the dbhome, oraenv, coraenv files already exist, you are prompted to overwrite them. At each prompt, enter y and press
Enter to overwrite the file.
[n]: y
[n]: y
[n]: y
======================================================================
The next example shows a complete OMA install using the /u01/scripts/installagent.sh script.
login as: root

root@192.168.4.13's password: ovsroot
Last login: Tue Oct 21 16:13:19 2008
# /u01/scripts/installagent.sh
==================================================================
Start installing Management Agent 10.2.0.4 X86
==================================================================
Extracting the source files...
Provide Management Service hostname that the Management Agent willl communicate with.
Enter OMS hostname:v45oms10g1

Provide the Agent Registration password so that the Management Agent can communicate with Secure Management Service.
Enter Agent Registration Password:

Cloning agent...
Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2008-10-21_05-44-30PM. Please wait ...Oracle Universal Installer,
223 of 243 18.04.2011 16:35

Version 10.2.0.4.0 Production

Copyright (C) 1999, 2007, Oracle. All rights reserved.
You can find a log of this install session at:

/u01/app/oraInventory/logs/cloneActions2008-10-21_05-44-30PM.log
.................................................................................................... 100% Done.
Installation in progress (Tuesday, October 21, 2008 5:44:40 PM EDT)

..................................................................... 69% Done.
Install successful
Linking in progress (Tuesday, October 21, 2008 5:44:46 PM EDT)

. 70% Done.
Link successful
Setup in progress (Tuesday, October 21, 2008 5:45:17 PM EDT)

.................... 100% Done.
Setup successful
End of install phases.(Tuesday, October 21, 2008 5:45:24 PM EDT)

Starting to execute configuration assistants
The following configuration assistants have not been run. This can happen for following reasons - either root.sh is to be run before
config or Oracle Universal Installer was invoked with the -noConfig option.
--------------------------------------
The "/u01/app/oracle/product/agent10g/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped
or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this
script with passwords (if any) before executing the same.
The "/u01/app/oracle/product/agent10g/cfgtoollogs/configToolAllCommands" script contains all commands to be executed by the
configuration assistants. This file may be used to run the configuration assistants outside of OUI. Note that you may have to update this
script with passwords (if any) before executing the same.
--------------------------------------
The cloning of cloneagent10g was successful.
Please check '/u01/app/oraInventory/logs/cloneActions2008-10-21_05-44-30PM.log' for more details.
Running /u01/app/oracle/product/agent10g/root.sh...
Running Oracle10 root.sh script...
The following environment variables are set as:

ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/agent10g
Enter the full pathname of the local bin directory: [/usr/local/bin]:

[n]: y
Copying dbhome to /usr/local/bin ...
[n]: y
Copying oraenv to /usr/local/bin ...
[n]: y
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
Enter password of database account SYSMAN:

Oracle Enterprise Manager 10g Release 4 Grid Control 10.2.0.4.0.
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
Agent is already stopped... Done.
Securing agent... Started.
Requesting an HTTPS Upload URL from the OMS... Done.
Requesting an Oracle Wallet and Agent Key from the OMS... Done.
Check if HTTPS Upload URL is accessible from the agent... Done.
Configuring Agent for HTTPS in CENTRAL_AGENT mode... Done.
EMD_URL set in /u01/app/oracle/product/agent10g/sysman/config/emd.properties
Securing agent... Successful.
Stopping the agent using /u01/app/oracle/product/agent10g/bin/emctl stop agent
224 of 243 18.04.2011 16:35


Agent is Not Running
Running agentca using /u01/app/oracle/product/agent10g/oui/bin/runConfig.sh ORACLE_HOME=/u01/app/oracle/product/agent10g
ACTION=Configure MODE=Perform RESPONSE_FILE=/u01/app/oracle/product/agent10g/response_file RERUN=TRUE
INV_PTR_LOC=/etc/oraInst.loc COMPONENT_XML={oracle.sysman.top.agent.10_2_0_1_0.xml}
Perform - mode is starting for action: Configure
Perform - mode finished for action: Configure
You can see the log file: /u01/app/oracle/product/agent10g/cfgtoollogs/oui/configActions2008-10-21_05-46-19-PM.log

Stopping agent ... stopped.
Stopping crond: [ OK ]
Starting crond: [ OK ]
Starting agent ....... started.
---------------------------------------------------------------
EMD upload completed successfully
===================================================================================
Management Agent 10.2.0.4 X86 cloning completed successfully.
The following env variables are set for user oracle:
ORACLE_BASE = /u01/app
AGENT_HOME = /u01/app/oracle/product/agent10g
===================================================================================

#
Watch this space

Step 7: Access the Enterprise Manager Grid Control Portal

In Step 6 we installed OMA on the OMS virtual machine using the installagent.sh script. Next we will access the Enterprise Manager
Grid Control portal using the sysman account and the password that was selected during Step 5.
Once logged into the Enterprise Manager Grid Control portal please refer to http://download.oracle.com/docs/cd/B16240_01
/welcome.html for the Enterprise Manager Grid Control configuration and setup documentation.
1. Point a web browser to the FQDN of the OMS virtual machine i.e. http://FQDN:4889/em/ to access the Enterprise Manager Grid
Control portal. From the Login page enter sysman as the User Name and the password that was selected during Step 5. Then press
the Login button to proceed as shown in Figure 43.
225 of 243 18.04.2011 16:35

2. Figure 44 shows the Enterprise Manager Home page.
Watch this space

Step 8: Change the Ownership of the OMR and OMS Virtual

Machines from the My Workspace Group

In Step 7 we accessed the Enterprise Manager Grid Control portal using the sysman account and the password that was selected
during Step 5. Next we will change the ownership of the OMR and OMS virtual machines from the default My Workspace group to the
operations group.
Note: You will need to have created groups in Oracle VM Manager to complete this step.
By default when a virtual machine is created in Oracle VM Manager, it’s owned by the administrator that created the virtual machine.
By default when a virtual machine is created it will be placed in the My Workspace group, which makes the virtual machine visible
exclusively to the virtual machine owner. For example, both the OMR and OMS virtual machines from the above examples are only
visible in Oracle VM Manager by the administrator that created the virtual machines. To allow other Oracle VM Manager users to
view and manage the virtual machine the Deploy action is used to change the ownership from the My Workspace group to a different
group. To use the Deploy action the virtual machine must be powered off.
226 of 243 18.04.2011 16:35

The next section will walk through the procedure to change the ownership of the OMR virtual machine.
1. From Oracle VM Manager ensure that the OMR virtual machine is Powered Off and then select the OMR virtual machine.
Once the OMR virtual machine is selected click the More Actions drop down menu and select the Deploy action. Then click the
Go button to start the Deploy action as shown in Figure 45.
2. From the Virtual Machine Information page enter the Virtual Machine Name in the text box, select the desired group from the
Group Name drop down list and select the desired server pool name from the Server Pool Name drop down list as shown in Figure
46. Click Next to proceed.
3. From the Confirm Information page click the Confirm button to start the deploy action as shown in Figure 47.
4. As shown in Figure 48 the status of the My Workspace virtual machine changed from Powered Off to Deploying and the
deployed virtual machine status is Creating.
227 of 243 18.04.2011 16:35

When the Deploy action completes, each virtual machine’s status changes to Powered Off. The deploy action results in two identical
virtual machines with different group membership. The original virtual machine remains in the My Workspace group and the
Deployed virtual machine will be in the group selected from the Virtual Machine Information page. Repeat the process for the OMS
virtual machine to change the group membership.
Watch this space
Oracle VM Centralized Logging

Oracle VM for x86 Troubleshooting Made Easier
Author Martin Foster
Editor Roddy Rodstein
By default Oracle VM 2.x logs all events locally. Logging events locally makes troubleshooting Oracle VM server pool issues a
challenge, because different log information is being echoed to different local log files. In this chapter, we will walk through a
centralized logging configuration for Oracle VM that makes troubleshooting Oracle VM much easier when compared to the default
local Oracle VM logging configuration.
As of Oracle VM 2.2, the Oracle VM agent's logging functionality is customizable by using the Python Logger class configuration file.
The Oracle VM Manager application runs on OC4J, a JSP container that has log4j style configuration capabilities. Both log4j and
python's logger do support logging to syslog.
Change Log
Revision Change Description Updated Date

By
1 Draft Release Roddy 03/28/11

Rodstein
Table of Contents
Change Log
Upgrade the Oracle VM Server and Manager Local Syslog Daemon
Oracle VM Server Syslog Configuration
...Example /etc/ovs-agent/logger_server.ini file
How to Configure Rsyslog on Oracle VM Server
...Example /etc/rsyslog.conf file
...Example /etc/sysconfig/rsyslog file
How to Configure the Central Log Host
...Example /etc/sysconfig/rsyslog file
The Oracle VM Manager Syslog Configuration
...OC4J Logging
228 of 243 18.04.2011 16:35

...Oracle VM Manager Logging Configuration Files

…...Example /etc/rsyslog.conf file
...How to Make Rsyslog Read the oc4j.log file
...Oracle VM Manager /etc/rsyslog.conf and Adding imfile Support
…...Example /etc/rsyslog.conf file
...Empty the Existing Oracle VM Manager oc4j.log
...Configure /etc/rsyslog.conf to Forward local2 Events
Configuring the Central Log Host
...Example central log host /etc/rsyslog.conf file
How to Configure the Oracle VM Server's Logrotation
Upgrade the Oracle VM Server and Manager Local Syslog Daemon

The first step to configure centralized logging for Oracle VM is to upgrade the local syslog daemon to the Oracle Linux 5.5 rsyslog
package on the Oracle VM servers and on the Oracle VM Manager host.
Note: The default Oracle VM 2.x server configuration does not have rsyslog.
The next three steps show how to install and configure rsyslog on an Oracle VM 2.x server and on an Oracle VM Manager x86 or
x86-x64 host:
1. Download and install the Oracle Linux 5.5 rsyslog rpm (3.22 at time of writing) using wget and the rpm programs.
Oracle VM Server: The Oracle VM server will always use the i386 RPM regardless of the hardware platform, i.e. both x86 or x86-x64
servers both use a x86 dom0 and will use the i386 RPM package.
Oracle VM Manager: Depending on the hardware/OS platform for your Oracle VM Manager host, use the i386 RPM package for x86
or the x86_64 package x64.
The next two examples show how to download and install the rsyslog rpm package for the i386 and x86-64 platforms using wget and
the rpm programs.
I386
# wget http://public-yum.oracle.com/repo/EnterpriseLinux/EL5/5/base/i386/rsyslog-3.22.1-3.el5.i386.rpm
# rpm -Uvh rsyslog-3.22.1-3.el5.i386.rpm
x86-64
# wget http://public-yum.oracle.com/repo/EnterpriseLinux/EL5/5/base/x86_64/rsyslog-3.22.1-3.el5.x86_64.rpm
# rpm -Uvh rsyslog-3.22.1-3.el5.x86_64.rpm
2. In the next example, we use the syslog configuration file for rsyslog. We also disable syslog and enable rsyslog:
# cp /etc/syslog.conf /etc/rsyslog.conf
# chkconfig syslog off
# chkconfig rsyslog on
# service syslog stop
# service rsyslog start
3. Next, check /var/log/messages to validate that rsyslog has started. For example, type “tail /var/log/messages”
Oracle VM Server Syslog Configuration

The /etc/ovs-agent/logger_server.ini file stores the logger class configuration and is read when the Oracle VM agent starts.
Note: For more information please refer to: http://docs.python.org/library/logging.html#sysloghandler
The next list shows the changes that will be made to the ovs-agent Python logger:
1. Maintain the various handlers that Oracle uses, for consistency with Oracle support.
2. Propagate all handers to the parent (root) handler; all information logged by the ovs-agent will be available at this handler.
3. Set the log level to NOTSET, which is everything (more than DEBUG).
4. Forward logs to local syslog over udp/514 (default syslog port).
5. Set all loggers to "propagate=1", so they forward up logs to their parent handlers.
6. By default performance and macip logging doesn't propagate up. We need "propagate=1" for centralization.
7. Write to unix socket /dev/log, which must be created by rsyslog!
Note: We don't use the localhost 514/udp destination because this will create a message from a hostname of localhost or 127.0.0.1,
which is of no use for centralized logging. By writing to the socket, the syslog daemon appends its hostname, which is necessary for
229 of 243 18.04.2011 16:35

centralized logging.
8. Send all to syslog by altering the root handler

9. Create a separate formatter that matches the syslog "name: message" style.
Example /etc/ovs-agent/logger_server.ini file

The following example shows a logger_server.ini file that meets the above prerequisites:
# cat /etc/ovs-agent/logger_server.ini
[loggers]
keys=root,performance,operation,query
[logger_root]
handlers=root
level=NOTSET
[logger_operation]
qualname=ovs.operation
handlers=operation
level=DEBUG
propagate=1
[logger_performance]
qualname=ovs.performance
handlers=performance
level=DEBUG
# default propagate is 0
propagate=1
[logger_query]
qualname=ovs.query
handlers=query
level=DEBUG
propagate=1
[logger_macip]
qualname=ovs.macip
handlers=macip
level=DEBUG
# default propagate is 0
propagate=1
;----------------------------------------------------------------------
[handlers]
keys=root,performance,operation,query,macip
[handler_root]
class=handlers.RotatingFileHandler
;append to log file, and file size is 1M with 3 archive files
args=("%(log.dir)s/ovs_root.log", "a", 1024*1024, 3)
formatter=ovs
level=DEBUG
# Syslog handler
# - log to local syslog daemon, which can forward to central loghost
# - using unix socket, which must match the socket created by the syslog daemon
# - could use UDP to localhost, which loses the originating host information
# (get messages tagged with an IP that makes no sense centrally, like 127.0.0.1
# for the localhost config)
# - SysLogHandler unix socket: args=('/dev/log', handlers.SysLogHandler.LOG_LOCAL3)
# - SysLogHandler to 514/udp: (('localhost', handlers.SYSLOG_UDP_PORT),
handlers.SysLogHandler.LOG_LOCAL3)
# - if the level is set on the syslog handler, all messages sent through that handler
# inherit this level. Preferable to set level on a per-logger basis
# - if level inherited by the loggers, then a simple formatter can be used that best
matches
# the expectation of syslog parsers. see formatters section for details. eg:
# format=%(name)s: %(message)s
#
# address: /dev/log unix socket
230 of 243 18.04.2011 16:35

# facility: LOG_LOCAL3
# level: do not set at the handler level
# ref: http://docs.python.org/library/logging.html#sysloghandler
#
class=handlers.SysLogHandler
#args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_LOCAL3)
args=('/dev/log', handlers.SysLogHandler.LOG_LOCAL3)
formatter=syslog
[handler_operation]
args=("%(log.dir)s/ovs_operation.log", "a", 1024*1024, 3)
formatter=ovs
level=DEBUG
[handler_performance]
args=("%(log.dir)s/ovs_performance.log", "a", 1024*1024, 3)
formatter=ovs
level=DEBUG
[handler_query]
args=("%(log.dir)s/ovs_query.log", "a", 1024*1024, 3)
formatter=ovs
level=DEBUG
[handler_macip]
args=("%(log.dir)s/ovs_macip.log", "a", 1024*1024, 2)
formatter=ovs
level=DEBUG
;----------------------------------------------------------------------
[formatters]
keys=ovs,syslog
[formatter_ovs]
class=logging.Formatter
format=%(asctime)s %(levelname)s=> %(message)s
datefmt="%Y-%m-%d %H:%M:%S"
# formatter for syslog

# - no datefmt required, syslog tags with event time
# - if level is not specified by the handler, then we do not need to add
# it to the formatter
# - formatter options
# with level in syslog message field: format=%(name)s: %(levelname)s %(message)s
# closest to traditional syslog: format=%(name)s: %(message)s
#
[formatter_syslog]
class=logging.Formatter
format=%(name)s: %(message)s
How to Configure Rsyslog on Oracle VM Server

The next list shows the steps to configure rsyslog on an Oracle VM server as root:
1. Create a UNIX syslog socket at /dev/log.

2. Eliminate the local3 channel from /var/log/messages to keep the ovs-agent logs in one place and not to spread the ovs-agent logs
to multiple logfiles.
3. Route all local3 facility messages to a file, /var/log/ovs-agent/all.log.
4. Route all local3 facility messages to a central log host, over port 601/tcp.
The following /etc/rsyslog.conf file example contains rsyslog specific configuration elements. The "@@" remote forwarding
terminology means "use TCP", whereas "@" means "use UDP".
Enter your system details in the following sections:
231 of 243 18.04.2011 16:35

<MAIL PROXY HOST NAME GOES HERE>

<FQDN OF CENTRAL LOG HOST>
Example /etc/rsyslog.conf file
# cat /etc/rsyslog.conf
# ----------------------------- Queues (required for forwarding)
-----------------------------
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
# ----------------------------- Modules & Functions -----------------------------

# module: kernel logs, klogd replacement
$ModLoad imklog
# module: MARK messages (usually disable)

$ModLoad immark
# module: input udp

# NOTE: documetation says InputUDPServerRun, that's wrong
$ModLoad imudp
$UDPServerAddress *
$UDPServerRun 514
# module: input tcp (works fine)

$ModLoad imtcp
$InputTCPServerRun 601
# module: input socket

$ModLoad imuxsock
$InputUnixListenSocketHostName /dev/log
$InputUnixListenSocketIgnoreMsgTimestamp on
# ----------------------------- Templates -----------------------------

# Use traditional timestamp format
# eg: "Jan 1 15:00:01 "
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# RFC5424 format
# also known as RSYSLOG_SyslogProtocol23Format, and
draft-internet-ietf-syslog-protocol-23 (now RFC5424)
# the "1" in "<%PRI%>1 " denotes syslog protocol version 1, as per the RFC
# eg: <21>1 2011-01-01T16:09:05+00:00 <MAIL PROXY HOST NAME GOES HERE> perdition 14185 - -
$template RFC5424FMT,"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID%
%STRUCTURED-DATA% %msg%\n"
# ----------------------------- Destinations -----------------------------

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
kern.* /var/log/kernel
# Log anything (except mail) of level info or higher.

# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local3.none /var/log/messages
# The authpriv file has restricted access.

authpriv.* /var/log/secure
# MAIL: Log locally & forward a copy to syslog1-syd.internal for further analysis
mail.*
-/var/log/mail/mail.log;RFC5424FMT
mail.*
@@syslog1-<FQDN OF CENTRAL LOG HOST>:601
# Log cron stuff

cron.* /var/log/cron
232 of 243 18.04.2011 16:35

# Everybody gets emergency messages

#*.emerg *
# Save news errors of level crit and higher in a special file.

#uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log

local7.* /var/log/boot.log
# All, save mail

*.debug;mail.none;authpriv.none;cron.none -/var/log/debug
# Oracle VM: Log locally & forward a copy to syslog1-syd.internal for further analysis
local3.*
-/var/log/ovs-agent/all.log;RFC5424FMT
local3.*
@@syslog1-<FQDN OF CENTRAL LOG HOST>:601
Example /etc/sysconfig/rsyslog file

The next example shows a rsyslog file.
# cat /etc/sysconfig/rsyslog
# Options to syslogd
# -m 0 disables 'MARK' messages.
# -rPortNumber Enables logging from remote machines. The listener will listen to the specified port.
# -x disables DNS lookups on messages received with -r
# See syslogd(8) for more details
SYSLOGD_OPTIONS="-c3 "
# Options to klogd
# -2 prints all kernel oops messages twice; once for klogd to decode, and
# once for processing with 'ksymoops'
# -x disables all klogd processing of oops messages entirely
# See klogd(8) for more details
KLOGD_OPTIONS="-x"
Next, enable the changes by reloading rsyslog by typing, “service rsyslog reload”. After rsyslog has been reloaded, there should be an
empty file named all.log in the /var/log/ovs-agent/ directory.
Next, reload the Oracle VM agent by typing “service ovs-agent stop --disable-nowayout; service ovs-agent start” as shown in the next
example.
# service ovs-agent stop --disable-nowayout; service ovs-agent start

OVSAgentServer shutdown...
OVSAgentServer forced stop.
OVSAgentServer is now starting...
OVSAgentServer started.
After the Oracle VM agent has been reloaded, messages will appear in the local /var/log/ovs-agent/all.log file, as shown in the next
example.
<158>1 2011-01-25T01:03:03.245088+00:00 virtual5-syd root - - - OVSAgentServer start serving!

<158>1 2011-01-25T01:03:08.633781+00:00 virtual5-syd ovs.operation - - - clusterm_init_sr:
success.
<158>1 2011-01-25T01:03:08.658913+00:00 virtual5-syd ovs.operation - - - cluster_init_sr:
success.
<158>1 2011-01-25T01:03:10.863599+00:00 virtual5-syd ovs.operation - - - ha_set_shutdown_mode:
remove shutdown mode flag.
<158>1 2011-01-25T01:03:14.944380+00:00 virtual5-syd ovs.performance - - -
sys_vmstat_info:cpu_idle=0.99;mem_free=427
…
Note: If you see "localhost.localdomain" or "127.0.0.1" instead of the hostname in the all.log file, use the unix socket instead of udp in
configuring syslog for the Oracle VM agent.
How to Configure the Central Log Host

The next list shows the configuration steps to configure the central log host.
1. Receive remote syslog over port 601/tcp

2. Write local3 messages to /var/log/ovs-agent/all.log
3. Keep local3 message from other major logfiles
233 of 243 18.04.2011 16:35

Note: For the /etc/rsyslog.conf file, please see the Centralized Log Host full rsyslog.conf section.
Example /etc/sysconfig/rsyslog file

The next example shows the /etc/sysconfig/rsyslog file with the above prerequisites.
# cat /etc/sysconfig/rsyslog
# Options to syslogd
# -m 0 disables 'MARK' messages.
# -rPortNumber Enables logging from remote machines. The listener will listen to the specified port.
# -x disables DNS lookups on messages received with -r
# See syslogd(8) for more details
#SYSLOGD_OPTIONS="-c3 -x -m 0 -r514 -t601,4096" # syslog compat
SYSLOGD_OPTIONS="-c3 " # rsyslog
# Options to klogd
# -2 prints all kernel oops messages twice; once for klogd to decode, and
# once for processing with 'ksymoops'
# -x disables all klogd processing of oops messages entirely
# See klogd(8) for more details
KLOGD_OPTIONS="-x"
Next, reload the rsyslog service by typing, “service rsyslog reload”. After the rsyslog service has been reloaded, the central log host
will receive messages to /var/log/ovs-agent/all.log.
Notes: The rsyslog configuration must allow inbound 601/tcp. Check the central log host firewall to ensure that inbound 601/tcp is
enabled. The central log host's /var/log/ovs-agent directory must exist. If SElinux is in use, it must have the "user_u:object_r:var_log_t"
context
The Oracle VM Manager Syslog Configuration

OC4J uses java.util.logging, which cannot directly log to syslog. To log Oracle VM Manager events centrally, we will use rsyslog's
"imfile" module to read in the existing oc4j logfile and send it to the central log host line by line.
OC4J Logging
The Oracle VM Manager application is a J2EE Web application, running in an OC4J container. Assuming OC4J is installed to /opt/oc4j,
then the default logging is configured via the
“/opt/oc4j/j2ee/home/config/j2ee-logging.xml” file.
Oracle VM Manager Logging Configuration Files

The deployment descriptors for Oracle VM Manager application are installed by default to:
/opt/ovs-manager-2.2/bin/default-server.xml (the non-ssl server)

/opt/ovs-manager-2.2/bin/secure-server.xml (the ssl server)
By default the deployment descriptors both point to the j2ee-logging.xml configuration file. So we only need to edit the
j2ee-logging.xml configuration file.
How to Make Rsyslog Read the oc4j.log file

The default logfile for OC4J is in the /var/log/ovm-manager/oc4j.log file. Our configuration will read the /var/log/ovm-manager/oc4j.log
file to the local2/debug priority with the rsyslog imfile module.
Note: The configuration will result in the lose of the event severity.
Oracle VM Manager /etc/rsyslog.conf and Adding imfile Support

Next, we will add the “input file” module to /etc/rsyslog.conf. This configuration will write the state file to “$WorkDirectory”.
Note: The full imfile module documentation can be found here: http://www.rsyslog.com/doc/imfile.html
234 of 243 18.04.2011 16:35

Example /etc/rsyslog.conf file
# module: input file
# - emits each line of the given file to syslog, for apps that don't do syslog
# - keeps track of position, file rotation
$ModLoad imfile
$InputFileName /var/log/ovm-manager/oc4j.log
$InputFileTag OVM:
$InputFileStateFile state-oc4j
$InputFileSeverity debug
$InputFileFacility local2
$InputRunFileMonitor
Empty the Existing Oracle VM Manager oc4j.log

When the imfile module starts, it will read every line in the existing oc4j.log file, which is too much information for the imfile module
to start. Empty the oc4j.log file by typing “cp /dev/null /var/log/ovm-manager/oc4j.log”.
Configure /etc/rsyslog.conf to Forward local2 Events

Next, add the following lines to the /etc/rsyslog.conf file to forward local2 events to the central log host. Enter your central log host
FQDN in the <CENTRAL LOG HOST> section.
# Oracle VM Manager: Log locally & forward a copy to syslog1-syd.internal for further analysis
# - Oracle VM Manager to local2, not logging to file here as the input is a file
local2.* @@<CENTRAL LOG HOST>:601
Configuring the Central Log Host

On the central log host add the following line to /etc/rsyslog.conf to receive local2 events.
# Oracle VM: Log locally

# - Oracle VM Manager on local2
# - Oracle VM agent on local3
local2.* -/var/log/ovm/ovm-managers.log;RFC5424FMT
local3.* -/var/log/ovm/ovs-agents.log;RFC5424FMT
Next, restart the rsyslog service on the central log host and on Oracle VM Manager. After the rsyslog service is restarted, log host
should see inbound messages:
<151>1 2011-01-25T13:12:01+00:00 vmmanager1-syd OVM - - - INFO: ServerPoolSync.run: sync virtual

machines status succeeded.
<151>1 2011-01-25T13:12:01+00:00 vmmanager1-syd OVM - - - INFO: ServerPoolSync.run: sync cluster
master status succeeded.
<151>1 2011-01-25T13:12:01+00:00 vmmanager1-syd OVM - - - INFO: ServerPoolSync.run: sync agent
version succeeded.
<151>1 2011-01-25T13:12:01+00:00 vmmanager1-syd OVM - - - INFO: master server is:10.200.28.202
Example central log host /etc/rsyslog.conf file

The next example shows the complete central log host rsyslog.conf file.
# ----------------------------- Queues (required for forwarding) -----------------------------
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
# ----------------------------- Modules & Functions -----------------------------
# module: kernel logs, klogd replacement
$ModLoad imklog
# module: MARK messages (usually disable)

$ModLoad immark
235 of 243 18.04.2011 16:35

# module: input udp

# NOTE: documentation says InputUDPServerRun, that's wrong
$ModLoad imudp
$UDPServerAddress *
$UDPServerRun 514
# module: input tcp (works fine)

$ModLoad imtcp
# ----------------------------- Templates -----------------------------

# Use traditional timestamp format
# eg: "Jan 1 15:00:01 "
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# RFC5424 format
# also known as RSYSLOG_SyslogProtocol23Format, and draft-internet-ietf-syslog-protocol-23 (now
RFC5424)
# the "1" in "<%PRI%>1 " denotes syslog protocol version 1, as per the RFC
# eg: <21>1 2011-01-01T16:09:05+00:00 <MAIL PROXY HOST NAME> perdition 14185 - -
$template RFC5424FMT,"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID%
%STRUCTURED-DATA% %msg%\n"
# ----------------------------- Destinations -----------------------------

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
kern.* /var/log/kernel
# Log anything (except mail) of level info or higher.

# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local3.none /var/log/messages
# The authpriv file has restricted access.

authpriv.* /var/log/secure
# MAIL: Log locally & forward a copy to sysdev1-syd3 for further analysis
# - RFC5424 parseable output
mail.* -/var/log/mail/mail.log;RFC5424FMT
mail.* @@<SMTP HOST>:601
# Log cron stuff

cron.* /var/log/cron
# Everybody gets emergency messages

#*.emerg *
# Save news errors of level crit and higher in a special file.

#uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log

local7.* /var/log/boot.log
# All, save mail

*.debug;mail.none;authpriv.none;cron.none -/var/log/debug
# Oracle VM: Log locally

# - OVM manager on local2
# - OVS agent on local3
local2.* -/var/log/ovm/ovm-managers.log;RFC5424FMT
local3.* -/var/log/ovm/ovs-agents.log;RFC5424FMT
How to Configure the Oracle VM Server's Logrotation

The next example show the recommended Oracle VM server and the central log host configurations by creating an executable file
named ovs-agent in the /etc/logrotate.d/ directory.
# cat /etc/logrotate.d/ovs-agent
/var/log/ovs-agent/all.log {
create 0644 root root
236 of 243 18.04.2011 16:35

missing ok
# keep for 14 days

rotate 14
# dateext tags with date, max 1 rotation/day

dateext
# compression parameters
compress
compresscmd /usr/bin/bzip2
compressext .bz2
compressoptions -9
# notify syslog or rsyslog

postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
/bin/kill -HUP `cat /var/run/rsyslogd.pid 2> /dev/null` 2> /dev/null || true
endscript
}
The above configuration can be tested verbosely by typing “logrotate -v -f /etc/logrotate.d/ovs-agent”.

Watch this space
Virtualization Policy Project Introduction

The Virtualization Security Policy Project
Second Edition
Author: Roddy Rodstein, CISSP

Limits of Liability and Disclaimer of Warranty

This publication contains information protected by copyright. This publication may not be duplicated in any way without the express
written consent of the publisher, except in the form of brief excerpts or quotations for the purpose of review. The information
contained herein is for the personal use of the reader and may not be incorporated in any commercial programs, other books,
databases, or any kind of software without the written consent of the publisher. Making copies of this book or any portion, for any
purpose other than your own, is a violation of United States copyright laws.
Warning and Disclaimer

Every effort has been made to make this publication as complete and as accurate as possible, but no warranty or fitness is implied.
The information provided is on an "as is" basis. The authors and the publisher shall have neither liability nor responsibility to any
person or entity with respect to any loss or damages arising from the information contained in this publication.
The information found in this publication was gathered from many different sources in the computing world. It is provided for
informational purposes only. Use common sense in applying these concepts and tips. Screen shots may vary from environment to
environment. Please verify correctness and applicability in a test environment first and then deploy to your production
environment(s).
Introduction
I would like to introduce the Virtualization Security Policy Project Second Edition. The Virtualization Security Policy Project Second
Edition consists of an Enterprise Architecture Introduction, a Hardware and Software Sunset Policy, a Server Virtualization Policy, and
a Server Virtualization Standards.
The Enterprise Architecture Introduction provides the framework for the Virtualization Security Policy Project and runs through a
brief introduction of Enterprise Architecture to
illustrate how your virtualization technologies fit within your Enterprise Architecture.
The Hardware and Software Sunset Policy define an organization’s hardware and software sunset policy.
The Server Virtualization Policy and Server Virtualization Standards define an organization’s server virtualization requirements and
237 of 243 18.04.2011 16:35

minimum standards.
You can use the policies and standards as is or modify them to meet your specific business requirements.
I welcome your comments about the Virtualization Security Policy Project. Please let me know if you would like to see additional
policies added to the Virtualization Security Policy Project to help support your virtual infrastructure.
Enterprise Architecture Introduction

The Virtualization Policy Project begins with a high level overview of Enterprise Architecture (EA) and concludes with example IT
policies. Because Enterprise Architecture is a field unto itself, a detailed review of its principles, processes, and approach is beyond
the scope of this publication. The goal of this publication is to explain how virtualization technologies fit within an Enterprise
Architecture.
The purpose of Enterprise Architecture is to establish an Enterprise wide blueprint used to achieve business objectives while
maximizing the business value of information technology. An Enterprise Architecture is a “blueprint” that describes an organization’s
strategic direction, security and regulatory requirements, information technology portfolio and their inter-dependencies, baseline and
target architectures, and the processes to implement technologies. In business terms, Enterprise Architecture is accomplished by
efficiently achieving an organization’s mission with minimal investment in information technology; and in technical terms, by
optimizing business operations, effective information technology planning, information technology budgeting, information technology
acquisition, human resource utilization, and the implementation of security controls.
After the goals and stakeholders of an Enterprise Architecture project have been identified, a framework is selected to help
implement and support the Enterprise Architecture through its entire life cycle. There are a number of frameworks, such as Cobit,
ISO/IEC 17799, ITIL, and many others that represents a variety of methodologies and toolsets to fulfill the requirements of any size or
type of organization. Frameworks provide methodologies, standards, guidelines, and formats that can be used as is or adapted to
meet specific requirements. Because organizations have different missions and business objectives, no single framework will be right
for each situation. Organizations typically select a framework or a mixture of frameworks to meet their requirements.
Enterprise Architecture has well defined principles and processes, along withan approach that generates a comprehensive layered
policy infrastructure used to communicate management’s goals, principles, instructions, procedures, and response to laws and
regulatory mandates. A policy infrastructure consists of tier 1, tier 2 and tier 3 policies that encompass people, systems, data, and
information. A policy infrastructure consists of policies, standards, procedures, baselines, and guidelines.
Tier 1 policies are at the top layer of the policy infrastructure and address broad organizational issues, vision and direction. Most
organizations will develop and support up to a dozen tier 1 policies. An example tier 1 policy is an Employee Practices Policy or a
Conflict of Interest Policy. Global in scope, Tier 1 policies are high level documents that define vision and direction.
Tier 2 policies are topic specific, and tier 3 policies are application or system specific. Standards are tier 2 policies that describe
system design concepts, implementation steps, and detailed configurations. Procedures are tier 2 & 3 policies that provide step by
step compulsory measures, communicating best practices in using information systems and organizational data/information. Baselines
are tier 3 policies that are application or system specific and describe step by step instructions to implement technologies.
Guidelines are tier 3 documents, offering application, system, or procedural specific best practices. Guidelines are
recommendations unlike policies, standards, procedures, and baselines, which are compulsory.
Figure 1 shows the organization of Enterprise Architecture’s layered policy infrastructure.
Figure 1 represents Enterprise Architecture’s layered policy infrastructure, starting with tier 1 policies which address broad
organizational issues, vision, and direction. The next layer, General Organizational Policy, consists of tier 1 policies in which
management makes security statements, explains roles and responsibilities, and defines which assets are considered valuable. The
following layer, Practical Implementation Policies, contains tier 2 and 3 policies which are topic, application, and system specific and
are used to enforce upper layer policies. The lower layer consists of tier 2 and 3 policies which are topic and technology specific and
are used to enforce higher layer policies.
Figure 2 shows the work flow of a policy infrastructure.
238 of 243 18.04.2011 16:35

A policy infrastructure contains confidential information relating to running a business and the publication, distribution and storage of
that information should be strictly monitored. Many organizations leverage the human resource department and secured
intranet solutions to distribute and control access to policies.
An Enterprise Architecture groups together infrastructure components within topic specific domains. An example of Enterprise
Architecture domains are infrastructure, applications, network, and security. After an organization has defined its Enterprise
Architecture domains, all infrastructure components are grouped within their corresponding domain and reviewed individually and as
a single cohesive unit. Layered policies are developed for each domain and each individual technology within a domain.
Table 1 shows the Enterprise Architecture domain structure that will be used throughout this publication. The example encompasses
five domains split between two high level groups; infrastructure and applications. The five domains are platform, network, software,
data / information, and security.
Enterprise Architecture Scope
Infrastructure Applications
Platform Software
Network Data/Information
Security
An organization’s mission and business objectives drive its Enterprise Architecture domain structure. As we have all learned, there is
no ‘one size fits all’ with information technology, and Enterprise Architecture is no exception. Enterprise Architecture is flexible and
can be molded to suit any organization’s mission and business objectives.
Table 2 shows a variation of the above Enterprise Architecture domain structure.

Enterprise Architecture Scope
Platform
Network
Software
Data/Information
Security
Access Domain
Integration Domain
Privacy Domain
Project Management Domain
Systems Management Domain

Each of the domains within an Enterprise Architecture will have its corresponding layered policy infrastructure, with tier 1 & 2
policies, tier 2 & 3 standards, procedures, baselines, and guidelines.
Watch this space

239 of 243 18.04.2011 16:35


Hardware and Software Sunset Policy

The following example Hardware and Software Sunset Policy defines an organization’s hardware and software sunset policy. This
policy is intended for informational purposes only.

Purpose
The purpose of this policy is to establish hardware and software sunset requirements. In an ongoing effort to meet business
requirements, reduce IT costs and provide reliable, high-quality IT services, <Company Name> periodically sunsets (retires), old
hardware and software. Once sunsetted, active support and all business services for the product are discontinued. Sunsetting older
versions of hardware and software allows <Company Name> to focus resources on enhancing IT services, and providing support for
more current, secure and stable products. In most cases, replacement costs for products identified for sunset are generally less than
the expenses of continued support and maintenance. The Sunset policy will result in better customer service and reduced costs. This
policy provides controls to ensure that Enterprise issues are considered along with business objectives when sunsetting hardware and
software.
Scope
The scope of this policy encompasses server, desktop and network hardware platforms, operating systems and application software.
Policy
Products that have reached the end of their life cycle and are no longer supported by a vendor will be given a sunset date. The sunset
date is when the product is scheduled to be removed from production. The sunset date will be set far enough in advance to give
<Company Name> at least a budget cycle to fund and plan for the replacement. When a particular version of hardware or software is
scheduled to be sunsetted, <Company Name> will provide the affected users with advance notice via email.
A Sunset list will be used to track all hardware and software sunset dates. In order to keep the sunset list up to date, <Company
Name> will update the sunset list quarterly with hardware and software for review. Department managers with staff that use
products on the sunset list will be notified quarterly via email regarding the sunset review process and sunset dates.
If you are currently using application software that has been designated sunset and would like to extent support, you will need to
acquire a version that meets the current minimum standards as defined in <Company Name> Software Standards. If you are
currently using hardware that has been designated sunset, any technical issues with the unit will trigger a replacement process with
a unit that meets the current minimum standards as defined in <Company Name> Hardware Standards.
List 1 shows the sunset categories:
Hardware four years or older.

Operating systems that have reached their sunset date or are no longer supported by the vendor.
Proprietary application software that is no longer supported by the vendor.
Open Source application software that is no longer supported by the community.
Application software that does not support <Company Name> centralized authentication and authorization system.
Review Cycle
This policy is subject to annual review.
Compliance
All information technology investments shall conform to existing policies in order to ensure the integrity and interoperability of
computing platforms. Any employee found to have violated this policy may be subject to disciplinary action, up to and including
termination of employment.
Related Policies
Platform Infrastructure Standard
Xen.org @ OSCON
Xen.org GSoC 2011 Update
KVM Forum 2011 scheduled for August 15-16
Libvirt 0.9.0 Released
Virt-Manager 0.8.7 Released: New UI Features
Proxmox VE 1.8 Released: Supports qemu-kvm 0.14
Xen 4.1 releases
Xen hack-a-tron day 1
Xen.org accepted for GSoC 2011
Xen.org spring clean
Server Virtualization Policy

The following example Server Virtualization Policy defines an organization’s server virtualization requirements and minimum
standards. This policy is intended for informational purposes only.
240 of 243 18.04.2011 16:35

Purpose
The purpose of this policy is to establish server virtualization requirements that define the acquisition, use, and management of server
virtualization technologies. This policy provides controls that ensure that Enterprise issues are considered along with business
objectives when making server virtualization related decisions.
Platform Architecture policies, standards and guidelines will be used to acquire, design, implement and manage all server
virtualization technologies.
Scope
The scope of this policy encompasses all new and existing workloads.
Responsibilities
The CEO and CIO ensure that policies are followed in order to establish contracts, review procurement requests and to develop and
manage services.
Policy
<Company Name>’ legacy IT practice was to dedicate one physical server to a single workload. The result of <Company Name>’
legacy IT practice was excessive server underutilization, an ever-expanding data center footprint and excessive data center power
consumption.
Server virtualization software allows the consolidation of new and existing workloads onto high capacity x86 servers. Consolidating
workloads onto high capacity x86 servers allows <Company Name> to reduce the x86 server inventory, which in turn decreases the
data center footprint and data center power consumption.
<Company Name> will migrate all new and existing workloads from physical servers to virtual machines. All workloads that cannot
be migrated to a virtual machine will be subject to <Company Name>’ Hardware and Software Sunset Policy.
Server Virtualization Software Requirements:
Support industry-wide open-standards

Embedded security
Single centralized management console
Support industry standard management tools
Support industry standard backup and recovery tools
Interoperate with other platform technologies
Support industry standard x86 hardware
Support industry standard storage
Support unmodified guest operating systems
Migrate running guests without interruption
Add disks to a running guest
Snapshot running guests
Revert to a previous snapshots on a running guest
Automatically detect a hardware failure and restart guests on another physical server
Functionality to configure role based access for the administrative console
Support LDAP for authentication and authorization for administrative console
Encrypt all intra host and administrative console traffic
Integrated graphical CPU, memory, disk and network performance monitoring, alerting, and historical reporting for hosts and
guests.
Retain performance data for up to one (1) year
Functionality to manage host CPU, memory, storage and network resource allocation
Functionality to manage guest CPU, memory, disk and network resource allocation
Functionality to create, stop, start, pause, migrate, clone and provision guests
Functionality to automatically load balance guests across multiple hosts
Consolidated logging for hosts and guests that log date and time of all administrative user actions
Functionality to convert x86 physical servers to virtual machines
Encrypted remote administrative console access
Review Cycle
Compliance
computing platforms.
Any employee found to have violated this policy may be subject to disciplinary action, up to and including termination of employment.
Related Policies
Server Virtualization Standard
Server Virtualization Guidelines
Xen.org @ OSCON
241 of 243 18.04.2011 16:35


Xen 4.1 releases
Server Virtualization Standards

Standards are used to provide the uniform use of technologies to drive consistency and reproducibility, lower operational costs, and
enable faster deployments of technologies and functions. The following example Server Virtualization Standards policy shows how an
organization defines their Server Virtualization Standards.
Server Virtualization Standards

Purpose
The purpose of these standards is to define Enterprise wide server virtualization requirements in order to provide opportunities to
meet strategic and tactical Information Technology objectives. These standards define a template and a set of requirements used to
implement and support server virtualization software.
Platform Architecture policies, standards and guidelines will be used to acquire, design, implement and manage all server
virtualization software.
Scope
The scope of this policy encompasses all server virtualization software.
Responsibilities
The CEO and CIO ensure that policies are followed in order to establish contracts, review procurement requests and to develop and
manage services.
Standards:
Table 1 shows the Technology Classifications used to define <Company Name>’ technology life cycle.
Current Technologies that are approved and recommended

for use.
Contain Technologies that are being phased out over the
next 3 to 5 years.
Retire Technologies that will be phased out and replaced.

A discontinuance date has been set.
Research Technologies that may become current. Research
technologies should be used only for test
environments for evaluation purposes.
Current
Table 2 lists the current server virtualization technologies.
Products Platforms Category Technology

Classification
Oracle VM 2.2 Hypervisor Production, Test, Current
whithout a host and Development
operating
system. Servers
will be x86-64
platform.
VMware ESXi Hypervisor Production, Test, Current
3.5 whithout a host and Development
operating
system. Servers
will be x86-64
platform.
VirtualBox 3.0. Workstations Test and Current
running Linux Development
or Windows. only
Contain
Table 3 lists the contain technologies that are being phased out over the next 3 to 5 years.

Classification
242 of 243 18.04.2011 16:35

VMware ESX Hypervisor All categories Contain

Server 3.x whithout a host
operating
system. Servers
will be x86-64
platform.
VMware Server Servers and All categories Contain
1.x Workstations
running Linux
or Windows.
VMware Workstations All categories Contain
Workstation 5.x running Linux
or Windows.
Retire
Table 4 lists the retire technologies which are being phased out. Referance the Hardware and Software Sunset Policy for the software
retirment plan.

Classification
VMware ESX Hypervisor All categories Retire on 12/31/08
Server versions whithout a host
prior to v. 2.5 operating
system. Servers
will be x86-64
platform.
VMware GSX Servers running All categories Retire on 12/31/08
Server versions Linux or
prior to v. 3 Windows.
VMware Server Servers running All categories Retire on 12/31/08
versions prior to Linux or
v. 2 Windows.
VMware Workstations All categories Retire on 12/31/08
Workstation running Linux
versions prior to or Windows.
v.5
Reaearch
Table 5 lists the research technologies. Research technologies could become current standards. The use of research technologies is
restricted to test environments.

Classification
KVM Linux only. Test only Research
Microsoft Does not Test only Research
Hyper-V need a host
OS.
Review Cycle
Compliance
computing platforms.
Any employee found to have violated this policy may be subject to disciplinary action, up to and including termination of employment.
Related Policies
Server Virtualization Guidelines
Xen.org @ OSCON
Xen 4.1 releases
243 of 243 18.04.2011 16:35

Oracle VM

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Oracle VM

Caricato da

Copyright:

Formati disponibili

The Underground Oracle VM Manual http://itnewscast.

The Underground Oracle VM Manual

Revision Change Description Updated By Date

1.0 First Release of the Roddy 05/2009

2.0 Second Release of the Roddy 09/2009

3.0 Third Release of the Roddy 08/2010

4.0 Forth Release of the Roddy 12/2010

1 Complete rewrite of Roddy 03/08/11

1 Oracle VM Centralized Roddy 03/29/11

1 Complete rewrite of Roddy 04/03/11

Limits of Liability and Disclaimer of Warranty

Warning and Disclaimer

About the Author

1 of 243 18.04.2011 16:35

Watch this space

Chapter 1: Oracle VM Introduction

Revision Change Description Updated By Date

1 Complete rewrite of Roddy 03/08/11

The Evolution of Oracle VM for x86

2 of 243 18.04.2011 16:35

Oracle VM for x86 Introduction

Figure 1 shows the Oracle VM Manager administrative portal.

3 of 243 18.04.2011 16:35

Oracle VM for x86 Virtualization Modes

Guest Hardware Hardware

4 of 243 18.04.2011 16:35

Oracle VM for x86 Packaging and Pricing

Accelerating Application Deployments with Oracle VM for x86

5 of 243 18.04.2011 16:35

1. Virtual Machine Cloning

Virtual Machine Cloning

List 2: The steps to download and deploy an Oracle VM template.

6 of 243 18.04.2011 16:35

List 3: Oracle’s Oracle VM templates collection as of this writing.

Virtualization and Management

Oracle VM Manager 2.2 and 2.1.5

E-Business Suite 12.1.3

Oracle WebLogic Server on JRockit Virtual Edition 11g R1 (10.3.2)

Database & Real Application Clusters (RAC)

Oracle Real Application Clusters (RAC) 11g Release 2

Oracle Linux 4 and 5

Zeus Traﬃc Manager

Oracle VM Templates Packaging and Pricing

Oracle VM Template Builder

7 of 243 18.04.2011 16:35

Oracle VM Builder Packaging and Pricing

Oracle Virtual Assembly Builder

Note: Introspection is Oracle's terminology for application proﬁling.

Oracle Virtual Assembly Builder Packaging and Pricing

8 of 243 18.04.2011 16:35

Oracle JRockit Virtual Edition

Numbers Linux JRockit Virtual

Command Parameters +-50000 100

Disk Image Size +-10 GB 1 GB

Oracle JRockit Virtual Edition Packaging and Pricing

The Oracle VM Management Pack Plug-in

9 of 243 18.04.2011 16:35

Figure 3 shows the Oracle Enterprise Manager administrative console.

Capability Oracle VM OEM /

Oracle VM server, Oracle VM *

*Oracle Linux, Red Hat *

Database and application *

Oracle Linux, Red Hat