0002 V7000 VMware Storage Practices

Deploying IBM Storwize V7000 in VMware Environments
Best practices
Systems and Technology Group ISV Enablement December 2010
Copyright IBM Corporation, 2010.
Table of contents
Abstract..................................................................................................................................... 1 Executive summary.................................................................................................................. 1
Purpose of the document ................................................................................................................... 1 Assumptions ...................................................................................................................................... 1
Product overview...................................................................................................................... 2
VMware ............................................................................................................................................. 2 IBM Storwize V7000........................................................................................................................... 2
Strategic Configuration Considerations ................................................................................. 5

Managing the Information Infrastructure ............................................................................................. 5 RAID Options for Data Protection ....................................................................................................... 5 Tiers for efficiency and performance................................................................................................... 6 Virtualized Volumes ..........................................................................................................................11
Attaching VMware ESXi to IBM Storwize V7000 ................................................................... 13

SAN Fabric Zoning for VMware ESXi Hosts ......................................................................................13 Setting up VMware ESXi Boot from SAN ...........................................................................................14
Summary................................................................................................................................. 26 Appendix A: Materials list...................................................................................................... 27 Appendix B: IBM Storwize V7000 best practices a quick reference ................................ 28
SAN zoning.......................................................................................................................................28 Storage Pools ...................................................................................................................................28
Appendix C: Resources ......................................................................................................... 30 Trademarks and special notices ........................................................................................... 31
Deploying IBM Storwize V7000 in VMware environments Copyright IBM Corporation, 2010.
Abstract
This paper provides insight into the value proposition of the IBM Storwize V7000 for a VMware environment to increase the storage utilization, availability and overall reduction of costs. It outlines a plan for the configuration, setup, and management of the environment.
Executive summary
IBM System Storage virtualization technologies enable clients to virtualize more systems, faster and with less storage than traditional legacy storage architectures. VMware on IBM Storwize V7000 solutions focus in the areas of increasing data protection, reducing infrastructure costs through storage efficiencies. This paper reviews the configuration and installation practices for leveraging VMware vSphere together with IBM Storwize V7000 systems. IBM and VMware have continually provided world class virtualization solutions that can reduce clients total cost, and provide advanced storage features reducing overall downtime. In this paper these techniques have been documented and are referred to as best practices. After reviewing this paper the reader will be able to easily install and configure IBM Storwize V7000 systems along with VMware vSphere to build a completely virtual infrastructure for improved utilization, flexibility and storage efficiency.
Purpose of the document

The intent of this paper is to provide architectural, deployment, and management guidelines for customers who are planning or have already decided to implement VMware on the IBM Storwize V7000 systems. It provides a brief overview of the VMware technology concepts, key solution architecture considerations layout considerations; and solution, deployment, and management guidelines for implementing VMware on IBM Storwize V7000 systems. The purpose of this reference configuration is to show the following how IBM Storwize V7000 systems can improve efficiency through ease of use and increased storage utilization. This paper does not provide detailed performance numbers or advanced high availability disaster recovery techniques, these topics will be covered in future papers for IBM Storwize V7000 systems in VMware environments. This paper is also not intended to be any kind of formal certification. For detailed information regarding hardware compatibility please see the VMware Hardware Compatibility List (HCL) and IBM System Storage Interoperation Center (SSIC) websites. VMware HCL: see http://www.vmware.com/resources/compatibility/search.php IBM SSIC: see http://www03.ibm.com/systems/support/storage/config/ssic/displayesssearchwithoutjs.wss?start_over=yes
Assumptions
While every attempt has been made to clearly describe the various installation processes required for this solution, this paper assumes persons with essential knowledge in the following areas: VMware vCenter Server ESXi Installation Virtual Machine Filesystem (VMFS) and Raw Device Mapping (RDM)
Deploying IBM Storwize V7000 in VMware environments Copyright IBM Corporation, 2010 1
vMotion, High Availability (HA) and Distributed Resource Scheduler (DRS) Guest OS installation on a ESXi host Storage subsystems and terminology
This paper will demonstrate how to enable VMware Boot from storage area network (SAN) on the IBM Storwize V7000 systems.
Product overview
VMware
VMware products provide enterprise-class virtual machines (VMs) that increase server and other resource utilization, improve performance, increase security, and minimize system downtime thereby reducing the cost and complexity of delivering enterprise services. By leveraging existing technology, VMware enables the roll-out of new applications with less risk and lower platform costs.
VMware vSphere Hypervisor

VMware vSphere is a feature-rich suite that delivers the production-proven efficiency, availability, and dynamic management needed to create a responsive data center. The suite includes: VMware ESXi Server: Platform for virtualizing servers, storage, and networking. VMware VMFS: High-performance cluster file system for storage virtualization. VMware Virtual SMP: Multi-processor support for VMs. VMware vCenter Server: Centralized management, automation, and optimization for IT infrastructures. VMware High Availability (HA) Cost-effective high availability for VMs. VMware DRS: Dynamic balancing and allocation of resources for VMs. VMware vMotion: Live migration of VMs without service interruption.
IBM Storwize V7000

The IBM Storwize V7000 solution provides a modular storage system that includes the capability to virtualize external SAN-attached storage as well as its own internal capacity. The IBM Storwize V7000 solution is built upon the IBM SAN Volume Controller technology base and utilizes technology from the IBM System Storage DS8000 family. IBM Storwize V7000 systems provide a number of preset configuration options that are aimed at simplifying the implementation process. It also provides automated wizards, called Directed Maintenance Procedures (DMP), to assist in resolving any events that may occur. IBM Storwize V7000 provides an active-active solution. This is illustrated in Figure 1.
VMware vSphere
ESXi Servers access to storage is independent of the physical storage layer
IBM Storwize V7000 virtualizes the external disk systems as well as its own internal storage
Figure 1 IBM Storwize V7000 virtualizing external and internal storage
The IBM Storwize V7000 systems can provide up to 240 TB of physical internal capacity, with support for Redundant Arrays of Independent Disks (RAID) 0, 1, 5, 6 and 10. In addition Storwize V7000 has the capability to virtualize external storage controllers with the ability to expand and support virtualization of up to 32 PB allowing for the reuse of existing legacy storage controllers.
IBM Storwize V7000 Concepts

The Storwize V7000 contains at least one controller enclosure. The controller enclosure consists of two node canisters; the two node canisters are part of a cluster and act as a single processing unit and form an I/O group. The I/O group is the control point for volume access within the Storwize V7000 cluster and each node canister is restricted to a single I/O group as shown in Figure 2.
VMware vSphere
Figure 2 Storwize V7000 virtualization components
In addition to the controller enclosure the Storwize V7000 systems may have between one to nine expansion enclosures connected through a redundant serial-attached SCSI (SAS) connections. Each controller enclosure and each expansion enclosure can support a combination of solid-state disk (SSD), SAS 15 K, 10K and nearline SAS drives. Internal capacity as well as any capacity shared from external storage controllers are grouped into storage pools. A storage pool is a collection of managed disks with similar characteristics; it acts as a container that can be classified as a tier of storage which will contain the virtualized volumes presented out to individual host systems. The managed disks within a storage pool are broken down into extents, you can think of an extent as a slice of a particular managed disk. The extents are what are used to create the virtualized volumes within the storage pool. Volumes within a storage pool exist in any one of the following three forms; Striped. A striped volume is the allocation of extents, one extent at a time across each managed disk in a storage pool until the space required for the volume has been satisfied.
Sequential. In a sequential volume extents are allocated from a single managed disk and then progresses to the next managed disk until the space required for the volume has been satisfied. Image. An image mode volume is a special volume that has a direct relationship with a single managed disk in a storage pool. Image mode volumes are typically used for the purpose of data migration to IBM Storwize V7000 systems.
For more information about IBM Storwize V7000, see http://www.ibm.com/storage/storwizev7000
Strategic Configuration Considerations

Before you can effectively design a storage virtualization strategy, you must consider and determine the primary goals; performance, reliability, growth, manageability, cost and function. After you have determined what goals are best for your environment, consider these guidelines while implementing those goals. This section is intended to provide an overview of IBM Storwize V7000 concepts and characteristics to help with the planning for your virtualization strategy. This information is provided as background with best practices being described later in this document.
Managing the Information Infrastructure

When considering storage tiers consider the virtual machines proposed application, different applications have differing I/O profiles. Will the virtual machine be used in test, development or production, is the application structured transactional data or unstructured? When considering the placement of a virtual machines storage capacity, consideration should be taken as to what services the virtual machine will provide. Isolation of not only application workload but also the purpose should be considered. This also helps when building your disaster recovery objectives by helping to isolate production from development and test. I/O Characteristics o Random compared to sequential o Read compared to write Protection Requirements o Production o Development o Test Structured or unstructured
By defining storage tiers you can provide the flexibility to effectively manage allocation as well as host and application recovery requirements.
RAID Options for Data Protection

RAID, is a technology that provides increased storage reliability through redundancy by combining multiple disk drives into a logical unit where all drives are interdependent. The disk drives are placed into a RAID array with characteristics of the particular level of RAID assigned. Table 1 lists the RAID level presets that are supported for internal SAS drives in Storwize V7000, SSD drives have their own presets that is discussed later in this paper.
NOTE: External storage devices presented to IBM Storwize V7000 systems are not eligible for configuration into internal RAID arrays. For external storage devices the RAID configuration is maintained and controlled within the external storage controller.
Level
Description Block-level striping without parity or mirroring Mirroring without parity or striping Block-level striping with distributed parity Block-level striping with double distributed parity Mirrored sets in a striped set
Width Goal 8
Drive Count 1-8
Space Efficiency 1
Fault Tolerance 0 (none)
1/n
n-1 disks
3 -16
1-1/n
1 disk
12
5 - 16
1-2/n
2 disks
2 16 (even only)
1/n
1-disk/2
Table 1: IBM Storwize V7000 RAID Levels
Tiers for efficiency and performance

When managing internal and external capacity under Storwize V7000 this capacity is assigned to storage pools. These storage pools are containers for managed resources to be virtualized that have the same characteristics for performance, protection and reliability. By effectively managing your storage pools, you are more capable to offload over-provisioned resources and allow the strategic placement of individual application workloads and leverage the internal and backend characteristics of storage systems to optimally configure virtualization strategy to best service your needs.
Internal Capacity
Internal capacity in Storwize V7000 is configured into internal RAID arrays during the setup and configuration using predefined settings based on the drive characteristics and preset best practices. From an internal RAID array a single managed disk encompassing the full capacity of the array is created and available to be placed into a storage pool. For internal storage pools, Storwize V7000 does not allow the intermixing of different technologies or RAID characteristics in a single storage pool. There is one exception to this rule which creates a hybrid pool that will be described later in this paper. If a suitable storage pool with matching characteristics is not available, then a storage pool will be created to contain the new managed disk object. This is done as a best practice as differing technologies have the potential to work against each other for performance and reliability. Allowing the intermixing of different technologies has the potential to reduce the overall storage pool efficiency to the lowest performing technology. Consider SAS over nearline SAS where capacity and performance can vary significantly. Adding a nearline SAS backed managed disk to a storage pool
with SAS backed managed disk results in the overall pool performance characteristics of the nearline SAS eliminating any benefit of the higher performing SAS backed managed disk. This is done to help optimally configure the available internal capacity based on the characteristics and placement within the expansion containers. For a complete list of internal disk and RAID presets, refer to the URL: http://www.ibm.com/storage/storwizev7000
External Capacity
The Storwize V7000 system is built using the same virtualization technology as the IBM System Storage SAN Volume Controller (SVC), with the same capabilities to virtualize heterogeneous storage controllers. When virtualizing external storage controllers it is best practice to assign the storage objects, managed disks, to storage pools with the same characteristics. When dealing with external managed disks, Storwize V7000 will not prevent you from adding managed disks with different characteristics into the same storage pool, however always remember that reliability and performance for the storage pool will be at the lowest element in the storage pool. Consider that you have an external storage array that you would like to bring under the control of the Storwize V7000 for virtualization. Within the storage array you have a combination of FC and SATA disks with differing RAID characteristics. You have RAID arrays configured for RAID 5 using SCSI FC 15K RPM disks and RAID 6 using SATA disks for higher capacity as nearline storage. Intermixing the various technologies and RAID levels within a storage pool will result in the potential for the entire storage pool to have the performance characteristics as the RAID 6 array using the SATA disks in this case. In addition this would also have the potential of reducing the overall availability of the storage pool. In this case to the lowest availability element, RAID 5 where a single disk failure in the RAID array places volumes that are striped within the storage pool across the RAID 5 array in jeopardy for performance and additional drive failure. When working with external storage devices consider the stripe width of the underlying RAID technology for the proposed managed disk. Standardize on array width and logical unit number (LUN) size to be created out of the array. The Storwize V7000 system has the ability to support managed disk capacities up to 1 PB. As a best practice create a single LUN from each array to be presented to the Storwize V7000 system.
External storage fabric zoning

Storwize V7000 system supports attachment to external storage through SAN fabric only. When attaching external storage each node in Storwize V7000 must see the same external storage controller ports. As a best practice two redundant fabrics should be used for redundancy in the event one of the fabrics should go offline for either a planned or unplanned outage. In each fabric create a zone with just the IBM Storwize V7000 ports. Create additional zones for the external storage systems to be virtualized. In each fabric create a zone with the IBM Storwize V7000 ports from each node canister, along with up to a maximum of eight (8) FC ports from the external storage system. The Storwize V7000 supports up to maximum of sixteen (16) ports from a given external storage system to be virtualized, eight (8) in each fabric.
Zone Storwize V7000 Zone
Fabric A node1-port1 / node1-port3 /node2port1 / node2-port3 node1-port1 / node1-port3 / node2port1 / node2-port3 ExtCntrl-A-port1 /ExtCntrl-B-port1
Fabric B node1-port2 / node1-port4 / node2port2 / node2-port4 node1-port2 / node1-port4 / node2port2 / node2-port4 ExtCntrl-A-port2 / ExtCntrl-B-port2
Storwize V7000 ExtCntrl Zone
Table 2: Zoning for external devices
Figure 3 shows the logical view of the externally attached disk.
Storwize V7000 External Storage Array
Figure 3 Logical view for externally attached disk
Storage pools
Storage pools as mentioned previously are a collection of managed disks (MDisks), that are grouped together in order to provide capacity for virtualized volumes. All MDisks in the pool are split into
extents of the same size. Volumes are then allocated out of the storage pool and are mapped to a host system. MDisks can be added to a storage pool at any time to increase the capacity of the storage pool. When adding MDisks to a storage pool ensure that each MDisk matches the characteristics desired of the storage pool. Do not intermix MDisks with differing RAID characteristics or from different controllers. Care should be taken to ensure that each storage pool contains MDisks from a single controller when working with external attached MDisks. There are use cases where it maybe desirable to have MDisks from more than one controller of the same make and model. However as a general rule and best practice this should be avoided as the virtualized volumes within a storage pool can become inaccessible if one of the backend controllers becomes unavailable in a storage pool with multiple external storage controllers for any reason. Figure 4 shows the creation of a storage pool using the Storwize V7000 management GUI.
Figure 4 Storage Pool Creation
Hybrid storage pool IBM Easy Tier

Storwize V7000 system has a classification of storage pool called a hybrid pool. A hybrid storage pool takes advantage of the intermixing of SSD managed disks and standard managed disks through the use of IBM Easy Tier.
SSD devices offer higher performance and lower latency over traditional spinning disk drives at typically higher cost per GB of capacity. Although an application or virtualized host may require the higher performance that SSD devices provide, in general only a portion of a volume or volumes really require the added benefits that an SSD device provides. The Easy Tier function allows for the effective use of SSD devices within Storwize V7000 by monitoring the I/O characteristics of a virtualized volume or storage pool and migrating only the frequently accessed extents within a volume or storage pool to the higher performing SSD managed disks. Easy Tier protects your investment by reducing the need for host or application dedicated SSD virtualized volumes to meet performance requirements by dynamically migrating only the extents that will benefit from the SSD performance characteristics and removing the need to dedicate a entire storage pool or volume to a set of SSD managed disks. Storwize V7000 is able to take advantage of internal as well as external SSD managed disks. While internal SSD MDisks are known to the system, external managed disk characteristics are not known. To allow external SSD managed disks, Storwize V7000 allows the user to specify which external managed disks are SSD devices and which are non-SSD devices allowing the Easy Tier function to take advantage of any existing external SSD capacity in the environment. SSD devices are not suited for all application loads and have specific characteristics where they are best suited. Use of Easy Tier reduces overall cost and investment of SDD devices by effectively managing their usage for I/O characteristics that will actually benefit from the added performance. Figure 5 shows diagram of a hybrid pool using SSD managed disks.
Figure 5 Easy Tier
Virtualized Volumes
When creating virtualized volumes from storage pools on Storwize V7000 you are presented with the option to create one of four volume types; generic (thick), thin-provisioned, generic mirrored and thinprovisioned mirrored. The generic mirrored and thin-provisioned mirrored options allow for a volume to be mirrored between two different storage pools. This can be useful when migrating to a storage pool with a larger extent size, or when mirroring is preferred between two external storage subsystems. The ability to thin provision volumes to the VMware ESXi hosts provides some advantages over generic or thick provisioning, but also carries with it a risk of over provisioning and experiencing an out-of-space condition within a storage pool. Thin-provisioned volumes have a defined size viewed by the ESXi hosts as any other volume without the physical allocation of capacity until data is written from the ESXi hosts.
Thin provisioning
Some typical advantages of thin provisioning are: Reduce storage management efforts Exploit over provisioning in the file system and database without the underlying physical capacity Add real capacity transparently to the storage pool later as required.
Improved capacity utilization Physical capacity does not get allocated until actually used Unused capacity for a set of volumes resides in a shared pool of overall available capacity Contingency capacity can be shared between volumes where the average contingency capacity per volume can be less than fully committed capacity Higher utilization of existing physical storage capacity, reduces the need to acquire additional physical capacity or defer to a later point in time.
Thin provisioning storage allocation with VMware ESXi

Table 3 shows the various configuration steps and effects when using thin provisioning at the disk storage level with VMware ESXi.
Configuration step Map thin-provisioned volume to ESXi hosts Create VMFS datastore
Effect on storage pool capacity allocation No allocation of physical capacity for the volume in the storage pool Small allocation of physical capacity on the volume within the storage pool to write VMFS metadata No allocation of additional capacity on the volume within the
Create virtual machine on VMFS and
create .vmdk file
storage pool. The .vmdk file is created, however capacity is not allocated. Allocation of capacity on the volumes in the storage pool is dependent on the file system used in the virtual machine and amount of data written to the .vmdk file. Amount of data allocated on the volume in the storage pool is equal to the amount of data in the .vmdk. If the user chooses to make the disk compatible with Fault Tolerance, the disk type is eagerzeroedthick. This means all blocks are zeroed out and there will be no thin provisioning savings on the storage pool.
Install operating system in virtual machine
Deploy a virtual machine from a template
Delete data within the virtual machine
Capacity remains allocated on the volumes in the storage pool but volumes might be reused by the virtual machine. Capacity remains allocated but might be reused by the ESXi host.
Delete the virtual machine and delete the virtual machines .vmdk file
Table 3: Thin provisioning effects
Thin provisioning at the storage level and ESXi server level can be complimenting for space efficiency and storage allocation. For additional information on Thin Provisioning in a VMware environment, see http://www.vmware.com/files/pdf/VMware-DynamicStorageProv-WP-EN.pdf
Attaching VMware ESXi to IBM Storwize V7000

The following section describes the steps necessary to setup VMware ESXi servers to boot from SAN in a cluster configuration. The purpose of running this configuration is to show the ease of use when attaching VMware ESXi hosts in a cluster configuration on IBM Storwize V7000. Some of the advantages for booting from SAN include: Disaster recovery - Boot images stored on disk arrays can be easily replicated to remote sites where standby servers of the same HW type can boot quickly, minimizing the negative effect a disaster can have to the business. Clone - Boot images in clones can be quickly reverted back to a point-in-time, saving time and money in rebuilding a server from scratch. Quick deployment of servers - Master boot images stored on disk arrays can be easily cloned using clone capabilities providing rapid deployment of additional physical servers. Centralized management - Master image is located in the SAN. Upgrades and patches are managed centrally and are installed only on the master boot image which can be then cloned and mapped to the various servers. Greater storage consolidation - Because the boot image resides in the SAN, there is no need to purchase internal drives. Greater protection - Disk arrays provide greater data protection, availability and resiliency features than servers. For example, Storwize V7000 volume mirroring functionality provides additional protection in the event of a storage controller failure
SAN Fabric Zoning for VMware ESXi Hosts

Assuming each host has a Fibre Channel (FC) connection to two fabrics then in each fabric create a zone with a single host port initiator and one target port from each node canister in IBM Storewize V7000. When connecting from a VMware ESXi host, limit the number of paths from the host to no more than four. As a best practice create the host zones with a single initiator. Do not group multiple initiators from the same host or additional hosts into the same zone, use one host initiator port per zone. Zone
Storwize_V7000_ESXi01
Fabric A
node1-port1 / node2-port1 ESXi01-port1
Fabric B
node1-port2 / node2-port2 ESXi01-port2 node1-port4 / node2-port4 ESXi02-port2
Storwize_V7000_ESXi02
node1-port3 / node2-port3 ESXi02-port1
Table 4: ESXi Fabric Zones
If the ESXi host has two dual port host bus adapters, then revise the zoning above to allow access from each host initiator port to a single target port on a single node to keep the number of paths to each volume to a maximum of four. VMware ESXi 4.1 has a maximum of 256 SCSI devices per ESXi host with a maximum number of paths per host of 1024. The number of SCSI devices per host is limited if the number of paths hits 1024 paths. For example, 8 paths per SCSI device would yield 128 SCSI devices possible on a host (1024 / 8). As a result the recommendation is to limit the number of paths to anyone volume to no more than 4 paths. This will allow for the maximum number of 256 SCSI devices to attach to the ESXi host.
Object SCSI devices
VMware maximum 256
Paths per volume
Description The maximum number of SCSI devices supported by the VMware software. Each path to a volume equates to a single SCSI device IBM Storwize V7000 attachment recommendation.
Table 5: Maximum configuration for VMware ESXi 4.1 with Storwize V7000
Setting up VMware ESXi Boot from SAN

Creating the host object on Storwize V7000
To create the host object on Storwize V7000, in the management GUI interface select the Host Icon > All Hosts. In the All Hosts view, select New Host. For VMware ESXi Boot from SAN implementations on Storwize V7000, only fiber channel boot support is available at the time of this writing. For the latest update on iSCSI SAN boot support with Storwize V7000, see http://www.ibm.com/storage/storwizev7000 For the purpose of this paper we will be booting two VMware ESXi servers over SAN. Figure 6 shows the create host dialogue box for one of the ESXi servers.
Figure 6 Create Host
To create additional VMware ESXi hosts, follow the zoning guidelines and host creation discussed earlier for each ESXi host.
Boot Volume Creation and host mapping

Each ESXi host must use a dedicated volume for SAN boot. In this section you will create two volumes, one for ESXi01 and one for ESXi02. In the Storwize V7000 management interface select the Volume Icon > All Volumes. In the All Volumes section select New Volume. A dialogue box as shown in Figure 7 is displayed.
Figure 7 Volume creation
After selecting the desired preset for Generic, Thin Provisioned, Mirrored or Thin Mirror you have the option to select the storage pool that these volumes will provisioned from. In this example, there is a storage pool named VM_BOOT_POOL. Enter the desired volume name and capacity. The volume name must start with a letter and may contain numbers, spaces and underscores. In this example a combination of host name and intended purpose for the volume is used as the volume name, ESXi02_BOOT_VOL. You may create additional volumes by selecting the + symbol to the right of the capacity. If you click Create and Map to Host after the volumes have been created, the Modify Mappings window (as shown in Figure 8) appears. Select the host that you are working with, and from the left panel select the volume you wish to map to the host. Use the > symbol to map the volume to the host. To work with additional host mappings, click Apply instead of OK. Clicking Apply will keep the Modify Mapping window open and you can select another host in the upper-left corner of window.
Figure 8 Mapping Boot Volume
Preparing VMware ESXi Host

Refer to your individual server hardware manuals and vMware ESXi 4.1 Fiber Channel Configuration Guide. For newer systems with UEFI BIOS, additional steps maybe required to allow for VMware ESXi hosts to properly attach to SAN devices. While testing, the team used the IBM System x 3650 M3. To allow for fiber channel attached devices on VMware ESXi servers, you may need to enable Legacy Boot under the Boot menu. You can move the Legacy option to the top of the boot options when installing on IBM System x with UEFI BIOS. This setting is not specific for boot from SAN, even when booting from a locally attached boot device you may still need to enable Legacy boot to allow for the fiber channel devices to be discovered following a reboot of the ESXi server. Note: When setting up IBM BladeCenter for boot from SAN remove all connections to local disk if present. Figure 9 shows how you can change or set the boot order.
Figure 9: Setting boot order
For additional details on UEFI on IBM System x, see http://www01.ibm.com/support/docview.wss?uid=psg2MIGR-5079769 For specific instructions on configuring your individual host bus adapters for SAN boot refer to the VMware Fiber Channel SAN Configuration Guide document URL: http://www.vmware.com/pdf/vsphere4/r41/vsp_41_san_cfg.pdf For additional host bus adapter recommended settings on Storwize V7000, see http://www.ibm.com/storage/storwizev7000
Installing VMware ESXi on Storwize V7000 volume as a boot target for ESXi installation.
The next few figures show the configuration steps on the server to successfully choose a SAN volume and enable the fiber channel adapter to enable boot from SAN for Qlogic host bus adapters. For Emulex host bus adapters refer to the VMware Fiber Channel SAN Configuration Guide mentioned above. During boot when prompted press < Ctrl-q > or < Alt-q > to enter the Qlogic Fast!UTIL as shown in figure 10. Select the adapter to configure, target port WWPN and volume, to be used as the boot device. Below are the configuration steps to prepare the server for boot from SAN. Refer to the IBM Storwize V7000 Host Attachment Guide for the latest recommended adapter settings for VMware ESXi servers,
To prepare the server to boot from SAN: 1. Restart the server. 2. When you see the QLogic banner, press the Ctrl - q keys to open the FAST!UTIL menu panel. 3. From the Select Host Adapter menu, select the Adapter Type QLA2xxx. 4. From the Fast!UTIL Options menu, select Configuration Settings. 5. From the Configuration Settings menu, select Host Adapter Settings. 6. From the Host Adapter Settings menu, select the following values: a) Host Adapter BIOS: Enabled b) Frame size: 2048 c) Loop Reset Delay: 5 (minimum) d) Adapter Hard Loop ID: Disabled e) Hard Loop ID: 0 f) Spinup Delay: Disabled
g) Connection Options: 1 - point to point only h) Fibre Channel Tape Support: Disabled i) Data Rate: 2
7. Press Esc to return to the Configuration Settings menu. 8. From the Configuration Settings menu, select Advanced Adapter Settings. 9. From the Advanced Adapter Settings menu, set the following parameters: a) Execution throttle: 100 b) Luns per Target: 0 c) Enable LIP Reset: No d) Enable LIP Full Login: Yes e) Enable Target Reset: Yes f) Login Retry Count: 8
g) Port Down Retry Count: 8 h) Link Down Timeout: 10 i) j) Command Timeout: 20 Extended event logging: Disabled (might be enabled for debugging)
k) RIO Operation Mode: 0 l) Interrupt Delay Timer: 0
10. Press Esc to return to the Configuration Settings menu.

Press Esc. 11. From the Configuration settings window select Save changes to save the updates. 12. From the Fast!UTIL Options menu, Select Host Adapter and repeat steps 3 through 12 if more than one QLogic adapter was installed. 13. Restart the server. Repeat the process for any additional ESXi servers to be installed. The following figures are provided as visual reference of the configuration and installation process.
Figure 10: Entering Fast!UTIL
Figure 11: Selecting SAN volume for boot
Enable the Qlogic ROM BIOS to allow attached SAN devices to be used as a boot device. For ESXi servers that will be booting from local disk, leave the Qlogic ROM BIOS Disabled.
Figure 12: Enable Qlogic ROM BIOS
Figure 13 shows VMware ESX installation.
Figure 13: VMware ESXi Installation
Figure 14 shows the selecting of the SAN volume as the target for installation.
Figure 14: Selecting SAN volume
Figure 15: Installation complete
Adding additional volumes to the VMware vSphere cluster

After the successful installation of the VMware ESXi hosts and the vSphere Cluster configuration, create new volumes and host mappings for use as datastores within the cluster for the guest OS and application data. When creating the new volumes keep in mind that you can select different storage pools for the different types of application data. As an example, create two storage pools using managed disks that are protected using different RAID technologies. For the ESXi boot volumes and Guest OS datastores the team is using the
storage pool named VM_BOOT_POOL with RAID 6 protected nearline SAS managed disks. For application data the team provisioned volumes from the storage pool named VM_APP_POOL containing SAS managed disks using RAID 5 as protection. As a best practice when mapping the same volumes to multiple ESXi hosts keep the SCSI ID consistent between the ESXi hosts. This will help to maintain a consistent view of the storage devices during any troubleshooting tasks that may come up related to the storage view between ESXi servers. Figure 16 shows the Modify Mappings window that enables you to modify host mappings.
Figure 16: Modify host mappings
Diagnostic partition
For diskless ESXi hosts that boot from SAN, multiple ESXi host systems can share one diagnostic partition on a SAN volume. If more than one ESXi host system is using a volume as a diagnostic partition, that volume must be mapped so that all the servers are able to access it. Consider creating a separate volume for use as the diagnostic partition. Each ESXi host will require a minimum of 100MB storage space, so the size of the volume determines how many servers can share it. Each ESXi host is mapped to a diagnostic slot. If there is only one diagnostic slot on the storage device, then all ESXi hosts sharing that device also map to the same slot, which can create problems. For example, suppose you have configured 16 ESXi hosts in your environment and have allocated enough capacity for 16 slots, 1.6 GB, it is unlikely that any core dumps will be mapped to the same location on the diagnostic partition, even if two ESXi hosts perform a core dump at the same time. Figure 17 shows the final volume to host mapping for both ESXi01 and ESXi02.
Figure 17: Host mapping view
Discovering the volumes on vMware ESXi

From each ESXi Server rescan for new storage devices on the configuration tab and verify the volumes and paths from the Storwize V7000 are displayed correctly. Figure 18 shows the view from the storage adapters on the configuration tab showing two fiber channel adapters connected to two target ports per adapter with ten volumes accessible down each path. Consider changing the fiber channel device names to friendly names. In the example below in Figure 18 the team renamed the devices under ESXi to match the volume naming convention that was used in Storwize V7000. To rename a volume select the Configuration tab and then Storage Adapters from the top panel on the left. Select the fiber channel adapter in the Storage Adapters view to see the volumes associated with the adapter.
Figure 18: ESXi volume renaming
VMware ESXi Multipath Selection

VMware ESXi provides multipathing support natively in one of three forms, most recently used (MRU), fixed path, and round robin, but does not support dynamic load balancing. Storwize V7000 is an active / active virtualized storage subsystem, VMware and IBM recommend using round robin as the preferred multipathing policy. Using round robin reduces the complexity and management overhead required in maintaining MRU or fixed path policies. The default policy for new Storwize V7000 volumes discovered by ESXi is fixed path policy as of this writing. To change the preferred path management policy, from the Storage Adapters view on the Configuration tab, select the fiber channel adapter as shown in Figure 18, by right clicking a storage device and selecting Manage Paths. Select round robin from the path selection drop down menu and select the change button to the right. Figure 19 shows Manage Paths dialogue window.
Figure 19: Path selection policy
Create datastores and install Guest OS

After verifying volume access configure the VMware datastores and install your guest OS and applications in accordance with your virtualization strategy. When creating your datastores, a good practice is to name the datastores according to their intended purpose. In Figure 20 we have used VM_GUEST_BOOT_0n for the guest OS installations and VM_APP_VOL_0n for application data, n being a number. By using an effective and common naming convention between VMware and Storwize V7000 you are more easily able to locate volumes using a common name over using the SCSI ID or volume serial number.
Figure 20 vSphere Cluster Storage Resource Allocation
Summary
The combination of VMware software and IBM Storwize V7000 system offers significant advantages for companies consolidating servers and storage to reduce costs and increase operational efficiency. A virtualized storage infrastructure with VMware and Storwize V7000 can significantly improve administrator productivity, reduce performance bottlenecks, increase storage utilization and reduce the footprint of your storage systems in your datacenter. The Storwize V7000 system offers a range of built-in high performance, efficiency and flexibility features to help you achieve these goals. These features include: Built-in support for virtualization to help you flexibly and efficiently support the diverse storage demands of your business applications Automated data tiering to help you balance the cost of storage with the value of data Thin provisioning to reduce the amount of idle storage capacity you need to maintain Simple administration to potentially double administrator productivity, (so your administrators can spend their time helping to solve business problems)
IBM Storwize V7000 simplifies a company's overall storage infrastructure through virtualization, creating a dynamic virtualization infrastructure that adapts rapidly to changing needs for storage efficiency and data protection.
Appendix A: Materials list

Hardware Storage Vendor IBM IBM Hosts IBM IBM Software Hosts VMware VMware Microsoft vSphere ESXi Windows Server 2003 Enterprise Edition-SP1 (x86) 4.1 4.1 2003 Build 258902 Build 260247 Operating system Name XIV Storwize V7000 IBMX3650M3 IBMX3650M3 Version 10.2.2 6.1 D6E148B D6E148B UEFI BIOS Version UEFI BIOS Version Description
Appendix B: IBM Storwize V7000 best practices a quick reference

This section is a quick reference with a short description of best practices.
SAN zoning
For fault tolerance implement redundant fabrics, Storwize V7000 supports connectivity up to four (4) individual fabrics. Attach the Storwize V7000 ports and any external storage ports in the core switch of each fabric. External storage controllers and Storwize V7000 ports should be in the same switch or director.
VMware ESXi host zoning

Use single initiator zoning. Limit the number of paths to only a maximum of four paths to any volume.
External storage controller zoning

Zone external controller ports to all node canister ports on Storwize V7000 cluster. Create separate zones for each external storage subsystem. Do not group multiple storage subsystems in a single zone.
IBM Storwize V7000 Cluster Zoning

Create a zone that contains all the node canister ports in the Storwize V7000 system in a single zone.
Storage Pools
All managed disks within a storage pool should be of the same technology and RAID level. Use the same number of physical disks in each external RAID array assigned to a specific storage pool.
External storage pools

Create one LUN per RAID array; each external RAID array should only be included in one storage pool. Storage pools should only contain managed disks only from a single external storage controller.
Thin Provisioning
To maximize performance select a grain size of 256 KB when creating a thin provisioned volume
When using thin provisioned volumes as a FlashCopy source or target specify the same grain size for the volume and FlashCopy function. Set the cache mode to readwrite to maximize performance. With the cache mode set to none, the IBM Storwize V7000 can not cache the thin provisioned metadata, which decreases performance.
Migrating Volumes
Volumes can only be migrated only between storage pools with the same extent size. When adding Image mode managed disks for migration, ensure that the target storage pool has the same extent size as the storage pool holding the image-mode disks. Extent size of the storage pool cannot be changed dynamically. For striped or sequential volumes and storage pools with different extent sizes use the volume mirroring function.
VMware ESXi migration using IBM Storwize V7000 migration capability

To migrate existing VMware ESXi hosts to the IBM Storwize V7000 system using the Storwize V7000 migration capability you must change settings on the VMware ESXi host so that copies of the volumes can be recognized by the system after the migration is completed. To enable volume copies to be recognized by the VMware ESXi hosts, complete one of the following actions. Enable the EnableResignature setting. Disable the DisallowSnapshotLUN setting.
Alternately you can use Storage vMotion to migrate to the IBM Storwize V7000 system. Once the storage for the VMware ESXi hosts is virtualized behind the IBM Storwize V7000 system, the restrictions noted above do not exist when using the Storwize V7000 migration function during technology upgrades or redistribution within the Storwize V7000 system.
Appendix C: Resources
These websites provide useful references to supplement the information contained in this paper: IBM Systems on PartnerWorld ibm.com/partnerworld/systems IBM Publications Center www.elink.ibmlink.ibm.com/public/applications/publications/cgibin/pbi.cgi?CTY=US IBM Storwize V7000 http://www.ibm.com/storage/storwizev7000 IBM Redbooks ibm.com/rebooks IBM System x http://www-03.ibm.com/systems/x/hardware/index.html Transitioning to UEFI and IMM White Paper http://www-01.ibm.com/support/docview.wss?uid=psg2MIGR-5079769 IBM System Storage Interoperation Center (SSIC) http://www03.ibm.com/systems/support/storage/config/ssic/displayesssearchwithoutjs.wss?start_over=y es vMware http://www.vmware.com VMware vSphere 4.1 Documentation http://www.vmware.com/support/pubs/vs_pages/vsp_pubs_ESXi41_i_vc41.html Hardware Compatibility Guide http://www.vmware.com/resources/compatibility VMware Fiber Channel SAN Configuration Guidehttp://www.vmware.com/pdf/vsphere4/r41/vsp_41_san_cfg.pdf VMware Dynamic Storage Provisioning http://www.vmware.com/files/pdf/VMware-DynamicStorageProv-WP-EN.pdf
Trademarks and special notices

Copyright IBM Corporation 2010. All Rights Reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml. VMware is a registered trademark or trademark of VMware, Inc. in the United States, other countries, or both. Qlogic is a registered trademark or trademark of Qlogic Corporation in the United States, other countries for both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be
given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

0002 V7000 VMware Storage Practices

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

0002 V7000 VMware Storage Practices

Caricato da

Copyright:

Formati disponibili

Deploying IBM Storwize V7000 in VMware Environments

Systems and Technology Group ISV Enablement December 2010

Copyright IBM Corporation, 2010.

Strategic Configuration Considerations ................................................................................. 5

Attaching VMware ESXi to IBM Storwize V7000 ................................................................... 13

Appendix C: Resources ......................................................................................................... 30 Trademarks and special notices ........................................................................................... 31

Purpose of the document

VMware vSphere Hypervisor

IBM Storwize V7000

Figure 1 IBM Storwize V7000 virtualizing external and internal storage

IBM Storwize V7000 Concepts

Figure 2 Storwize V7000 virtualization components

For more information about IBM Storwize V7000, see http://www.ibm.com/storage/storwizev7000

Strategic Configuration Considerations

Managing the Information Infrastructure

RAID Options for Data Protection

Drive Count 1-8

Fault Tolerance 0 (none)

Table 1: IBM Storwize V7000 RAID Levels

Tiers for efficiency and performance

External storage fabric zoning

Zone Storwize V7000 Zone

Storwize V7000 ExtCntrl Zone

Table 2: Zoning for external devices

Figure 3 shows the logical view of the externally attached disk.

Storwize V7000 External Storage Array

Figure 3 Logical view for externally attached disk

Figure 4 Storage Pool Creation

Hybrid storage pool IBM Easy Tier

Figure 5 Easy Tier

Thin provisioning storage allocation with VMware ESXi

Create virtual machine on VMFS and

create .vmdk file

Install operating system in virtual machine

Deploy a virtual machine from a template

Delete data within the virtual machine

Attaching VMware ESXi to IBM Storwize V7000

SAN Fabric Zoning for VMware ESXi Hosts

node1-port3 / node2-port3 ESXi02-port1

Table 4: ESXi Fabric Zones

Object SCSI devices

VMware maximum 256

Paths per volume

Setting up VMware ESXi Boot from SAN

Figure 6 Create Host

Boot Volume Creation and host mapping

Figure 7 Volume creation

Figure 8 Mapping Boot Volume

Preparing VMware ESXi Host

Figure 9: Setting boot order

k) RIO Operation Mode: 0 l) Interrupt Delay Timer: 0

10. Press Esc to return to the Configuration Settings menu.

Figure 10: Entering Fast!UTIL

Figure 11: Selecting SAN volume for boot

Figure 12: Enable Qlogic ROM BIOS

Figure 13 shows VMware ESX installation.

Figure 13: VMware ESXi Installation

Figure 14: Selecting SAN volume

Figure 15: Installation complete

Adding additional volumes to the VMware vSphere cluster

Figure 16: Modify host mappings

Figure 17: Host mapping view

Discovering the volumes on vMware ESXi

Figure 18: ESXi volume renaming