Open Replicator Data Migration of File Server Clusters to VMware

2008 EMC Proven TM Professional Knowledge Sharing

Brian Russell, Sr. Storage Administrator Michael Aldo, Windows System Engineer A Leading Healthcare Insurer

EMC Proven Professional Knowledge Sharing 2008

Table of Contents
INTRODUCTION AND BACKGROUND ........................................................................................................... 2 OUR APPROACH ......................................................................................................................................... 2 PLANNING THE STORAGE MIGRATION ........................................................................................................ 3 Migration Overview .............................................................................................................................. 3 Its All In the Details! (Prep-Work) ..................................................................................................... 4 What is Open Replicator? ..................................................................................................................... 4 Open Replicator Considerations........................................................................................................... 5 Software / Licensing Used in this migration ......................................................................................... 5 Open Replicator Prep-Work ................................................................................................................. 5 Document Existing MSCS File Cluster Capacity.................................................................................. 6 Create New Storage for VMware Virtualized MSCS Cluster................................................................ 8 Document Open Replicator Pairs ....................................................................................................... 14 Zone and Mask New DMX-3 Capacity to VMware............................................................................. 16 FA to FA Zoning for Open Replicator ................................................................................................ 17 SAN Cabling........................................................................................................................................ 18 FA to FA Masking for Open Replicator .............................................................................................. 18 VMWARE / MS CLUSTER PREP-WORK ..................................................................................................... 19 Fresh-Build MSCS .............................................................................................................................. 20
Adding Shared Storage ................................................................................................................................... 21 Disaster Recovery Cluster Restore.................................................................................................................. 22

IMPLEMENT MIGRATION .......................................................................................................................... 24 Create Microsoft Cluster .................................................................................................................... 26 Configuration of the Second Node ...................................................................................................... 28 P2V Migration with VMware Converter............................................................................................. 28
Physical-to-Virtual Migration ......................................................................................................................... 28 Adding Shared Storage ................................................................................................................................... 29 Virtual Machine Configuration ....................................................................................................................... 29 Physical-to-Virtual Cleanup Tasks.................................................................................................................. 30

OPEN REPLICATOR POST-MIGRATION TASKS ........................................................................................... 30 CONCLUSION ............................................................................................................................................ 30

Disclaimer: The views, processes or methodologies published in this article are those of the authors. They do not necessarily reflect EMC Corporations views, processes or methodologies.

Introduction and Background

We would like to share a real-world challenge and our solution. Our challenge was to migrate a large file server cluster into a virtualized server environment. Thats not all, we had to combine this with a storage array hardware refresh. We are a Symmetrix shop with a growing VMware infrastructure. The Storage Team has adopted a rolling hardware refresh strategy. We replace the oldest storage array (EMC Symmetrix DMX-3000) allowing us to introduce newer technology (EMC Symmetrix DMX-3) every year. This hardware refresh requires us to migrate all hosts off the 4-year old disk array over to the new disk array. Four Windows 2003 Microsoft Cluster Server (MSCS) File Servers, and eight nodes connecting to 16 TB of protected storage are included. Simultaneously, the Wintel Team has a similar hardware refresh strategy requiring them to replace the MSCS File Servers going out of maintenance. Given the complexity and increased business dependence on the environment, the team had to develop an approach that caused minimal downtime and allowed quick rollback.

Our Approach
We decided to migrate the physical File Clusters to our existing VMware infrastructure along with presenting new Symmetrix DMX-3 storage to these virtual hosts. We copied the file server data to the new storage array via Open Replicator.

Figure 1 Migration Overview

EMC ProvenTM Professional Knowledge Sharing 2008

We combined multiple supported solutions from EMC, VMware and Microsoft to facilitate our two concurrent migrations. We only experienced a single, brief downtime during migration; this was one of the most exciting outcomes of our approach!

Planning the Storage Migration

Migration Overview
Our MSCS File Servers access storage on two separate Symmetrix DMX-3000 arrays. To consolidate storage and simplify the future VMware environment, we decided to replicate all capacity from both existing arrays to a single, new Symmetrix DMX-3. In this article, we will follow one of our four MSCS File Servers through the migration process.

The next few sections will demonstrate how we presented new, replicated storage to VMware ESX 3.0 Servers. We recreated the physical cluster nodes as Virtual Machines to operate across multiple ESX Servers. The MSCS physical disk resources, which hold the file share data, now point to the replicated SAN LUNs using a VMware feature called Raw Device Mapping (RDM).

The Storage and VMware administrators performed most of the preparatory work in advance and in parallel.
Figure 2 illustrates the flow of major tasks and their

dependencies. It is a valuable reference as you read the steps presented in this article.

EMC ProvenTM Professional Knowledge Sharing 2008

Figure 2 Migration Task Flowchart

Throughout this document, Open Replicator may be abbreviated as OR.

Its all In the Details! (Prep-Work)

Give yourself plenty of time to plan the storage migration. There are many components involved in any storage configuration change, and you do not want any surprises on migration day. Open Replicator introduces some additional configuration planning, but it is well worth it. Where we would normally leverage tape restore or network file copy, we can now perform block-level replication between two independent disk frames and allow host access to storage while data is in transit. The overhead of this copy process runs entirely on the Symmetrix DMX over the SAN to remote devices.

What is Open Replicator?

Open Replicator for Symmetrix is software that provides a method for copying data from various types of arrays within a Storage Area Network (SAN) infrastructure to or from a Symmetrix DMX array. We configure and control Open Replicator sessions using the symrcopy command (available in Enginuity version 5671 and above). control Symmetrix frame needs to be at version 5671 or above.
EMC ProvenTM Professional Knowledge Sharing 2008

Only the

Two licenses are available for Open Replicator. Open Replicator/LM is used for online data pulls only (to Symmetrix DMX). Open Replicator/DM is used for everything except online pulls. For more information about Open Replicator for Symmetrix please refer to the Solutions Enabler Symmetrix Open Replicator CLI Product guide available on Powerlink.

Open Replicator Considerations

We must consider some Open Replicator restrictions. For example, the target capacity for each device must be equal to or larger than the source, although protection levels and Meta volume configuration do not need to be identical. Most commonly, we

leverage Open Replicator to perform in-place migrations where we present new (replicated) storage to an existing UNIX/Windows host. In the end, the data on the new storage looks identical to the host with the exception of the LUNS that may be larger than the original.
Note: For detailed considerations and restrictions, refer to the EMC Solutions Enabler Symmetrix Open Replicator manual.

Software / Licensing Used in this migration

Always consult the EMC Support Matrix for current interoperability information. o o o o o o VMware ESX Server Enterprise 3.0.1 VMware VirtualCenter 2.0.1 VMware Converter Enterprise Edition 3.0.2 Windows Server 2003 R2 Microsoft Cluster Server 2003 Solutions Enabler 6.4.04 was used with the following features: o License Key for BASE / Symmetrix o License Key for ConfigChange / Symmetrix o License Key for Device Masking / Symmetrix o License Key for SYMAPI Feature: RcopyOnlinePull / Symmetrix Open Replicator/LM required on the control Symmetrix

Open Replicator Prep-Work

Here are the steps we need to prepare for: o o o o o o o Create new storage for Virtualized MSCS Cluster (OR Control devices) Set FA Port Flags for new VMware Storage (OR Control Devices) Map new VMware Storage to FA ports FA to FA Zoning (open replicator session) FA to FA Masking (open replicator session) SAN Cabling (ISLs and Hosts) Create Open Replicator Pairing File

EMC ProvenTM Professional Knowledge Sharing 2008

Document Existing MSCS File Cluster Capacity

First, we collected information about the existing capacity assigned to the MSCS Servers. We will be creating identical storage on the DMX-3, so we need to know the number of Symmetrix Devices we are replicating as well as capacity of each device.

Solutions Enabler (symmaskdb) output shows all devices masked to each cluster node on both DMX-3000s (see below). This output displays the Symmetrix device IDs,

capacity, and director mapping. All of this information is helpful in prep-work.

The MSCS Cluster Nodes A and B server names are fscl1a and fscl1b, respectively, and these names may be interchanged in our examples.

symmaskdb -sid 7667 list capacity -host fscl1a Symmetrix ID Host Name : 000187827667 : fscl1a

symmaskdb -sid 7667 list capacity -host fscl1b Symmetrix ID Host Name : 000187827667 : fscl1b

Identifiers Found : 10000000c93aea8f 10000000c93ae961 Device -----000B 003B 0F79 0F91 0FA9 0FC1 0FD9 0FF1 1159 1163 11B3 Cap(MB) ------2 2 414270 414270 621405 414270 414270 414270 129459 129459 4315 Attr ---(M) (M) (M) (M) (M) (M) (M) (M) Dir:P ---7C:1 10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1

Identifiers Found : 10000000c93ae9a6 10000000c93ae905 Device -----000C 003C 0F79 0F91 0FA9 0FC1 0FD9 0FF1 1159 1163 11B3 Cap(MB) ------2 2 414270 414270 621405 414270 414270 414270 129459 129459 4315 Attr ---(M) (M) (M) (M) (M) (M) (M) (M) Dir:P ---7C:1 10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1 7C:1,10C:1

----------------------------MB Total:2955992 GB Total: 2886.7 symmaskdb -sid 6776 list capacity -host fscl1a Symmetrix ID Host Name : 000187886776 : fscl1a

----------------------------MB Total:2955992 GB Total: 2886.7 symmaskdb -sid 6776 list capacity -host fscl1b Symmetrix ID Host Name : 000187886776 : fscl1b

Identifiers Found : 10000000c93ae961 10000000c93aea8f Device -----15F7 16F3 170B Cap(MB) ------621405 414270 414270 Attr ---(M) (M) (M) Dir:P ---8C:1, 9C:1 8C:1, 9C:1 8C:1, 9C:1

Identifiers Found : 10000000c93ae905 10000000c93ae9a6 Device -----15F7 16F3 170B Cap(MB) ------621405 414270 414270 Attr ---(M) (M) (M) Dir:P ---8C:1, 9C:1 8C:1, 9C:1 8C:1, 9C:1

----------------------------MB Total:1449945 GB Total: 1416.0

----------------------------MB Total:1449945 GB Total: 1416.0

Figure 3 Using symmaskdb command to show host capacity

Note: You must first rename the alias wwn (awwn) in the masking database in order for the above symmask list capacity command to work with a user defined host name. We have Solutions Enabler installed on each cluster node, so we found the easiest way to accomplish this was to use the symmask discover hba rename -v command (as opposed to using symmask rename).

EMC ProvenTM Professional Knowledge Sharing 2008

As you can see here, symmask discover performs the operation for each visible Symmetrix:
c:\Program Files\EMC\SYMCLI\bin>symmask discover hba -rename -v Symmetrix ID : 000187827667 Device Masking Status : Success WWN ip Address Type User Name WWN ip Address Type User Name : : : : : : : : 10000000c93aea8f N/A Fibre fscl1a/10000000c93aea8f 10000000c93ae961 N/A Fibre fscl1a/10000000c93ae961

Symmetrix ID : 000187886776 Device Masking Status : Success WWN ip Address Type User Name WWN ip Address Type User Name : : : : : : : : 10000000c93ae961 N/A Fibre fscl1a/10000000c93ae961 10000000c93aea8f N/A Fibre fscl1a/10000000c93aea8f

The symmaskdb list capacity output in Figure 3 shows twelve devices assigned to each node, excluding the gatekeepers. We need to determine the association between logical volumes and each Symmetrix device, if they need to be created on the new storage array, and if they need to be included in the Open Replicator session.

We pulled output from two more commands to begin matching these devices with logical volumes. First, we used Solutions Enabler (syminq) to capture output from each node of the cluster. You can also use sympd list, which has cleaner output but requires you to update the local symapi database (symcfg discover) on each node. The second command is an NT Resource Kit utility, called dumpcfg.exe. (We also used this utility later in the Disaster Recovery Cluster Restore section of the MSCS Fresh-Build for VMware). The output from both syminq and dumpcfg commands is shown below. Figure 4 shows how the dumpcfg Disk Number, Volume Label, and Drive letter related to the syminq PHYSICALDRIVE# and Symmetrix ID. This relationship is important as we prepared our pairing spreadsheet for Open Replicator.

EMC ProvenTM Professional Knowledge Sharing 2008

Figure 4

(dumpcfg.exe output related to syminq)

Using this detail about each Symmetrix device in the MSCS Cluster, we worked with our Windows System Administrator to determine which volumes needed to be re-created on the DMX-3 and which needed to be replicated. In this experience, all shared disk

resources had to be re-created and all had to be replicated, with one exception. The MSCS Quorum devices did not need to be replicated because we performed a fresh build of the MSCS inside VMware.

Note: You will need to replicate the Quorum if you use the VMware Converter approach to moving the MSCS Nodes to VMware

Create New Storage for VMware Virtualized MSCS Cluster

Now that we have documented our existing MSCS Capacity, we know we have 11 Meta Devices which represent our file share data volumes and one Hyper Volume assigned and used as the cluster quorum drive. There were three different size volumes used for file share volumes. All of this storage, including cluster quorum, had to be re-created on the new DMX-3.

EMC ProvenTM Professional Knowledge Sharing 2008

The chart in Figure 5 represents the three size volumes used for the file share disks on the physical clusters. The chart also shows how we determined the number of required Hyper Volume Extensions (HVE) to form our new Meta Devices. The original capacity consisted of (2) 126.42 GB volumes, (7) 404.56 volumes and (2) 606.84 GB volumes. To support Open Replicator, we had to create devices of equal or greater capacity. To arrive at our new HVE count (per volume), we simply divided the original capacity in MB by the standard HVE we use on the DMX-3 and rounded up from there (making the new volumes slightly larger). On the new DMX-3, our standard HVE size is 35700 MB which equates to 38080 cylinders on this 64k Track FBA System. For example, we took the (2) 129459 MB original volumes and divided those by the new HVE size of 35700 MB to get 3.6 required HVEs. Obviously, we cannot form a new Meta using 3.6 HVEs, so we just rounded up to 4. Since we needed 2 of these Meta Volumes, we required 8 total HVEs. The larger HVEs yielded a slightly larger Meta Volume 142800 MB.

The total HVEs Required column in Figure 5 simply represents the rounded number of HVEs multiplied by the quantity of Meta Volumes required.

Qty

Original Capacity MB

Divide by Standard HVE Size And round up for new HVE / 35700 = 3.6 round to 4 / 35700 = 11.6 round to 12 / 35700 = 17.4 round to 18

Total HVEs Required 8 84 36

New Meta Size (MB)

2 7 2

129459 414270 621405

35700 * 4 = 142800 35700 * 12 = 428400 35700 * 18 = 642600

Figure 5 New Storage Device Size Calculations

We added the total HVEs required column and determined that we needed free space to support a total of (128) 38080 cylinder (35700 MB) HVEs.
Note: In DMX (FBA) 64K Track architecture, you specify a number of cylinders to determine the Logical Volume size. Cylinder = 15 Tracks; TRACK = 128 blocks; One BLOCK = 512 bytes.

DMX3 Logical Volume of n cylinders has a useable block capacity of n * 15 * 128 So, one cylinder is 983040 bytes (960 kilobytes)

Important: To control how Solutions Enabler will report on device sizes, note the following parameter in the symapi options file: SYMAPI_TRACK_SIZE_32K_COMPATIBLE. If this

parameter is set to DISABLED, it will report tracks in native format based on frame type. See the

EMC Solutions Enabler Symmetrix CLI Command Reference for more details 9
EMC ProvenTM Professional Knowledge Sharing 2008

Look for Free Space We tier our capacity inside the DMX-3 using disk groups to isolate spindle speeds and drive sizes. We targeted the file server capacity on 300 GB, 10k RPM drives using RAID-5 protection. We used 2-Way Mir protection for the cluster quorum devices.

First, we identified free configured space (unmapped HVEs) on the 300 GB drives. Then we looked for available unconfigured capacity to be created into the remaining required HVEs. Using the symdisk list by_diskgroup command we identified the 300 GB drives in disk_group 1 and also more in disk_group 5. To identify the unmapped HVEs, we used the symdev list noport [disk_group #] command, where disk_group # was disk_group 1 and disk_group 5. We found (44) free 35700 MB Raid-5 HVEs too, but we still need (84) more.

We checked if there was enough free space on the disks before we created new HVEs inside either of these disk groups. We took the output from symdisk list [disk_group 1], for example, and entered it into Excel to arrive at the total number of HVEs we could create in a disk_group We divided the Free MB column by 35700 and then passed the results to the FLOOR function to round down to the nearest integer. Then we added all the results to determine how many HVEs we could make. We had plenty of capacity in this disk group; so we went on to create the Hypers.

A Word about Physical Distribution You need to be mindful of the underlying physical spindles whenever you are creating devices and forming meta volumes. This is a little more complicated when using RAID5 devices. Our DMX-3 is configured for RAID5 (3+1) which means each RAID5 hyper will span (4) physical spindles. The symdev output in Figure 6 shows a (12) member meta volume (device 12C1) consisting of RAID5 hyper volumes. You see that each RAID5 hyper volume spans (4) physical disks. Our goal was to ensure that all members of a meta volume are on different physical spindles. Because this is a (12) member meta consisting of RAID5 hypers of which each span (4) spindles, the meta volume is spread across (48) physical disks. The Enginuity code spreads the RAID5 hyper devices

provided you have enough physical spindles.

EMC ProvenTM Professional Knowledge Sharing 2008

10

#symdev -sid 6491 show 12c1 (filtered output showing Meta Volume breakdown Raid-5 along with HyperVolume Raid member to physical disk association)
RAID-5 Hyper Devices (3+1): { Device : 12C1 (M) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------02D:D5 334 22 11912 4 RW N/A 1 286102 06D:D5 246 22 11912 2 RW N/A 1 286102 12D:D5 252 22 11912 1 RW N/A 1 286102 16D:D5 334 22 11912 3 RW N/A 1 286102 } Device : 12C2 (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------01A:D7 358 22 11912 3 RW N/A 1 286102 05A:D7 276 22 11912 1 RW N/A 1 286102 11A:D7 271 22 11912 2 RW N/A 1 286102 15A:D7 359 22 11912 4 RW N/A 1 286102 } Device : 12C3 (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------02C:C7 94 22 11912 1 RW N/A 1 286102 06C:C7 94 22 11912 3 RW N/A 1 286102 12C:C7 94 22 11912 2 RW N/A 1 286102 16C:C7 94 22 11912 4 RW N/A 1 286102 } Device : 12C4 (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------01B:C7 94 22 11912 4 RW N/A 1 286102 05B:C7 94 22 11912 2 RW N/A 1 286102 11B:C7 94 22 11912 3 RW N/A 1 286102 15B:C7 94 22 11912 1 RW N/A 1 286102 } Device : 12C5 (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------02A:Da 413 22 11912 4 RW N/A 1 286102 06A:Da 301 22 11912 2 RW N/A 1 286102 12A:Da 333 22 11912 1 RW N/A 1 286102 16A:Da 414 22 11912 3 RW N/A 1 286102 } Device : 12C6 (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------01C:Ca 142 22 11912 4 RW N/A 1 286102 05C:Ca 142 22 11912 2 RW N/A 1 286102 11C:Ca 142 22 11912 3 RW N/A 1 286102 15C:Ca 142 22 11912 1 RW N/A 1 286102 } Device : 12C7 (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------02B:C8 118 22 11912 1 RW N/A 1 286102 06B:C8 118 22 11912 3 RW N/A 1 286102 12B:C8 119 22 11912 2 RW N/A 1 286102 16B:C8 119 22 11912 4 RW N/A 1 286102 } Device : 12C8 (m) {

11

EMC ProvenTM Professional Knowledge Sharing 2008

-------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------02A:C7 94 22 11912 1 RW N/A 1 286102 06A:C7 94 22 11912 3 RW N/A 1 286102 12A:C7 94 22 11912 2 RW N/A 1 286102 16A:C7 94 22 11912 4 RW N/A 1 286102 } Device : 12C9 (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------01D:C7 94 22 11912 4 RW N/A 1 286102 05D:C7 94 22 11912 2 RW N/A 1 286102 11D:C7 94 22 11912 3 RW N/A 1 286102 15D:C7 94 22 11912 1 RW N/A 1 286102 } Device : 12CA (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------02B:D7 366 22 11912 4 RW N/A 1 286102 06B:D7 278 22 11912 2 RW N/A 1 286102 12B:D7 285 22 11912 1 RW N/A 1 286102 16B:D7 367 22 11912 3 RW N/A 1 286102 } Device : 12CB (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------01C:D7 357 22 11912 3 RW N/A 1 286102 05C:D7 275 22 11912 1 RW N/A 1 286102 11C:D7 269 22 11912 2 RW N/A 1 286102 15C:D7 357 22 11912 4 RW N/A 1 286102 } Device : 12CC (m) { -------------------------------------------------------------Disk DA Hyper Member Spare Disk DA :IT Vol# Num Cap(MB) Num Status Status Grp# Cap(MB) -------------------------------------------------------------02D:D7 358 22 11912 4 RW N/A 1 286102 06D:D7 270 22 11912 2 RW N/A 1 286102 12D:D7 276 22 11912 1 RW N/A 1 286102 16D:D7 358 22 11912 3 RW N/A 1 286102 } } }

Figure 6 Symdev Output (12) Member Meta on (48) Spindles

Create Hyper Volumes (symconfigure) We created (84) Raid-5 35700 MB hypers and one 4316 MB hyper using free space in disk_group 1 using Solutions Enabler (symconfigure) to perform a Symmetrix Configuration change. Quorum. The single hyper volume will be used as the MSCS Cluster

Figure 8 shows the symconfigure file we used to create our new volumes.

EMC ProvenTM Professional Knowledge Sharing 2008

12

Form Meta Volumes We created another file to be used in a configuration change session to form the meta volumes. Figure 8 shows the symconfigure file we used to form our Meta Volumes.

These are the required meta volumes: o o o Create (2) 4-member meta volumes Create (7) 12-member meta volumes Create (2) 18-member meta volumes

Set FA Port Flags for new VMware Storage We created another symconfigure file to be used in a change session to update the bit settings for the VMware servers. We consulted the following EMC Techbook for our port settings: VMware ESX Server Using EMC Symmetrix Storage Systems Solutions Guide. For more details on the SPC2 bit settings, see the EMC Host Connectivity Guide for VMware ESX Server. And of course, always consult the EMC Support Matrix for up-to-date port setting requirements. To view port flags on one of our New DMX-3 FA ports, we used the following command: symcfg -sid 6491 -sa 10d -p 0 -v list
SCSI Flags { Negotiate_Reset(N) Soft_Reset(S) Environ_Set(E) HP3000_Mode(B) Common_Serial_Number(C) Disable_Q_Reset_on_UA(D) Sunapee(SCL) Siemens(S) Sequent(SEQ) Avoid_Reset_Broadcast(ARB) Server_On_AS400(A4S) SCSI_3(SC3) SPC2_Protocol_Version(SPC2) SCSI_Support1(OS2007) } : : : : : : : : : : : : : : Disabled Disabled Disabled Disabled Enabled Disabled Disabled Disabled Disabled Disabled Disabled Enabled Enabled Disabled

Fibre Specific Flags { Volume_Set_Addressing(V) Non_Participating(NP) Init_Point_to_Point(PP) Unique_WWN(UWN) VCM_State(VCM) OpenVMS(OVMS) AS400(AS4) Auto_Negotiate(EAN) }

: : : : : : : :

Disabled Disabled Enabled Enabled Enabled Disabled Disabled Enabled

Figure 7 Symmetrix FA Port Flags Configured for VMware

NOTE: From an array management best practice perspective, we do not mix operating system types on the same fibre channel port. We can then manage flags/characteristics/attributes at the fibre port level. It is possible to set the port with heterogeneous characteristics and then manage the individual characteristics or attributes on an initiator basis using the symmask command. Just remember that the settings must be compatible with every system using that fibre port channel

13

EMC ProvenTM Professional Knowledge Sharing 2008

when setting flags at the port level. Please refer to the Solutions Enabler Symmetrix Array Controls CLI Product Guide and the Solutions Enabler CLI Command Reference guide available on Powerlink.

Map New VMware Storage to FA ports The final Symmetrix configuration change was to create a mapping file to assign all new meta volumes and the quorum LUN to the FA ports dedicated to our VMware hosts. See Figure 8 for the config change mapping file.

Symmetrix Configuration Changes For detailed instructions on using the symconfigure command, please see the EMC Solutions Enabler Symmetrix Array Controls CLI. (We used version 6.4)
symconfigure sid SymmID f preview prepare commit

CmdFile

Figure 8 Steps to Configure New Space

Document Open Replicator Pairs

We created a detailed spreadsheet to track the original logical volume relationship of Open Replicator Remote (source) devices paired with Open Replicator Control (target) devices. We populated the spreadsheet with syminq.exe output from each physical cluster node and matched the information in the dumpcfg.exe output. We also matched the newly created Open Replicator Control devices based on capacity, and included the new LUN address information in both HEX and DEC format. The VMware administrator found this pairing sheet valuable; he could compare source Logical Volume to the new

EMC ProvenTM Professional Knowledge Sharing 2008

14

DMX-3 device sizes with decimal LUN ID. The decimal LUN ID is needed to identify volumes scanned into VMware and assign Raw Device Mapping to the correct virtualized cluster nodes. HEX LUN addresses were pulled from our mapping

configuration file. You can also use list LUN address for devs already mapped to a director.

The following command would list addresses for FA7DA: symcfg sid 6491 -fa
p1 address list.

7d

We then used Excels HEX2DEC function to convert the addresses

to decimal.

Figure 9 Open Replicator Pairing Sheet

Create Open Replicator Pairing File to be Used with Symrcopy We used the pairing spreadsheet to isolate which devices we needed to replicate. The correct format to be used with Open Replicator:
#OR_CONTROL FILE FSCL1 #OR CONTROL DEVS ON LEFTREMOTE DEVS ON RIGHT symdev=000190106491:12C1 symdev=000190106491:12CD symdev=000190106491:0EDE symdev=000190106491:12D9 symdev=000190106491:12E5 symdev=000190106491:12F1 symdev=000190106491:0FBA symdev=000190106491:0FBE symdev=000190106491:0EF0 symdev=000190106491:12FD symdev=000190106491:1309 symdev=000187827667:0F79 symdev=000187827667:0F91 symdev=000187827667:0FA9 symdev=000187827667:0FC1 symdev=000187827667:0FD9 symdev=000187827667:0FF1 symdev=000187827667:1159 symdev=000187827667:1163 symdev=000187886776:15F7 symdev=000187886776:16F3 symdev=000187886776:170B

Figure 10 Open Replicator Pairing file

15

EMC ProvenTM Professional Knowledge Sharing 2008

Note: For detailed syntax of the symrcopy command, refer to the EMC Solutions Enabler Symmetrix CLI Command Reference manual.

Zone and Mask New DMX-3 Capacity to VMware

The VMware environment consists of 3 physical VMware servers (APESX1, APESX2, and APESX3). Each has four HBAs, two per fabric, zoned to DMX-3 SymmID 6491 FA7DA and FA10DA. The four HBAs are referred to as A, B, C and D (see Figure 11):

SYMMETRIX ID 6491 FA7DA 10000000c94cacb4 is APESX1-A 10000000c94caee6 is APESX2-A 10000000c962ac07 is APESX3-A 10000000c94cadfd is APESX1-C 10000000c9676a4a is APESX3-C 10000000c94cad67 is APESX2-C

SYMMETRIX ID 6491 FA7DA 10000000c94cacb3 is APESX1-B 10000000c94caee5 is APESX2-B 10000000c962ac06 is APESX3-B 10000000c94cadfe is APESX1-D 10000000c94cad68 is APESX2-D 10000000c9676a4b is APESX3-D

Figure 11 VMware HBA WWNs zoned to storage FAs

Note: Our VMware server paths to storage and masking are designed to support EMC Static Load balancing using VMware preferred paths as documented in the EMC Techbook, VMware ESX Server Using EMC Symmetrix Storage Systems Version 2.0.

We decided to mask MSCS Node A resources to HBAs A and B and similarly mask the Node B resources to HBAs C and D to support load balancing on our VMware paths to storage. Later, the VMware administrator will configure preferred paths in VMware.
#Mask devs to (ESX A and C) WWN's on FA7DA #Devs associated with MSCS Cluster NODE-B Resources: symmask -sid 6491 -wwn 10000000c962ac07 add devs 12D9,12E5,12F1,0FBE,0EF0,1309 symmask -sid 6491 -wwn 10000000c94caee6 add devs 12D9,12E5,12F1,0FBE,0EF0,1309 symmask -sid 6491 -wwn 10000000c94cacb4 add devs 12D9,12E5,12F1,0FBE,0EF0,1309 #Devs associated with MSCS Cluster NODE-A Resources: symmask -sid 6491 -wwn 10000000c94cad67 add devs 12C1,12CD,0EDE,0FBA,12FD -dir symmask -sid 6491 -wwn 10000000c94cadfd add devs 12C1,12CD,0EDE,0FBA,12FD -dir symmask -sid 6491 -wwn 10000000c9676a4a add devs 12C1,12CD,0EDE,0FBA,12FD -dir #Mask devs to (ESX HBA B and HBA D) WWN's to FA10DA #These Devs associated with MSCS Cluster NODE-B Resources: symmask -sid 6491 -wwn 10000000c94cadfe add devs 12D9,12E5,12F1,0FBE,0EF0,1309 symmask -sid 6491 -wwn 10000000c94cad68 add devs 12D9,12E5,12F1,0FBE,0EF0,1309 symmask -sid 6491 -wwn 10000000c9676a4b add devs 12D9,12E5,12F1,0FBE,0EF0,1309 #Devs associated with MSCS Cluster NODE-A Resources: symmask -sid 6491 -wwn 10000000c94cacb3 add devs 12C1,12CD,0EDE,0FBA,12FD -dir symmask -sid 6491 -wwn 10000000c94caee5 add devs 12C1,12CD,0EDE,0FBA,12FD -dir symmask -sid 6491 -wwn 10000000c962ac06 add devs 12C1,12CD,0EDE,0FBA,12FD -dir

-dir 7d -p 0 -dir 7d -p 0 -dir 7d -p 0 7d -p 0 7d -p 0 7d -p 0

-dir 10d -p 0 -dir 10d -p 0 -dir 10d -p 0 10d -p 0 10d -p 0 10d -p 0

# This masking is designed to support VMware best practice for Static Load Balancing as well as distribute IO traffic between all 4 HBAs.

Figure 12 VMware Masking for MSCS Cluster devs

EMC ProvenTM Professional Knowledge Sharing 2008

16

I gave my VMware system administrator the OR pairing spreadsheet after we zoned and masked new devices to VMware. He immediately began scanning for the new storage devices. Using the new LUN ID references in the spreadsheet, which correspond to the original MSCS volumes, it was relatively easy to identify the scanned-in devices and assign storage to the new VMware Guest MSCS Nodes using Raw Device Mapping. See the VMware section later in this document entitled Adding Shared Storage.

No steps were required to align the new devices since the track partition alignment used on the original Windows operating systems was replicated to the new storage devices. Please remember that if partitions were not aligned properly before, you will not be able to modify the track alignment on the new storage devices.

Note: For more information on proper alignment of disk partition, refer to the EMC Engineering white paper Using diskpar and diskpart to Align Partitions on Windows Basic and Dynamic Disks.

FA to FA Zoning for Open Replicator

The output from syminq, on each MSCS node, confirms which FA ports were associated with each cluster HBA wwn. We then created new zones for the DMX-3000 FA ports and the DMX-3 FA ports Using Connectrix Manager . We immediately moved the new zones to our production zone set and activated them. This allowed us to proceed to the next step and prepare our OR masking using actual FA logon information. See Figure
13 to see our Open Replicator FA to FA topology and zone names we created.

17

EMC ProvenTM Professional Knowledge Sharing 2008

Figure 13 FA to FA Topology and OR Zone Names

SAN Cabling
The Open Replicator Remote Devices are on DMX-3000 FA ports configured at 2 GB. Open Replicator Control device FA ports are 4 GB capable. We should have been able to maximize the available throughput for all FA ports involved since we were pulling data from two different 2 GB capable DMX-3000s to 4 GB Control FAs.

The only potential bottleneck was the number of ISLs and hops between the FA ports involved in the Open Replicator session. So, we added (2) 2 GB ISLs (per fabric) directly between the two Connectrix directors attached to new and old storage.

FA to FA Masking for Open Replicator

We added Open Replicator masking entries to both Symmetrix DMX-3000s, allowing the DMX-3 Control FAs to access the remote storage devices. We knew which devices to include based on our MSCS File Cluster Capacity research done earlier.

With the FA to FA zones in place, we confirmed the zones and cabling were correct, and prepared our Open Replicator FA to FA masking file. We used the SYMCLI command, symmask list logins, to verify the DMX-3 Control FAs were logged onto the DMX3000 remote FAs. We ran the command once for each of the four remote FA to confirm we have good FA logons across the board. Actually, we ran our command before and after the FA to FA zoning was in place, so we could compare and identify the new logon.
symmask -sid 7667 list logins -dir 7c -p 1 Symmetrix ID : 000187827667

Director Identification : FA-7C Director Port : 1 Identifier ---------------10000000c93ae9a6 10000000c93aea8f 10000000c93aeb71 10000000c93aebde 5006048ad52e6e96 Type ----Fibre Fibre Fibre Fibre Fibre User-generated Node Name Port Name FCID --------------------------------- -----fscl1b 10000000c93ae9a6 690913 Yes fscl1a 10000000c93aea8f 690813 Yes dvcl1b 10000000c93aeb71 690f13 Yes dvcl1a 10000000c93aebde 690e13 Yes NULL NULL 757c13 Logged In -----Yes Yes Yes Yes No On Fabric ------

Yes

Figure 14 Display wwn login information for Symm7667 FA7CB

We then copied the new wwn info from the list logins output and put it into our masking file. Keep in mind, we are just creating a masking file to be used later, after the MSCS Physical Clusters are brought offline. Using the wwn information in Figure 14 and our cabling and zoning information in Figure 13, we can create our masking entries. The new

EMC ProvenTM Professional Knowledge Sharing 2008

18

logged on wwn in Figure 14 is from Symm 6491 FA7DA . We need to assign this wwn to Symm 7667 FA8CB and Symm 6776 FA9CB as follows:
symmask -sid 7667 -wwn 5006048ad52e6e96 add devs 0F79,0F91,0FA9,0FC1,0FD9,0FF1,1159,1163 -dir 7c -p 1

Using symmask login output from the other zoned DMX-3000 FA ports, we added the remaining masking entries to our masking file to be used for OR:
symmask -sid 7667 -wwn 5006048ad52e6e99 add devs 0F79,0F91,0FA9,0FC1,0FD9,0FF1,1159,1163 -dir 10c -p 1 symmask -sid 6776 -wwn 5006048ad52e6e99 add devs 15F7,16F3,170B -dir 8c -p 1 symmask -sid 6776 -wwn 5006048ad52e6e96 add devs 15F7,16F3,170B -dir 9c -p 1

Figure 15 FA to FA Masking entries added to support Open Replicator Session

VMware / MS Cluster Prep-work

We will outline both a fresh-build approach and a physical-to-virtual (P2V) migration using VMware Converter. We tried both methods and found the fresh-build approach to better fit in our environment; but we also saw great value in the P2V migration with Converter which might work best in other environments. The advantage to the freshbuild method is you are starting with no unneeded software, device drivers, or remnants from previous software. With the P2V migration, you need to uninstall any software and drivers left over from the source physical server. However, the shorter downtime with a P2V migration may outweigh the benefits from a cleaner operating system. We were able to select a slightly longer server downtime for the benefits of a cleaner environment.

We must complete same prep work on our ESX servers regardless of which method we choose. We created a separate RAID 1 logical drive on each of our ESX servers to store our virtual machines because the boot disk for Windows clusters must reside on local storage. I will refer to our ESX servers as apesx1 and apesx2. We created new datastores using VirtualCenter and labeled them vmcl1 on apesx1 and vmcl2 on apesx2 using the entire capacity. For the heartbeat network, we used a separate network card on each ESX server and connected them to a private VLAN created on our Cisco network switch. This isolated the heartbeat traffic following Microsoft best practices for clusters.

19

EMC ProvenTM Professional Knowledge Sharing 2008

We then created a new virtual machine port group called cluster-hb on each ESX server including this network card. The speed was set to 10 Mbps, half duplex per Microsoft best practice. Next, we masked all the new devices from the DMX-3 to apesx1 and apesx2. In VirtualCenter, we rescanned our storage adapters for the new LUNs with only the Scan for New Storage Devices option checked.

Figure 16

Fresh-Build MSCS
This fresh-build approach follows Disaster Recovery procedures we created for dissimilar hardware recovery. The benefit is less post-migration cleanup work, and we ran the new environment on a clean operating system build.

To create new guest VMs for Microsoft Clustering, you will need to reference VMware documentation Basic System Administration and Setup for Microsoft Cluster Service documents both available at vmware.com. Following the Clustering Virtual Machines across Physical Hosts procedures outlined in the Setup for Microsoft Cluster Service document, we created a new virtual machine on the datastore vmcl1 with two network adapters, two processors, 2048 MB of RAM, and a 24 GB boot disk. Install 32-bit Windows Server 2003, Enterprise Edition with service pack 2. We named the server fscl1a-new so that we could bring the new virtual server online at the same time as the source server we were migrating. When the build was complete, we used Virtual

Machine Console and installed VMware Tools.

Once complete, we used the clone procedure to create our second node on apesx2, vmcl2 datastore; but we needed to use the Customization Wizard to customize the new virtual machine. We specified fscl1b-new for the computer name and generated a new

EMC ProvenTM Professional Knowledge Sharing 2008

20

SID when prompted. When the clone was complete, we booted the server to complete the build. At this point we had two virtual servers built and we were ready to add shared disk and start the Microsoft Cluster configuration. Adding Shared Storage In this step, we were only concerned with the 4 GB quorum LUN. With both fscl1a-new and fscl1b-new powered off, the 4 GB quorum LUN was then added to each virtual cluster server with Raw Device Mappings. Referring to the Open Replicator Pairing Sheet, the quorum device 138E has a decimal LUN number of 9. Following the

VMware storage device naming convention <HBA>:<SCSI target>:<SCSI LUN>:<disk partition>, this quorum LUN 9 is listed as vmhba1:1:9:0. We stored the LUN mapping with the Virtual Machine and selected physical compatibility mode.

/vmfs/devices/disks/vmhba1:1:9:0 Figure 17

4.215 GB

You must select a new virtual device node other then SCSI 0. We selected SCSI 1 for our shared storage and assigned the quorum LUN to SCSI 1:0. After selecting Finish, we saw our new hard disk mapping and a new SCSI controller. Then we selected the SCSI controller, and verified the SCSI type was set to LSI Logic and the SCSI Bus Sharing was set to Physical.

Figure 18

21

EMC ProvenTM Professional Knowledge Sharing 2008

Then, we repeated this procedure on the second server. Once the quorum disk was added to both nodes, we were able to boot up fscl1a-new. We needed to reboot

because we added new hardware to the server. After the reboot, we now saw the quorum disk when opening Disk Management. We shut down fscl1a-new and powered up fscl1b-new. Then, we verified the existence of the quorum disk after a reboot. Again, we did not initialize the disk. Now we were able to install Microsoft Cluster Service.
NOTE: The Setup for Microsoft Cluster Service document references a Microsoft Cluster Service guide at http://www.microsoft.com/downloads/details.aspx?FamilyID=96f76ed79634-4300-9159-89638f4b4ef7&displaylang=en#EIAA. For our migration, we performed the cluster disaster recovery procedure we use for unlike hardware restores.

Disaster Recovery Cluster Restore Is this step, we installed Microsoft Cluster Service on fscl1a-new and then restored the cluster registry from the source physical cluster. This process restored all of our cluster groups and resources. At this point, you will need two utilities from the Windows 2000 Resource Kit: dumpcfg.exe and regback.exe. Dumpcfg.exe is used to gather the disk signatures from the source server so they can be written on the new virtual servers. Regback.exe is used to backup and restore the cluster registry.

First, we created a C:\Restore folder on the source server running the quorum resource, and copied both utilities to it. From a command prompt, we ran regback.exe

C:\Restore\clustername-clusbak machine cluster. Then we ran dumpcfg.exe > C:\Restore\clustername-dsksig.txt. We copied the restore folder to a network

share so it could be accessed by the new virtual servers. Next, we booted fscl1a-new using Disk Management to initialize the quorum disk. Dynamic Disk. We did not convert them to

We formatted the drive NTFS and assigned the drive letter Q. The

volume label is then always set to match the drive letter; so in this case, we set it to Drive Q.

At this point, we copied the Restore folder created on the source server to fscl1a-new to C:\. From a command prompt, we ran C:\Restore\dumpcfg.exe. This listed all the volumes with associated disk numbers and signature numbers for the new virtual server. In the [Volumes] section, each disk volume will have a Volume Label such as Volume Label: Drive Q.
EMC ProvenTM Professional Knowledge Sharing 2008

22

Example:
[Volumes] Volume #1: Volume name: \\?\Volume{3c8df3b7-eae0-11d8-bb40-505054503030}\ Drive letter: Q:

Volume Label: Drive Q

File System: NTFS Boot\Boot.ini\System Volume: Volume Type: Simple Volume \ Logical Drive Number of members: 1 Member #1: Partition - Disk: 1, StartingOffset: 32256 bytes,

Length: 4314 MB

Volume Label: Drive Q will have an associated Disk number and in this example Disk:1 Member #1: Partition - Disk: 1, StartingOffset: 32256 bytes, Length: 4314 MB

In the [Disks] section each disk will be listed by Disk Number and will list a signature. [DISKS]

Disk Number: 1 Signature: 4D3509BA

Disk Number: 2 Signature: 4D3509BC Disk Number: 3 Signature: EF7729A0 Disk Number: 0 Signature: B326B326

For the example of Volume Label: Drive Q, the signature is 4D3509BA for Disk Number: 1. We opened the C:\Restore\clustername-dsksig.txt file in notepad, and found the Volume Label used for our quorum drive in the [Volumes] section. We always use Drive Q. For Drive Q, it will list the disk number which you can use to find the original drive signature in the [DISKS] section. In our case, the original disk signature was 4D3509BB. This signature must be restored with dumpcfg.exe to the new quorum LUN presented to fscl1a-new. The Command line syntax is C:\Restore\dumpcfg.exe S <original
disk signature from clustername-dsksig.txt > <New Drive Disk Number on fscl1a-new server>. Following our example, we ran dumpcfg.exe -S4D3509BB 1.

Running dumpcfg.exe again listed all the disk signatures to confirm the quorum disk had the new restored signature.

Then we shut down fscl1a-new and booted fscl1b-new. When we opened Disk Management, we saw the quorum disk properly formatted with a Volume Label of Drive Q. The only thing we needed to do was change the drive letter mapping to Q. Next we shut down fscl1b-new. At this point, we were ready to start the migration. Both source servers should be shut down.

23

EMC ProvenTM Professional Knowledge Sharing 2008

Implement Migration
At this point, all Storage and VMware prep-work has been completed, and its time to begin the downtime cutover. Figure 19 illustrates the tasks we completed. The physical MSCS Nodes were taken offline and powered down to prevent access to the OR Remote devices during the copy operation.

Figure 19 Completed Tasks (grey) before we start Open Replicator Copy

With the OR FA to FA zoning already in place, we added the OR FA to FA masking entries to the DMX-3000s. Then, the Open Replicator Session was created and

activated. Figure 20 shows the syntax for creating the OR session. Notice we did not use the copy option. The default mode for the copy session is CopyOnAccess, which copies tracks as they are accessed by the hosts connected to the OR Control Devices. Basically, we wanted to get the MSCS File Servers running in VMware before allowing Open Replicator to copy all tracks. After we activated the OR session we told the VMware administrator that he could begin his work to bring up the MSCS Clusters in VMware using either the VMware Fresh-Build Approach or the VMware Converter Option (Physical-to-Virtual).

EMC ProvenTM Professional Knowledge Sharing 2008

24

Figure 20 Open Replicator Session Creation and Activation

Figure 20 illustrates how we started our hot pull operation from remote devices on two different Symmetrix DMX-3000s. We replicated 11 devices; this figure shows 2.

Once the MSCS File Servers and their resources were brought back online (running on VMware), we started the file copy and set the copy pace. We changed the Open

Replicator session mode to CopyInProg with the following syntax :

symrcopy -file or_device_pairs set mode -copy

Then we set the copy session pace:

symrcopy file or_device_pairs set pace 3

The default pace is 5. Pace goes from 0-9 with 0 being fastest. We did not want to affect our backup windows, so we decided to copy our data during business hours at a medium pace. Our users were not affected. While the tracks are

copying you can query the progress of the session per device as shown below.

25

EMC ProvenTM Professional Knowledge Sharing 2008

Symrcopy file or_device_pairs query i 30 c 10 Legend: R: (Remote Device Vendor Identification) S = Symmetrix, C = Clariion, . = Unknown. I: (Remote Device Specification Identifier) D = Device Name, W = LUN WWN, World Wide Name. Flags: (C): X . (D): X . (S): X . (H): X . (U): X . = = = = = = = = = = The The The The The The The The The The background copy setting is active for this pair. background copy setting is not active for this pair. session is a differential copy session. session is not a differential copy session. session is pushing data to the remote device(s). session is pulling data from the remote device(s). session is a hot copy session. session is a cold copy session. session has donor update enabled. session does not have donor update enabled. : or_control_cl1 Remote Device Flags Status Done ----------------------------------- ----- -------------- ---Identification -------------------------------000187827667:0F79 000187827667:0F91 000187827667:0FA9 000187827667:0FC1 000187827667:0FD9 000187827667:0FF1 000187827667:1159 000187827667:1163 000187886776:15F7 000187886776:16F3 000187886776:170B RI -SD SD SD SD SD SD SD SD SD SD SD CDSHU ----X..X. X..X. X..X. X..X. X..X. X..X. X..X. X..X. X..X. X..X. X..X. CTL <=> REM (%) -------------- ---CopyInProg 93 CopyInProg 94 CopyInProg 94 CopyInProg 94 CopyInProg 93 CopyInProg 94 CopyInProg 99 Copied 100 Copied 100 Copied 100 Copied 100

Device File Name

Control Device ---------------------------Protected SID:symdev Tracks ------------------ --------000190106491:12C1 434007 000190106491:12CD 376476 000190106491:0EDE 581598 000190106491:12D9 359717 000190106491:12E5 401421 000190106491:12F1 358848 000190106491:0FBA 13518 000190106491:0FBE 0 000190106491:0EF0 0 000190106491:12FD 0 000190106491:1309 0 Total Track(s) MB(s) --------2525585 157849

We terminated the Open Replicator session once all the tracks status indicate Copied:
Symrcopy file or_device_pairs terminate Execute 'Terminate' operation for the 11 specified devices in device file 'or_control_cl1' (y/[n]) ? y 'Terminate' operation execution is in progress for the device list in device file 'or_control_cl1'. Please wait... 'Terminate' operation successfully executed for the device list in device file 'or_control_cl1'.

Create Microsoft Cluster

Continuing with the fresh-build MSCS approach, we booted fscl1a-new, changed the name to fscl1a, and changed the network address of the production port group to match the source server IP. Then we shut down fscl1a and repeated for fscl1b-new by

renaming the server to fscl1b and changing the network address of the production port group to match the source server IP.

Once completed, we shut down fscl1b. We were now ready to create a new cluster on fscl1a using Microsoft standard practices with the cluster-hb network as the private network for internal cluster communications.

EMC ProvenTM Professional Knowledge Sharing 2008

26

Once completed, we stopped the Cluster Service and set it to manual startup. In the Start parameters field for the Cluster Service, we entered NoQuorumLogging, and clicked on the Start button in the properties window of the Cluster service. Then we navigated to Q: and renamed quolog.log under Q:\MSCS to quolog.log.old. We stopped the Cluster Service. Open Regedit; highlight HKLM\Cluster; and select File Unload Hive (On the menu bar), and click Yes at the prompt. At this point, we copied the cluster clustername-clusbak file from C:\Restore to C:\Windows\Cluster. clusdb to clusdb.old and cluster.log to cluster.old. clusbak to clusdb. Then we renamed

Then we renamed clustername-

Next, we started the Cluster Service with the resetquorumlog parameter.

Then we

opened Cluster Administrator, and clicked Yes to all. The following resources failed, as we expected: Virtual server network names (if Kerberos Authentication is enabled), third party server applications, File Shares, and disk resources other than Q. All third-party server applications needed to be reinstalled once the migration is completed; so we deleted any resources or groups associated with them.

In our case, we had a cluster-aware backup software so we needed to delete its Network Name, IP Address and Generic Service resources. If your Network Name resources have Kerberos Authentication enabled, you will need to delete them in Active Directory in order to start them. In Cluster Administrator, double click a failed network name resource and select the Parameters tab. Uncheck Enable Kerberos Authentication, click Apply, then choose Yes. In Active Directory Users and Computers, locate the Network Name and delete it. Back in Cluster Administrator, check Enable Kerberos

Authentication and click OK. Right click the Network Name resource and select Bring Online. Repeat for all failed Network Name resources. You can now stop the Cluster Service and remove the resetquorumlog parameter. manual start, and shut down fscl1a. Keep the Cluster Service on

The remaining new devices were added to fscl1a and fscl1b.

Following the same

procedure, we added the new devices to each server using Raw Device Mapping. The quorum LUN was assigned to SCSI 1:0; so each additional device was assigned to SCSI 1:1 and higher. We booted fscl1a, and like the addition of the quorum disk, we needed to reboot at the new hardware detected prompt. After the reboot, we opened Disk Management and verified all the newly added devices were visible. We changed all

drive letter mappings to match the Volume Labels, as we did with drive Q.
27
EMC ProvenTM Professional Knowledge Sharing 2008

This can be challenging for environments with a large number of devices.

In our

environment, we often need to assign a Drive Letter to an unused letter temporarily to free up the correct drive letter required. In this case, you will need to reboot to free up the letter. Once complete, we shut down fscl1a and booted fscl1b. Then we repeated the same Drive Letter mapping procedure on this server, and shutdown fscl1b when complete.

Configuration of the Second Node

It was now time to add the second node into the cluster. We booted fscl1a, and Then we opened

changed the Cluster Service to start automatically and started it.

Cluster Administrator, so that all of our resources would be online. In the left column, we right-clicked fscl1B, selected Evict Node, and then selected OK to the RPC error message. Then we booted fscl1b. Once online, you added this server to the cluster following the Microsoft standard procedure to add an additional node We tested the failover of a few resource groups just to verify the configuration and then installed our backup software. The migration was complete. All cluster resources remained online

during the Open Replicator copy process.

P2V Migration with VMware Converter

Using VMware Converter to take a snap shot of our source servers, we imported them on our ESX servers. The benefit of VMware Converter is less pre-work and a slightly shorter downtime when compared to the fresh -build approach

The VMware Converter application can take a snapshot of a running server; but for our cluster migration, we found it easier to take the source server completely offline. Before taking down the source servers, we set the Cluster Service to manual startup. We also found it beneficial to disable any vendor-specific software on the source servers prior to shutdown. Physical-to-Virtual Migration For our P2V migration, we followed the procedures outlined in the VMware Converter 3.0.2 User's Manual under Using the Converter Boot CD for Local Cold Cloning. We cloned one cluster node at a time to minimize downtime. We started by moving all cluster groups to fscl1b. After completing the P2V migration of fscl1a, we booted fscl1a

EMC ProvenTM Professional Knowledge Sharing 2008

28

and moved all resources back and started the migration of fscl1b. Fscl1a was migrated to apesx1 on the vmcl1 datastore and fscl1b went to apesx2 on the vmcl2 datastore. Adding Shared Storage When the P2V migration was complete, we shutdown the source servers and started the same Open Replication copy process used for the fresh-build MSCS approach. After the OR session was activated, we used VirtualCenter to map all the new devices with Raw Device Mappings to the newly created Virtual Machines fscl1a and fscl1b. We stored the LUN mapping with the Virtual Machine and selected physical compatibility mode. Just as we did with the fresh-build approach, we selected a new virtual device node other than SCSI 0. We selected SCSI 1 for our shared storage and assigned the quorum LUN to SCSI 1:0 and the remaining LUNs to 1:1 and higher. After selecting Finish, we saw the new hard disk mappings and a new SCSI controller. Then we

selected the SCSI controller and verified the SCSI type was set to LSI Logic and the SCSI Bus Sharing was set to Physical. Virtual Machine Configuration On the Virtual Machine properties page, we removed the USB and Serial devices captured by the converter process. We then set the RAM to 2048 MB, CPU count to two and set one network adapter on our production network and one on our cluster-hb network. Next boot fscl1a, and install VMware Tools. You will be prompted to reboot after new hardware is detected during the first boot-up after the migration. Also, some services that are tied to HP hardware will not start but can be ignored at this point.

After rebooting, we need to set the network configuration for both network adapters to match the source server. At this point, we can set Cluster Service to automatic Startup type and start it. Open Cluster Administrator and verify all resources are online. Next, boot fscl1b and install VMware Tools. After rebooting, we needed to set the network configuration for both network adapters to match the source server. Set the Cluster Service to automatic Startup type, and start it. We test failover of groups to verify

operation. At this point, the migration is complete, and all file shares are online during the Open Replicator.

29

EMC ProvenTM Professional Knowledge Sharing 2008

Physical-to-Virtual Cleanup Tasks We removed all software and left over devices from the physical hardware after we completed the Physical to Virtual conversion. We did this one server at a time so all shares remained online during the cleanup. We started by removing applications that were tied to our source physical hardware in Add or Remove Programs. Next we

removed old hardware devices by running the following command in a command prompt: set devmgr_show_nonpresent_devices=1, then Start DEVMGMT.MSC. Click View, and then click Show Hidden Devices on the menu bar. Right-click each dimmed devices, and click Uninstall. Once complete, we failed over all groups and repeated the cleanup procedure on the other node. This process took approximately two hours per server.

Open Replicator Post-Migration Tasks

Once the Open Replicator Session is terminated, it was time to clean up zoning and masking entries added to support the remote copy operation. After we were

comfortable running on the new Virtualized MSCS File Servers, we removed the physical MSCS File Servers from the Data Center and re-claimed our switch ports and SAN cabling.

Conclusion
In reality, it took a long time to perform the initial boot of virtual nodes after Raw Device Mapping the Open Replicator control devices. We noticed the single CPU in the guest VM was pinned at 100%. Our solution was to add a total of two CPUs to each VM, which was the target number of CPUs in our new VM guest configuration. additional CPU made the boot go much more quickly. The

We achieved an average sustained throughput of ~ 290 MB / second during our OR CopyInProg hot pull session. Our migration strategy required that we present the same number of LUNS to a new host configuration, which made Open Replicator the ideal vehicle to copy our file server data volumes. Since Open Replicator allowed us to

access (read/write) the control devices during the copy session, we virtually eliminated downtime compared to alternative copy methods. The MSCS File Server outage was about 1 hour from the time the physical clusters were brought offline to the point when users were able to access file share resources on VMware.

EMC ProvenTM Professional Knowledge Sharing 2008

30