srdfFailoverandFailback_sym001

Performing SRDF Failover and Failback

This procedure describes how to perform basic SRDF failover and failback operations between
the source (R1) devices and the target (R2) devices located at separate Symmetrix sites (A and
B).

Done

In a period of scheduled downtime for maintenance, or after a serious system problem which
has rendered either the host or Symmetrix array containing the source (R1) devices
unreachable, no read/write operations can occur on the source (R1) device. In this situation, a
failover operation should be initiated to make the target (R2) devices read/write enabled to their
local host(s) at site B.
After resolution of any system problems or completion of scheduled maintenance tasks, a
failback operation is performed to resume normal SRDF operations by initiating read/write
operations on the source (R1) devices, and stopping read/write operations on the target (R2)
devices. The target (R2) devices become read-only to their local host(s) while the source (R1)
devices are read/write enabled back to their local host(s).
These instructions use the Solutions Enabler SYMCLI command line interface to manage
devices by a device group. The user should be familiar with the concepts of SRDF, and have
some experience using Solutions Enabler SYMCLI commands before attempting this procedure.
This procedure is based on contents from the following EMC manuals:

EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide

EMC Solutions Enabler Symmetrix Array Management CLI Product Guide

These documents can be obtained from the EMC Powerlink website at:
http://Powerlink.EMC.com

1.

Failover device operations to the R2

The failover control operation can be performed by device group, composite group, or device
file:
symrdfgDgNamefailover
symrdfcgCgNamefailover
symrdff[ile]FileNamefailover

For example, to perform a failover on all the RDF pairs in the prod device group, enter:
symrdfgprodfailover

To perform a failover on one RDF pair with device DEV001 in the prod device group, enter:
symrdfgprodfailoverDEV001

To perform a failover on a list of RDF pairs in the device group prod, enter:
symrdfgprodfailoverDEV001DEV002

Note: To invoke a failover, the RDF pair(s) must already be in one of the following states:
Synchronized
Suspended
R1 Updated
srdfFailoverandFailback_sym001

 Partitioned (when you are invoking this operation from the target side)
Transmit Idle (with the immediate option if this operation is on the target side)
Note: This operation will be rejected if any of the device pairs are in the following states
without specifying the force option:

Split
SyncInProg
R1 UpdInProg
Invalid

When a failover is performed for each specified RDF pair in a device group:
If the source (R1) device is operational, the SRDF links are suspended.
If the source side is operational, the source (R1) device is Write Disabled to its local
host(s).
The target (R2) device is Read/Write Enabled to its local host(s).
Note: This operation will be rejected if any of the following occur:
The source has invalid remote (R2) tracks without specifying the symforce option.
The target has invalid local (R2) tracks without specifying the symforce option.
Consistency is enabled and the force option is not specified.
2.

Perform a query operation on the device group

The following query from the local host shows that the R1 devices are now write disabled (WD),
the RDF links have been suspended, and the R2 devices are read/write enabled (RW).
symrdfgprodquery

DeviceGroup(DG)Name:prod
DG'sType:RDF1
DG'sSymmetrixID:000000003264
Source(R1)ViewTarget(R2)ViewMODES

STLIST
StandardANA
LogicalTR1InvR2InvKTR1InvR2InvRDFPair
DeviceDevETracksTracksSDevETracksTracksMDASTATE

DEV001009CWD
00NR0054RW
00S..FailedOver
DEV002009DWD
00NR0055RW
00S..FailedOver
Total
Track(s)0000
MB(s)0.00.00.00.0
LegendforMODES:
M(odeofOperation):A=Async,S=Sync,E=Semisync,C=AdaptiveCopy
D(omino):X=Enabled,.=Disabled
A(daptiveCopy):D=DiskMode,W=WPMode,.=ACpoff

3.

Perform update of the R1 devices

Before issuing the failback command, an update of the R1 devices should be performed. The
srdfFailoverandFailback_sym001

function of the update operation is to minimize downtime when issuing a failback command,
which write disables the R2. While the R2 side remains accessible for reads and writes, the
symrdf update command from the local host takes a one-time snapshot of the remote (R1)
invalid tracks on the target (R2) side for each device in the group (16385 in each case) and
copies those tracks to the R1 side.
symrdfgprodnopromptupdate
AnRDF'UpdateR1'operationexecutionisinprogressfordevicegroup
'Rdf1Grp'.
Pleasewait...
SuspendRDFlink(s).......................................Done.
Mergedevicetracktablesbetweensourceandtarget.......Started.
Device:009C............................................Merged.
Device:009D............................................Merged.
Mergedevicetracktablesbetweensourceandtarget.......Done.
ResumeRDFlink(s)........................................Done.
TheRDF'UpdateR1'operationsuccessfullyinitiatedfordevicegroup'prod'.

4.

Query the device group to check the progress

The symrdf query command from the remote host shows that as an update session begins, the
local Symmetrix invalidates tracks (16385) on the source (R1) that need updating.
symrdfgprodquery
DeviceGroup(DG)Name:prod
DG'sType:RDF2
DG'sSymmetrixID:000000003265
Source(R2)ViewTarget(R1)ViewMODES

STLIST
StandardANA
LogicalTR1InvR2InvKTR1InvR2InvRDFPair
DeviceDevETracksTracksSDevETracksTracksMDASTATE

DEV0010054RW 163850RW009CWD 163850S..R1

UpdInProg
DEV0020055RW 163850RW009DWD 163850S..R1
UpdInProg
Total
Track(s)327700327700
MB(s)1024.00.01024.00.0
LegendforMODES:
M(odeofOperation):A=Async,S=Sync,E=Semisync,C=AdaptiveCopy
D(omino):X=Enabled,.=Disabled
A(daptiveCopy):D=DiskMode,W=WPMode,.=ACpoff

Note: As the update progresses, the number of local (R1) invalid tracks as viewed on the
source (R1) side keep decreasing because the tracks are being counted down from the
original snapshot taken at the beginning of the update process. Meanwhile, the remote (R1)
invalid tracks on the target (R2) side continue to be incremented as new writes are executed
there.

srdfFailoverandFailback_sym001

5.

Failback device operations to the R1

A failback, or source (R1) device takeover, is performed when you are ready to resume normal
SRDF operations by initiating read/write operations on the source (R1) devices, and stopping
read/write operations on the target (R2) devices. The target (R2) devices become read-only to
their local host(s) while the source (R1) devices are read/write enabled to their local host(s).
Note: All I/O should be terminated on the R2 side prior to issuing the failback command.
The devices should also be unmounted. Once the failback command is issued, the R1
devices can then be mounted and applications started.
Note: When the symrdf command is initiated, device external locks are set on all RDF
devices you are about to failback. Device external locks are then automatically released
when the control operation completes.
The failback control operation can be performed by device group, composite group,
or device file:
symrdfgDgNamefailback
symrdfcgCgNamefailback
symrdff[ile]FileNamefailback
SRDF Control Operations

For example, to initiate a failback on all the RDF pairs in the prod device group, enter:
symrdfgprodfailback

To initiate a failback on one RDF pair, DEV001, in the prod device group, enter:
symrdfgprodfailbackDEV001

To initiate a failback on a list of RDF pairs in the device group prod, enter:
symrdfgprodfailbackDEV001DEV002

Note: To invoke a failback, the RDF pair(s) must already be in one of the following states:

Failed Over
Suspended and Write Disabled at the source
Suspended and Not Ready at the source
R1 Updated
R1 UpdInProg

Note: The R2 may be set to Read/Write Disabled (Not Ready) if

SYMAPI_RDF_RW_DISABLE_R2=ENABLE is set in the options file.
Note: This operation will be rejected if any of the device pairs are in the Partitioned state
unless you invoke this operation from the source side and specify the force option.
When a failback is initiated for each specified RDF pair in a device group, the following occurs:
The target (R2) device is Write Disabled to its local host(s).
Traffic is suspended on the SRDF links.
If the target side is operational, and there are invalid remote (R2) tracks on the source
side (and the force option is specified), the invalid R1 source tracks are marked to refresh
from the target side.
srdfFailoverandFailback_sym001

The invalid tracks on the source (R1) side are refreshed from the target R2 side.
The track tables are merged between the R1 and R2 sides.
Traffic is resumed on the SRDF links.
The source (R1) device is Read/Write Enabled to its local host(s).

Note: This operation will be rejected if any of the following occur:

The source has invalid local (R1) tracks and the state is Partitioned.
The source has invalid remote (R2) tracks and if the target side is not reachable. (If the
target is reachable, use the force option to mark the changed tracks on the source side
to refresh from the target.)
The target side is reachable and the target has invalid local (R2) tracks.
Table of Contents

srdfFailoverandFailback_sym001