0 d9cdf

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.
Symmetrix Monitoring and Management Using Solutions Enabler
Course Overview
2007 EMC Corporation. All rights reserved.
Welcome to the first module in the Symmetrix Monitoring and Management course. In this module we will review the hardware and software architecture of the Symmetrix and the process for installing and configuring Solutions Enabler on a open systems host.
Symmetrix HW & SW Architecture - 1
Revision History
Rev Number
1.0
Course Date
September, 2007
Revisions
Complete
Copyright 2007 EMC Corporation. All rights reserved. These materials may not be copied without EMC's written consent. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
Course Objectives
y Provide a software view of the Symmetrix Architecture y Install and configure Solutions Enabler and Symmetrix Management Console y Using the appropriate commands query the Symmetrix to determine the physical and logical configuration y Modify the configuration to change system parameters, configure new devices, set port attributes, and map devices to front-end directors y Understand host to Symmetrix connectivity y Secure access to the Symmetrix using access controls y Gather real time performance statistics on the Symmetrix
2007 EMC Corporation. All rights reserved. Symmetrix HW & SW Architecture - 3
The objective of this course is to provide an understanding of the Symmetrix hardware and software architecture for a host prospective using Solutions Enabler.
Symmetrix Engineering Curriculum

Service Processor
Symmetrix SymmetrixInternals Internals
(4 (4days) days)
Solutions Enabler
Symmetrix Symmetrix Monitoring Monitoring& &Management Management
(2 (2days) days)
Advanced AdvancedSymmetrix Symmetrix Internals Internals

(2 (2days) days)
Configuration Configuration& &Management Management Symmetrix Replication Symmetrix ReplicationTechnology Technology (2 days) (2 days)
TimeFinder TimeFinder& &SRDF SRDF Workshop Workshop

(3 (3days) days)
For EMC Internal Engineering there are two Symmetrix training Paths. On the left is the Symmetrix Internals class that discuss the architecture, configuration, and monitoring of the Symmetrix from the hardware and Service Processor. Included is the use of SymWin to create an initial bin file and basic inlines commands. It also includes a brief introduction to Solutions Enabler. Advanced Symmetrix Internals is a follow-on course that traces an IO operation from the host through the Symmetrix for debugging and performance purposes. The TimeFinder & SRDF Workshop discusses the setup, monitoring, and control of the Symmetrix local and remote replication products using SymWin and Inlines commands on the Service Processor and Solutions Enabler. On the right is the new curriculum path and was designed for an internal audience who needs an understanding of the Symmetrix hardware and software architecture from a software prospective.
Day 1
y Hardware and Software Architecture review
Lab: Installing Solutions Enabler
y Understanding Symmetrix Configuration

Lab: Understanding Physical and Logical Configuration
y Configuring Symmetrix Devices

Lab: Configuring Symmetrix Devices
y Symmetrix Management Console

Lab: SMC Installation and Operations
Day 2
y Host Connectivity
Lab: Host Connectivity
y Monitoring Symmetrix Activity

Lab: Monitoring Symmetrix Performance Statistics
y Symmetrix Security
Lab: Symmetrix Security
Closing Slide
Module 1: Symmetrix Hardware and Software Architecture
Welcome to the first module in the Symmetrix Monitoring and Management course. In this module we will review the hardware and software architecture of the Symmetrix, and the process for installing and configuring Solutions Enabler on a open systems host.
Revision History
Rev Number
1.0
Course Date
July, 2007
Revisions
Complete
Course Objectives
Upon completion of this course, you will be able to: y Describe the hardware and software architecture of a Symmetrix DMX y List the Symmetrix configuration information, where it is located, and how it is maintained y Describe the communications path from a host to the Symmetrix for monitoring and management y List the Solutions Enabler daemons and briefly describe the functions they perform y Describe the SYMAPI database, where it resides, how it is created, and how it is accessed y Describe the Solutions Enabler software environment including directory structure, environment variables, options, and log files y Install Solutions Enabler y License Solutions Enabler features
We will start the module with a quick overview of the hardware and software architecture of the Symmetrix but the primary focus is on Solutions Enabler, the Command Line Interface (CLI) for monitoring and managing the Symmetrix.
Symmetrix DMX-3/4
y Front-end
Host Connectivity
Fibre Channel FICON/ESCON iSCSI
y Global Cache
Memory
y Back-end
Disk Directors DAE-LCC Disk Drives
All members of the Symmetrix family share the same fundamental architecture. The modular hardware framework allows rapid integration of new storage technology, while supporting existing configurations. There are three functional areas: y Shared Global Memory - provides cache memory y Front-end - the Symmetrix connects to the hosts systems using Channel Adapter a.k.a Channel Directors. Each director includes multiple independent processors on the same circuit board, and an interface-specific adapter board. Celerra Data Movers connect to the storage through the front-end. y Back-end is how the Symmetrix controls and manages its physical disk drives, referred to as Disk Adapters or Disk Directors. Like front-end directors, each director includes multiple independent processors on the same circuit board. What differentiates the different generations and models is the number, type, and speed of the various processors, and the technology used to interconnect the front-end and back-end with cache. With the current generation, the interconnections between the front-end, back-end, and global memory is based on a Direct Matrix Architecture Direct connections between every front-end/back-end director and every memory board.
Hardware Architecture
Fibre Channel host attach ESCON host attach FICON, Gigabit Ethernet, iSCSI
FC Controller Cntl Direct Direct Matrix Matrix FC Controller Cntl Direct Direct Matrix Matrix
ESCON Controller ESCON Controller
Cntl Direct Direct Matrix Matrix
Cntl Direct Direct Matrix Matrix
Multi-protocol CD Cntl Direct Direct Matrix Matrix
Control and communications signals

Environmental control and status signals
64 GB memory 64 GB memory 64 GB memory 64 GB memory 64 GB memory 64 GB memory 64 GB memory 64 GB memory Environmental control and status signals
Control and communications signals

Direct Direct Matrix Matrix Cntl FC (back-end)
A A B B A A B B

A A B B A A B B
Direct Direct Matrix Matrix Cntl FC (BE or FE)

A A B B A A B B

A A B B A A B B

A A B B A A B B

A A B B A A B B

A A B B A A B B

A A B B A A B B
Fibre Channel back-end directors

Service Processor
Modem
Batteries
Cooling
The advantage of Direct Memory Architecture can be appreciated when you visualize it as in the picture above. The Global Memory technology supports multiple regions and 16 connections on each global memory director. In a fully configured Symmetrix system, each of the sixteen directors connects to one of the sixteen memory ports on each of the eight global memory directors. These 128 individual point-to-point connections facilitate up to 128 concurrent global memory operations in the system.
System Bay
y Total of 16 independent front-end and backend directors
Eight PowerPC processors per director Up to 8 disk directors
Installed in pairs Up to 480 drives per disk-director pair
Up to 12 channel directors
Eight-port Fibre Channel 2Gb/4Gb Eight-port ESCON Four-port multi-protocol - FICON, iSCSI, and Gigabit Ethernet (for SRDF)
y Up to 512 GB Global Memory

Mirrored DDR technology Memory Vault protection
2007 EMC Corporation. All rights reserved. Symmetrix HW & SW Architecture - 6 6
The Symmetrix System Bay has: y Either two, four, six. or eight disk directors y Up to 12 channel directors (combined total of all directors is 16) y Up to eight power supplies, each of which has: A dedicated 2.2-kilowatt standby power supply (SPS) A 1 Service Processor with KVM (keyboard, video screen, and mouse) and dedicated uninterruptible power supply (UPS) Three cooling-fan assemblies, each containing three fans y Two power zones with independent power cables, each zone capable of powering the fully configured System Bay Symmetrix DMX-3/4 systems can be configured with up to 512 GB of total memory (256 GB useable). These models support extended Global Memory capabilities, using industry-standard DDR RAM technology DDR is Double data-rate Random Access Memory that achieves greater bandwidth by transferring data on the rising and falling edges of the clock signal. Effectively, it nearly doubles the transfer rate without increasing the frequency of the bus. Thus, a system with a 100 MHz front side bus has an effective clock rate of 200 MHz when DDR SDRAM memory is installed. The same system using SDR (single data rate) SDRAM, will not have its front side bus rate doubled and be limited to a 100 MHz front side bus speed.
Storage Bay
y Storage bays support up to 240 disk drives
Two groups of 120 drives Each group connects to a separate disk-director pair Groups can be daisy-chained linearly across cabinets Fully configured system supports up to 2,400 disks
120 drives
y 4Gb dual-ported Fibre Channel drives

2Gb with DMX-3
y Tiered storage with the Symmetrix DMX

High performance
73 GB and 146 GB 15,000 rpm Fibre Channel
Good price/performance
146 GB and 300 GB 10,000 rpm Fibre Channel
120 drives
Low cost
500 GB 7,200 rpm Fibre Channel 750GB SATA II (DMX-4)
y Online upgradeability and scalability

Add storage bays, directors, and disk drives
2007 EMC Corporation. All rights reserved. Symmetrix HW & SW Architecture - 7 7
Symmetrix DMX-3/4 series configurations allow scalable capacity and performance to consolidate systems. Base configurations are composed of a System Bay and one or more independent storage bays. The Storage Bay is configured for capacities of 120 or 240 disk drives. Each drive bay: y Has redundant power supplies with battery backups to provide standby power to all components y Has two power zones with independent power cableseach zone capable of powering the fully configured drive bay y Can be populated with any combination of 73, 146, and 300 GB 10,000 rpm drives; and 73 and 146 GB 15,000 rpm drives y 500 GB low-cost Fibre Channel drives y With DMX-4, 750GB SATA II drives for low cost archival requirements Additional capacity (upgrade) can be added online by adding drives, Disk Array Enclosures (DAEs), and/or additional drive bays. Disk directors can also be added non-disruptively. One of the distinctions between the between the DMX-3 and DMX-4 is that DMX-4 uses 4Gb Disk Array Enclosures that uses a cut-through-switching technology creating point-to-point connections to each driver. Earlier generations used arbitrated look technology.
Symmetrix DMX-3/4 950

y Entry-level Symmetrix DMX-3
128 GB Memory Two or four front-end/back-end Fibre Channel directors Native Gigabit Ethernet for replication and iSCSI FICON with DMX-4
y Flexible expandability
Nondisruptive upgrades Scale capacity and performance Up to 360 drives with one add-on storage bay
Bay 1
120 Drives
Bay 2 (Optional)
240 Drives
The Symmetrix DMX-3 950 provides an entry point into high-end availability and functionality for open-systems environments. It contains a six-slot backplane, supporting two memory directors with up to 64 GB of usable cache (128 GB raw) and two or four front-end/back-end directors. It can be configured with system components and up to 120 drives in the single bay60 drives above and 60 drives below the systems directors, which are housed in the middle of the cabinet. When needed, you can add one storage bay to the main bay, increasing the drive count by 240 for a total of 360 drives180 TBin a fully loaded Symmetrix DMX-3 950. The Symmetrix DMX-3 950 leverages Symmetrix DMX-3 technology and uses the latest release of Enginuity Operating Environment, with full support for nondisruptive upgradesfrom disks and storage bays to Fibre Channel directors and memory directors. This means the Symmetrix DMX-3 950 fully supports all of EMCs leading local and remote array-based replication software like the TimeFinder and SRDF families and Open Replicator for Symmetrix. And because the Symmetrix DMX-3 950 contains a six-slot backplane, it is optimized for the most efficient power and cooling in a compact footprint, providing significant operational savings. It is therefore optimized for data centers with limited space, power, and cooling. It is also ideally suited for: y Enterprise consolidation and tiering in a compact footprint y Dedicated array for a large application y Extended-distance replication, including full support for the SRDF family y High-performance, high-capacity backup to disk Symmetrix HW & SW Architecture - 8
Scalable and Expandable Configuration

DMX 950
EXPANSION
DMX-3/4: High-End Storage Array

INCREMENTAL SCALABILITY
Disk adapters Drive channels Number of disks Memory directors Maximum Global Memory Host connectivity
1 (DA) pair 816 32360 2 64 GB
1 DA pair 16 96240 28 72 GB
2 DA pairs 32 192960 28 144 GB
3 DA pairs 48 3601,440 48 216 GB
4 DA pairs 64 4802,400 48 256 GB
16 x Fibre Channel
(FC) 8 x Gigabit Ethernet remote replication 10 x Gigabit Ethernet iSCSI
48 x FC 24 x FICON 48 x ESCON 8 x Gigabit

Ethernet remote replication 40 x Gigabit Ethernet iSCSI

Ethernet remote replication 32 x Gigabit Ethernet iSCSI
Ethernet Ethernet remote remote replication replication 48 x Gigabit 24 x Gigabit Ethernet iSCSI Ethernet iSCSI
The Symmetrix DMX-3 950 is an ideal entry point for high-end configurations requiring one DA pair and 32 to 360 drives. The Symmetrix DMX-3 models are ideal for high-end configurations requiring performance and scaling capability to start as small as one DA pair and 96 drives, and grow to a maximum of four DA pairs and 2,400 disks. The incremental scalability that the Symmetrix DMX-3 provides allows you to meet your growth requirements by adding Disk Adapters, disk channels, and disk drives nondisruptively to the existing frame. This enables true pay-as-you-grow economics for high-growth storage environments.
Software Architecture
y Symmetrix architecture can be compared to MPP Server
Multiple independent processors
Front-end directors service host requests Back-end directors stage and destage data in and out of cache Intelligent disk drives process SCSI commands Intelligent Environmental and Communications Modules Service Processor
All working together to service host request and protect data Parallelism provides nearly infinite scalable performance
y Enginuity Operating Environment is the micro code that runs each processor in the Symmetrix
Allows the independent processors on each director to work together
The Symmetrix architecture is often compared to Symmetrical Multiprocessor (SMP) architecture used by server manufacturers, but it might be more correct to compare it to the Massively Parallel Processor (MPP) used for computational intensive applications where multiple independent processors are working in parallel to solve a problem. A well-known example of this is Deep-Blue the chess champion of a few years back. In the Symmetrix we have multiple independent front-end and backend processors working together to service host requests and protect data. The Direct Matrix Architecture and parallelism, with multiple independent processors provides nearly infinite scalable performance. Enginuity, a.k.a firmware or microcode, is the code that is loaded in the processors on the directors that allows the independent processors to work together. Components of Enginuity are also loaded in the XCM boards and in the Service Processor. Provides all system functionality including advanced features like local and remote replication Emulation code is loaded into each processor on each director y Downloaded from service processor to directors during code load y Zipped code persistently stored in EEPROM y Loaded to SDRAM (control store) on each processor on IMPL
Symmetrix Configuration
y Initial configuration is created using SymmWin to build an IMPL.bin file
a.k.a the bin file, contains all the configuration information for a Symmetrix
Physical hardware configuration
Directors Memory Physical Drives
Logical storage configuration

Emulation, number, size and data protection schemes for logical volumes Special volume attributes Volume front-end assignments Director flags
Operational parameters and features
y After initial configuration subsequent changes are made on-line

EMC can update the IMPL.bin Users can make changes on-line
EMC Control Center Configuration Manager Solutions Enabler CLI Symmetrix Management Console (SMC)
The Symmetrix is configured using a static configuration file called the IMPL.bin. The file is created initially using SymmWin and loaded into each director in the Symmetrix. When modifying a configuration, the current IMPL.bin file is pulled from the Symmetrix and edited use Symmwin. SymmWin is an EMC written graphical-based application for managing a Symmetrix. Capabilities include: y Building and modifying system configuration files (IMPL.bin) y Issuing Inlines commands, diagnostic, and utility scripts y Monitoring performance statistics y Automatically performs periodic error polling for errors and events. Certain errors will cause the service processor to Call Home. SymmWin runs locally on a Symmetrix Service Processor or on a standalone PC. Running on the service processor allows communications with an operational Symmetrix. Running it on a standalone system allows you to build a new configuration or view and modify an archived configuration file.
System Information Maintained by Enginuity

y Symmetrix File System (SFS)
Two SFS volumes are automatically defined when the bin file is created
6-Gigabyte FBA volumes Not host addressable
Contains:
Error codes are copied to the SFS from NVD for long-term storage Trace information Historical access information used by Dynamic Mirror Service Policy Optimizer data is used to plot a thermograph of hot and cold spindle Device masking database (VCMDB) Global Name Services (GNS) device group database Access Control (ACL) Audit information
Enginuity automatically reserves two volumes for internal use as a Symmetrix File System (SFS). This space is automatically allocated while initially loading the Enginuity Operating Environment on Symmetrix systems and is not visible to the host environment. The SFS is contained on two mirrored 6GB volumes. The SFS stores statistical data to provide a number of benefits: y Dynamically adjusting performance algorithms including dynamic mirror service policy y Load balancing and optimized performance y Problem identification and recovery y Enhanced system audit and investigation y Security and Access Control Enginuity also allows Quality of Service (QoS), giving the ability to set varying priority levels to applications residing within a Symmetrix to meet varying customer needs or agreements.
Host View of the Symmetrix

y The Symmetrix emulates disk drives
Opens Systems Host simply see the Symmetrix as one or more FBA SCSI disk drives Mainframe systems see the Symmetrix as a Logical Control Unit and one or more CKD disk drives
y Other than basic query data, the host has no knowledge of the Symmetrix internal configuration y EMC provided integration tools provide visibility and control
From a hosts perspective, the Symmetrix is simply seen as one or more FBA or CKD devices. Standard SCSI commands such as SCSI INQUIRY and SCSI READ CAPACITY return lowlevel physical device data, such as vendor, configuration, and basic configuration, but have very limited knowledge of the configuration details of the storage system. Vendor specific information such as director configuration, cache size, number of devices, mapping of physicalto-logical, port status, flags, etc. are accessible using these commands.
EMC Solutions Enabler Introduction

y Symmetrix Command Line Interface (SYMCLI) y Provides a host with a comprehensive command set for managing a Symmetrix storage environment
Invoked from the host OS command line Scripts that may provide further integration with OS and application
9 Detailed Configuration Information 9 Status 9 On-line Configuration Changes 9 Performance 9 Control
y Separate components licenses y Security and access controls

Monitor only Host-based and user-based controls
SYMCLI can be used to perform ad-hoc operations or incorporated into user developed scripts to integrate Symmetrix management and control with the application and host environment. .
Documentation
EMC Powerlink
Solutions Enabler documentation can be downloaded from Powerlink one document at a time or the full library can be downloaded as a CD image.
Downloading Solutions Enabler
y Solutions Enabler is provided to the customer on CD ROM but can also be downloaded from Powerlink y Also require license key(s)
Solutions Enabler software is host operating system specific and can be downloaded as an installation package from Powerlink.
Solutions Enabler Components

Component License
Base
symacl symcli symgate symreturn syminq symapisrv symconfigure symchg symconnect symchksum symqos -cp symhost sympart symhostfs symioctl symrdb symrslv symlvsymvg symmask symmaskdb
Commands
symaudit symcfg symdev symdrv sympd storsrvd symauth symcg symdg symevent symlabel symbcv symdisk symld symstat symqos
SYMAPI Server Configuration Manager DeltaMark (Change Tracker) Device Masking Double Checksum (Oracle PAK) Dynamic Cache Partitioning Mapping Solution (SRM) Open Replicator/LM Open Replicator/DM
symrcopy
To enable Solutions Enabler features, license keys must be installed to enable specific functionality.
Solutions Enabler Components (cont.)

Component License
Optimization (Control) SRDF/Synchronous SRDF/A (Asynchronous mode support) SRDF/Automated Replication SRDF/Consistency Groups SRDF/Star Symmetrix Priority Control TimeFinder/Mirror TimeFinder/Clone TimeFinder/Snap TimeFinder/Consistency Groups
symoptmz
Commands
symrdf
symioctl
symrdf set mode async
symreplicate symcg -cg CgName enable symstar symqos -pst symioctl symclone symsnap symreplicate start -consistent symsnap activate -consistent symclone activate -consistent
symmir symmir
symreturn
Solutions Enabler Integration with Enginuity
Third Party Applications
Host
EMC Software Applications SYMAPI
SYMCLI
Symmetrix
SIL (Symmetrix Interface Layer) Enginuity Operating Environment
y SYMCLI commands are built on top of SYMAPI library functions

Use system calls that generate low-level I/O SCSI commands that are sent to the Symmetrix
This illustrates the software layers and where each component resides. EMCs Solution Enabler APIs are the storage management programming interfaces that provide an access mechanism for managing the Symmetrix. They can be used to develop storage management applications. SYMCLI resides on a host system to monitor and perform control operations on Symmetrix arrays. SYMCLI commands are invoked from the host operating system command line (shell). The SYMCLI commands are built on top of SYMAPI library functions, which use system calls that generate low-level I/O SCSI commands to the storage arrays.
SYMAPI - SYMCLI: Whats the Difference?

y Both are used to query, configure, and control the Symmetrix
APIs are used by application and systems management programmers and are linked when the programs are compiled CLIs are typically used for ad hoc operations or called using scripting languages
y Both use the same dynamically-linked libraries

APIs are composed of C Programming-like function calls, with arguments and flags The CLIs are composed of command-line commands, with options and parameters
y API provide finer granularity y Each SYMCLI command invokes an API session
Calls SymInit() with RW access to the database Executes API call(s)
Execution logged in SYMAPI log file
SymExit() is called at the end of each command, and all dynamically allocated memory is automatically freed
y SYMAPI and CLI communicate from host to Symmetrix using the same mechanism
Sometimes the terms API and CLI are used interchangeably but there are core differences.
SYMAPI Database
y Symmetrix configuration and status information is a SYMAPI database on the management host
Reduces the number of system calls from the host Commands either act on information in the database or query the Symmetrix directly
SYMCLI commands
SYMAPI Database
To reduce the number of inquiries from the host to the storage arrays, configuration and status information is maintained in a Symmetrix host database file called the Symmetrix configuration database (default file name: symapi_db.bin). SYMCLI commands can either run in two modes y Online mode Query current status Use used for configuration and control operations Automatically attempt to gather the latest state information from the arrays Update the in-memory database and the configuration database file on the host In the event a configuration change has occurred, commands will load any updated configuration information y Commands that can execute in offline mode, such as symcfg list, retrieve data exclusively from the configuration database
Configuration Database Overview

y A collection of binary data
Flat file of binary data Physical and logical configurations information Operational status
SYMAPI Database
y Also user-defined information

User-defined logical grouping of devices (device groups) User-supplied physical and logical device names
y Location
In-memory database in process virtual address space per session In host file system when saved from in-memory database
<Root directory>\EMC\symapi\db\symapi_db.bin
Configuration and status information is maintained in a binary configuration file on the local host. Also contained is user defined management information such as device grouping and logical names that are assigned to help document the configuration. The database resides on disk and portions of it are called into memory as needed.
Discovery
y SYMAPI database is created/updated using the Discovery and/or Synchronization process
symcfg discover
SYMAPI Database
The SYMCLI commands are built on top of SYMAPI library functions, which use system calls that generate low-level I/O SCSI commands to the storage arrays. To reduce the number of inquiries from the host to the storage arrays, configuration and status information is maintained in a Symmetrix host database file (called the Symmetrix configuration database; symapi_db.bin by default). During initial command line session, or if a configuration change has occurred, the configuration database must be built, or rebuilt, with the current information for all physical devices connected to your host. To scan the hardware and rebuild the database, enter: symcfg discover. This command scans all SCSI buses, collects information about all the devices found, and rebuilds the database with the collected device information and parameters from all local and remotely attached Symmetrix devices. symcfg discover scans all SCSI buses on the host, not just those connected to Symmetrix arrays. This can take a significant amount of time to complete. Key points on Discovery: y Builds a database containing the most complete and current information for all physical devices connected to the host y The scan can take a significant amount of time y Re-run if Symmetrix configuration has changed y No significant output on screen
Solutions Enabler Daemons

y Optional daemons that support SYMCLI/SYMAPI operations
storapid Base Daemon storsrvd storevntd storgnsd storrdfd storstpd storwatchd Other daemons related to SRM and 3rd party database operations
storsrvd others storgnsd storwatchd storevtd
y Daemons execute as root, therefore applications do not need to be privileged y Start, listed and stopped using stordaemon commands
Example: stordaemon start storsrvd
storapid
SYMAPI Database
storapid (Base daemon) y Coordinates gatekeeper selection, Symmetrix locks, and parallel application syscalls to the operating system kernel storevntd (Event Daemon) y Monitors the Symmetrix and forwarded events to any client applications that have registered an interest. Can be configured to automatically log events using local log files, Unix syslog, Windows event log, and SNMP traps to an SNMP management application. storgnsd (GNS Daemon) y Provides support for global, distributed repository for DG and CG group definitions across Symmetrix arrays that are visible to all locally attached hosts. storsrvd (SYMAPI server) y Provides support for remote clients allowing them to execute symcli commands without having a direct connection to a Symmetrix or copy of the SYMAPI DB storstpd y Used to collect raw performance counters for a Symmetrix, directors, devices, disks, ports, LRU, and RDF/A. storrdfd y Provides consistency for SRDF/A MSC and RDF-ECA composite groups (CGs) in multiSymmetrix environments. storwatchd y Monitor the core Solutions Enabler daemons and automatically restart them if they crash. Note: Not all daemons are supported on all platforms.
In-Band Communicates
y Host to the Symmetrix communication is performed using standard SCSI Write Buffer/Read Buffer commands y Devices designated to receive commands are called Gatekeepers
Typically minimum sized volumes Simply used to pass commands and return response
Gatekeeper
SYMCLI Commands
The SCSI command architecture was originally defined for parallel SCSI. Today, the same command structure is also leveraged for Fibre Channel and iSCSI communications. Basically, in SCSI protocol, the initiator sends a SCSI command to the target which then responds. SCSI commands are sent in a Command Descriptor Block (CDB). The CDB consists of a one byte operation code followed by five or more bytes containing command-specific parameters. Solutions Enabler communicates to a device using the standard Write Buffer (3B) and Read Buffer Commands (3C). This device is called a Gatekeeper device but in reality, a Logical Unit (LUN) with a address like any other device. Once a gatekeeper has been successfully obtained, SYMCLI determines if a semaphore exists; if so, a lock is obtained on the device. Once the CDB sequence is processed, the gatekeeper is closed and the lock released, freeing the device for other processing.
Gatekeeper Devices
y A Gatekeeper can be any volume accessible to the host
Gatekeeper
(Typically<10MB) Appear like any other volume Usually configured as a small device Should not be used by the host for normal data processing
y Best practice is to dedicate devices as Gatekeepers

When a SYMCLI session is started, gatekeeper and database locks are used to avoid conflicts
Semaphore
Once the CDB sequence is processed, the gatekeeper is closed and the lock released, freeing the device for other processing
y Solutions Enabler commands executed on the Symmetrix Service Processor uses a pseudo Gatekeeper device
Storage Processor does not have direct access to any devices
When a Symmetrix array is installed, it is usually configures a number of Symmetrix small devices for use as gatekeeper devices. By standard convention, these are 6 cylinders in size, but need to be at least as large as the minimum volume size accessible by your host. Consult your host documentation for the minimum device size accessible by your particular host to determine the minimum gatekeeper device size for your environment.
SYMCLI Gatekeeper Selection

y y Any device that is visible to the host can be used as a Gatekeeper
Gatekeeper devices should not be used by the host system for normal IO
Selection order:
1. Defined and Associated gatekeepers 2. Defined gatekeepers 3. Small (< 10 cylinders) non-PowerPath devices, 4. Any small device in the following priority: 5. Standard non-RDF and non-meta devices 6. RDF R1 devices 7. RDF R2 devices
Best Practice is to define gatekeeper device

Places them higher on the gatekeeper candidate priority list making them more likely to be used as a gatekeepers
SYMCLI selects a gatekeeper based upon a preestablished priority list. 1. Defined and associated gatekeepers These devices have been explicitly designated as gatekeeper devices and associated with specific device groups. 2. Defined gatekeepers These devices have been explicitly designated as gatekeeper devices.
3. Small (< 10 cylinders) non-PowerPath devices Marked by the array with the inquiry gatekeeper flag. 4. Any small device in the following priority: y PowerPath y Count key data format 5. Standard non-RDF and nonmetadevices in the following priority: y Non-PowerPath y PowerPath 6. RDF R1 devices in the following priority: y Non-PowerPath y PowerPath 7. RDF R2 devices in the following priority: y Non-PowerPath y PowerPath
Gatekeeper Management
y Best Practice: Configure multiple gatekeepers per host
At least on gatekeeper for each application that is run concurrently Manage Gatekeepers to avoid conflicts
Automatically by the Base Daemon (storapid) Manually mange Gatekeepers using the SYMCLI symgate command
y Define a device as a gatekeeper

Adds the specified physical device name to the list of gatekeeper devices
Information maintained in SYMAPI DB on local host
Examples:
symgate define pd c2t0d2 (UNIX) symgate define pd physicaldrive2 (Windows)
y Associate a gatekeeper with a device group

Explicitly designates one or more gatekeepers to be used SYMCLI routes SCSI commands to any device in the device group via the associated gatekeeper with a device group Example:
symgate -g Oracle_app1 associate pd c2t0d2
While optional, configuring the base daemon (storapid) alleviates contention when there are limited gatekeeper resources available and also eliminates the need for every client to constantly select, open, lock, and ping for an available gatekeeper device for every online function. Additionally, the base daemon monitors Symmetrix External Locks (SEL) and Device External Locks (DEL), and will automatically release any lock(s) held by a crashed application. The base daemon also eliminates the need for Solutions Enabler applications to run as root. Alternatively, gatekeepers can be manually managed using the symgate command to define and associate gatekeepers for operations on specific devices. Gatekeeper definition and association information is managed in the SYMAPI database on the local host. Another option for altering the default gatekeeper selection criteria is configuring the following configuration files on the local host.
y gkavoid y gkselect
SYMAPI Server (storsrvd)

y Two types of SYMAPI configurations:
Local host directly connected to Symmetrix SYMCLI and uses local SYMAPI DB client Remote hosts direct SYM CLI commands to a remote server that is connected to the Symmetrix.
SYMCLI client SYMCLI client
y Secure communications
Socket Layer (SSL) Optional Kerberos Authenticated & encrypted connection
TCP/IP network
y On the server, storsrvd runs as a background process

stordaemon start storsrvd
storsrvd storapid SCSI or FC connected
y On the client:
Set Environment Variable: SYMCLI_CONNECT=SYMAPI_SERVER SYMCLI_CONNECT_TYPE=REMOTE Edit the netcnfg file to point the server(s)
SYMAPI Database
There are basically two types of SYMAPI configurations: Local and remote. With Local, the SMYCLI commands are issued directly on the host attached to the Symmetrix. With this type of configuration, the local SYMAPI DB is used. With remote, the SYMCLI commands are directed to a remote server that is connected to the Symmetrix. To configure remote operations: 1) Install Solutions Enabler software in the machine designated as the client, 2) Install the same Solutions Enabler software in the machine designated as the server. Invoke symlmf and apply the SYMAPI server license key. 3) Edit the netcnfg file in the client machine to include the host name or IP address of the server. 4) Set environment variables SYMCLI_CONNECT and SYMCLI_CONNECT_TYPE. 5) Issue a stordaemon start storsrvd command on the server machine.
Environment
y Environment variables can be y Examples: preset to streamline and SYMCLI_DG expedite command line session Default device group name Set common argument values for SYMCLI_NOPROMPT
a series of associated commands to eliminate repeated key strokes
y To view a list of environment variables that can be set for a given SYMCLI session:
symcli -env
Set =1 to disable verification prompt
SYMCLI_OFFLINE
Set=1 to access SYMAPI DB only
SYMCLI_SID
Specifies a default Symmetrix ID
y To view the environment variables that you currently have set:

symcli def
y Reference Solutions Enabler Commend Reference Guide
In addition to Solutions Enabler Environment variables, it is useful to include the symcli executables to the path. For UNIX: # set path=$path /usr/symcli/bin) - Unix C Shell # PATH=$PATH:/usr/symcli/bin; export PATH - Korn or Bourn shell For Windows, ensure the following SYMCLI directories are appended to the MS-DOS variable path: C:\Program Files\EMC\SYMCLI\bin
Options
y The options file contains parameters that change the default behavior of /var/symapi/db/options SYMCLI operations, SYMAPI calls, and control actions
Global parameters and restrictions Customize and streamline command line coding to your specific environment
y Examples:
SYMAPI_TRACK_SIZE_32K_COMPATIBLE= DISABLE | ENABLE SYMAPI_USE_GNS= ENABLE | DISABLE SYMAPI_MAX_CLIENTS= SYMAPI_LOGFILE_FORMAT
y Reference Solutions Enabler Installation Guide

In addition to the Environment variables, the options file can be used to modify the default behavior of Solutions Enabler.
Return Codes
y All SYMCLI commands return status or error codes
To view return code in UNIX:
echo $status (C shell) echo $? (bourn/korn shell)
To view return code in Windows:

echo %ERRORLEVEL%
For all commands:

0= CLI call completed successfully 1= CLI call failed
Others command specific return

Example: symcfg return code 24 Indicates: The Symmetrix configuration and the database file are NOT in sync
y Reference Solutions Enabler Commend Reference Guide
Solutions Enabler Debug

y Environment Variables SYMCLI_DEBUG=<value>
SYMCLI_DEBUG=32
For arguments passed to SYMCLI commands
SYMCLI_DEBUG=64
Lots of output can be very hard to read
Third Party Applications EMC Applications SYMAPI SIL (Symmetrix Interface Layer) Enginuity Operating Environment SYMCLI
SYMCLI_DEBUG=128
SRDF processing
SYMCLI_DEBUG=256
TimeFinder processing
y SYMAPI_DEBUG=<value>
Work for both SYMCLI and applications written using the SYMAPI SYMAPI_DEBUG_FILENAME Allows you to control the output
y Verify Environment: symcli -def

Where you enable debugging you will see extra information you do not normally see. Type something like "symdg create test". There are a number of possible values for the SYMAPI_DEBUG environment variable (not all of which are published); different values accomplish different debugging goals.
XML (Extensible Markup Language) Output

y Facilitates the automated processing of SYMCLI output y The data returned is identical to the standard command output, but symcfg list -output xml_element marked-up with tags y Advantage:
Call individual pieces of data by tag Relationship between different objects
<?xml version="1.0" standalone="yes" ?> <SymCLI_ML> <Symmetrix> <Symm_Info> <symid>000190102254</symid> <attachment>Local</attachment> <model>DMX3-24</model> <microcode_version>5771</microcode_version> <cache_megabytes>32768</cache_megabytes> <devices>65</devices> <physical_devices>10</physical_devices> </Symm_Info> </Symmetrix> </SymCLI_ML>
y Enabled using environment variable or CLI option

SYMCLI_OUTPUT_MODE=
xml Attribute-based tags xml_element element based tags standard default symcli output
<symcli command> -output <xml|xml_element|standard>

Output options provide a mechanism to facilitate the automated processing of SYMCLI command output. XML is a deterministic parsing tool that eases the processing of output data and provides advantages over screen-scraping tools like awk or perl. The XML industry standard is based on the experience of SGML( Standard Generalize Markup Language) and is endorsed by the World Wide Web Consortium. Details on XML can be found at: http://w3.org/XML.
Symmetrix Event Logs

y SYMCLI and SYMAPI log significant events and actions
UNIX: /var/symapi/log/symapi-yyyymmdd.log Windows: C:\Program Files\EMC\Symapi\log\symapi-yyyymmdd.log
y The log entries include the following event information:

Time tag of the event occurrence Process ID (PID) Source of the event (application name) Related (internal) API function call Name of the specific operation or event Variable event field that describes in detail the event or error
The Log files accumulate over time and can consume needed disk space. Periodically, you may need to purge the log files to conserve space. For a detailed list of possible Symmetrix events, refer to the EMC Solutions Enabler Symmetrix CLI Command Reference.
Logging Option Enable/Disable Logging Log File Name Log File Names Date Formats
Description
Disable logging by setting the environment variable SYMCLI_NOLOGGING to 1. Name and path of the default log file. Allows the creation of undated SYMAPI log files by setting the environment variable SYMAPI_DATED_LOGFILE_NAME Change date formats in the log entries by setting the environment variable
Specify the number of days to retain the log files. Maximum value: 1825 (or 5 years) Log File Retention Minimum value: 6 Default value: 0 maintains the log file forever (service processor default is 30)
Event Daemon (storevntd)

y Monitor Symmetrix, detecting and reporting events as they happen
UNIX, LINUX, and Windows environments
client
y The event daemon continually collects Symmetrix event information

Monitors in real-time Filters the events by severity and type
WARNING!
Hot Spare Invoked against disk!
y Responds:
Alerting client event applications Logs events to specified targets
Log files storevntd UNIX syslog or Windows Application Log Sends SNMP traps to SNMP management application
In UNIX, LINUX, and Windows environments, the event daemon (storevntd) enables you to monitor Symmetrix operations by detecting and reporting events as they happen. The event daemon continually collects Symmetrix event information in real-time, filters the events by severity and type, and responds by doing the following: y Alerting client event applications and/or: y Logging events to specified targets When using the daemon with a client event application (for example, the Symmetrix Management Console), the application registers with the event daemon, specifying the events in which it is interested. When used in this manner, the daemon will automatically start when the client application requests its services. When configuring the daemon to log events, you can specify to log the events to the UNIX Syslog, the Windows Event log, SNMP, and/or a file on disk. When used in this manner, you should configure the daemon to automatically start at system boot.
Auditing
y All control and configuration operations initiated by host applications are logged in a common audit log stored in the Symmetrix File System (SFS)
Maximum size of 40MB Circular log will overwrite oldest entries Audit records are read-only
y Auditing is a key security feature

Who, is doing what, when, from where Also logs attempted operations
y symaudit command used to examine and filter audit log entries

Directory Structure
y config Includes the license file and other files that determine the operation of the SYMAPI y daemons Includes storapid host daemon y db The SYMAPI database is in this directory y log Includes the STORAPI and SYMAPI log files
With normal installation some of the above directories would be created. These directories contain the extra tools and sample code the code the programmers uses to develop applications.
Directory Structure
y bin Command executables y daemons Daemon executables y doc Special Notes y man Manual pages
With normal installation some of the above directories would be created. These directories contain the extra tools and sample code the code the programmers uses to develop applications.
Installation Overview
y All components are packaged on distribution CD y Interactive install script y Installation options include
db, log, and config directories
y Select components to install y Enter license keys y Post Installation tasks

Edit path Environment variables Options file
The Solution Enabler software is installed from a distribution CD. Depending on the platform, mount or load the CD so the installation software can be accessed from the host. Start the installation process. During the installation process, you are required to enter the installation directories for such structures as the db, log, and configuration directories. The installation process also requires you to identify any installation options required. Detailed installation instructions are available for supported platforms in the Solution Enabler Installation Guide. SYMCLI provides environment variables that can be preset to streamline and expedite your command line session. These environment variables can be preset to common argument values for a series of associated commands, which eliminates repeated keystrokes for your given session. The options file, located in the SYMAPI configuration directory, contains behavior parameters that can be set to change defaults of certain behavior options within various SYMCLI components and associated SYMAPI calls. It can be used by advanced SYMCLI and SYMAPI users to customize and streamline command line coding to your specific environment.
Solutions Enabler Unix Installation Script

y Mount CD y Change directory to the location of the Solutions Enabler Kit y Run the install script
# # cd cd /<CD_ROM /<CD_ROM Mountpoint>/UNIX Mountpoint>/UNIX # # ./emc_install.csh ./emc_install.csh # # ------------------------------------------------------------------------------------------------------------------# # EMC EMC Installation Installation Manager Manager # # ------------------------------------------------------------------------------------------------------------------Copyright Copyright 2007, 2007, EMC EMC Corporation Corporation All All rights rights reserved. reserved. The The terms terms of of your your use use of of this this software software are are governed governed by by The The applicable applicable contract. contract.
The first step to installing Solutions Enabler on a UNIX platform is to mount the installation media. Each operation system will have its own mount commands for the cdrom file system.
Solutions Enabler Installation Directories

y Default Installation and working directories
Install root directory Default /opt/emc Working root directory Default /usr/emc
The The install install root root directory directory /opt/emc /opt/emc does does not not exist. exist. This This directory directory is is necessary necessary for for the the installation installation to to complete complete Successfully Successfully It It will will be be created created now. now. Is Is that that OK? OK? [y] [y] y y ./emc_install ./emc_install : : Creating Creating Directory Directory ----/opt/emc ----/opt/emc The The working working root root directory directory /usr/emc /usr/emc does does not not exist. exist. This This directory directory is is necessary necessary for for the the installation installation to to complete complete Successfully Successfully It It will will be be created created now. now. Is Is that that OK? OK? [y] [y] y y ./emc_install ./emc_install : : Creating Creating Directory Directory ----/usr/emc ----/usr/emc
If the root and working directories do not exist, the installation script will create them.
Solutions Enabler Products to be Installed

y Installation options
Install Install All All Solutions Solutions Enabler Enabler Shared Shared Libraries Libraries and and Run Run Time Time Environment? Environment? [y]: [y]: Install Install Solutions Solutions Enabler Enabler 64-bit 64-bit shared shared libraries libraries ? ? [N]: [N]: Install Install Symmetrix Symmetrix Command Command Line Line Interface Interface (SYMCLI) (SYMCLI) ? ? [y}: [y}: Install Install Solutions Solutions Enabler Enabler SRM SRM Database Database and and Run Run Time Time Components Components ? ? [N]: [N]: Install Install Options Options to to enable enable JNI JNI interface interface for for EMC EMC Solutions Solutions Enabler Enabler APIs APIs ? ? [N]: [N]:
After selecting the installation directories, two lists of Solutions Enabler products are displayed: The first list displays the Solutions Enabler that the install script detects that are already installed on the host. The second list is the products that are available on the install media to be installed. You will be prompted for additionall installation options for libraries and other host environment considerations.
Solutions Enabler Windows Installation

y Insert the CD into the CD-ROM drive
If autorun is enabled, the installation starts automatically If autorun is not enabled, run <CD-ROM drive>:wsk63rt.exe
For a Windows environment, you will process through a series of dialogs.
License
y License keys are installed to enable functionality
Using symlmf command
# # export export PATH=$PATH:/usr/symcli/bin PATH=$PATH:/usr/symcli/bin # # symlmf symlmf E E M M C C S S O O L L U U T T I I O O N N S S E E N N A A B B L L E E R R
SOLUTIONS SOLUTIONS ENABLER ENABLER LICENSE LICENSE MANAGEMENT MANAGEMENT FACILITY FACILITY
Register Register License License Key Key (y/[n]) (y/[n]) ? ? y y Enter Enter License License Key Key : : ####-####-####-$$$$ ####-####-####-$$$$
Licensed Components
y License keys are stored in: /var/symapi/config/symapi_license.dat
# # cat cat /var/symapi/config/symapi_license.dat /var/symapi/config/symapi_license.dat
License License License License License License License License License License License License License License License License License License License License License License License License License License License License Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: Key: ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### ####-####-####-#### SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI SYMAPI Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: Feature: BASE BASE / / Symmetrix Symmetrix CMODE CMODE / / Symmetrix Symmetrix ConfigChange ConfigChange / / Symmetrix Symmetrix ConnectivityBase ConnectivityBase / / All+Conn+SRM All+Conn+SRM ConnectivityZoning ConnectivityZoning /All+Conn+SRM /All+Conn+SRM DeltaMark DeltaMark / / Symmetrix Symmetrix DevMasking DevMasking / / Symmetrix Symmetrix OPTMZR OPTMZR / / Symmetrix Symmetrix ORAPAK ORAPAK / / Symmetrix Symmetrix ResourcePak ResourcePak / / All+Conn+SRM All+Conn+SRM ResourcePak ResourcePak / / Symmetrix Symmetrix SERVER SERVER / / Symmetrix Symmetrix SOLUTION_1 SOLUTION_1 / / Symmetrix Symmetrix SOLUTION_4 SOLUTION_4 / / Symmetrix Symmetrix

Help!
y In addition to the full set of documentation, there is extensive online information
man symcli symcli
Provides the version number
symcli -h
Describes how to use the SYMCLI command.
symcli -v
Displays the list of SYMCLI commands and a brief description of each command.
symcli -env
Displays a list of the environment variables that can be set for a SYMCLI session.
symcli -def
Displays a list of the environment variables that are set for the current SYMCLI session.
For more information about Solutions Enabler symcli commands, reference the Solutions Enabler Symmetrix CLI Command Reference.
Inquiry - syminq
y Performs a SCSI Inquiry and displays information about disk devices visible to host
Note Gatekeeper devices
C:>syminq C:>syminq Device Device --------------------------------------------------Type Type --------------------------------------------------\\.\PHYSICALDRIVE1 \\.\PHYSICALDRIVE1 \\.\PHYSICALDRIVE2 \\.\PHYSICALDRIVE2 \\.\PHYSICALDRIVE3 \\.\PHYSICALDRIVE3 \\.\PHYSICALDRIVE4 \\.\PHYSICALDRIVE4 \\.\PHYSICALDRIVE5 \\.\PHYSICALDRIVE5 \\.\PHYSICALDRIVE6 \\.\PHYSICALDRIVE6 \\.\PHYSICALDRIVE7 \\.\PHYSICALDRIVE7 \\.\PHYSICALDRIVE8 \\.\PHYSICALDRIVE8 \\.\PHYSICALDRIVE9 \\.\PHYSICALDRIVE9 GK GK \\.\PHYSICALDRIVE10 \\.\PHYSICALDRIVE10 GK GK
Product Product ----------------------------------------------------Vendor ID Rev Vendor ID Rev ----------------------------------------------------EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771
Device Device ---------------------Name ---------------------Name Ser Cap Ser Num Num Cap (KB) (KB) ----------------------------------------5400020080 919680 5400020080 919680 5400021080 919680 5400021080 919680 5400022080 919680 5400022080 919680 5400023080 919680 5400023080 919680 5400024080 919680 5400024080 919680 5400025080 919680 5400025080 919680 5400026080 919680 5400026080 919680 5400027080 919680 5400027080 919680 5400028080 9600 5400028080 9600 5400029080 9600 5400029080 9600
The syminq command issues a SCSI INQUIRY and optionally SCSI READ Capacity. Because it uses standard SCSI commands, it will return information about all devices seen by the host including CLARiiON and non-EMC storage. syminq h will provide online help information Note the Device Serial Number. For systems running Enginuity 5671 and higher, the maximum number of logical devices per Symmetrix array is 64,000 devices. In the output format of syminq command, the device serial number is in the format of SSNNNNNDDP. If the port number (P) is greater than 8 subtract 8 from the P field.
Discover and Verify

C:\>symcfg C:\>symcfg discover discover This This operation operation may may take take up up to to a a few few minutes. minutes. Please Please be be patient... patient...
C:\>symcfg C:\>symcfg list list S S Y Y M M M M E E T T R R I I X X
Mcode Mcode SymmID SymmID Attachment Attachment Model Model DMX3-24 DMX3-24 Version Version 5771 5771
Cache Cache Size Size (MB) (MB) 32768 32768
Num Num Phys Phys Devices Devices 10 10
Num Num Symm Symm Devices Devices 65 65
000190102254 000190102254 Local Local
C:\>symcfg C:\>symcfg verify verify -sid -sid 254 254
The The Symmetrix Symmetrix configuration configuration and and the the database database file file are are in in sync. sync.
Before issuing any SYMCLI commands, you need to initialize the SYMAPI database. This is done using the symcfg discover command. If the database is already initialized, a quick way to verify if it is current is to issue the verify command. The symcfg list command displays a list of all Symmetrix systems that are attached to the host. If it is part of an SRDF environment, Symmetrix systems that are on the other side of a SRDF link will be displayed as Remote.
Configuring Distributed (Remote) Operations

y Verify that the SYMAPI Server is running
C:\>symcfg C:\>symcfg services services list list S N S S Y Y M M A A P P I I N E E T T S E E R R V V I I C C E E S S Port Port Name Name Domain Domain Type Type Node Node Name Name Address Address Security Security
Number Number Level Level
----------------------- ----------- --------------- --------------------------------------- ----------------------------- ----------- ----------------SYMAPI_SERVER SYMAPI_SERVER TCPIP TCPIP w2k3-39-106 w2k3-39-106 10.127.39.106 10.127.39.106 2707 2707 ANY ANY
y On the local host set the Connection environment

C:\>symcli C:\>symcli -def -def Symmetrix Symmetrix Command Command Line Line Interface Interface (SYMCLI) (SYMCLI) Version Version V6.4.1.0 V6.4.1.0 (Edit (Edit Level: Level: 827) 827) built built with with SYMAPI SYMAPI Version Version V6.4.1.0 V6.4.1.0 (Edit (Edit Level: Level: 827) 827) Current Current settings settings of of the the SYMCLI SYMCLI environmental environmental variables: variables:
SYMCLI_CONNECT SYMCLI_CONNECT SYMCLI_CONNECT_TYPE SYMCLI_CONNECT_TYPE
: : SYMAPI_SERVER SYMAPI_SERVER : : REMOTE_CACHED REMOTE_CACHED
For remote operations: 1. On the host system directly connected to the Symmetrix, the storsrvd daemon should be running. 2. On the client host, set the environment variable SYMCLI_CONNECT to SYMAPI_SERVER. 3. On the client edit the configuration file <installdir>\emc\symapi\config\netcnfg and add the IP address of the host that is running the storsrvd daemon.
Software Architecture Summary

y Communicates with the Symmetrix through a gatekeeper device
Small Symmetrix Logical Volume presented to host SYMCLI Sends low level commands through these devices When running SYMCLI on the Service Processor, a pseudo gatekeeper is used
SYMAPI DB Environment Variables SYMCLI SYMAPI
GK
y Maintains a symapi database on the local host

Contains configuration and status information Created by running symcfg discover command
y The default location and name of the database file:

<Root directory>\EMC\symapi\db\symapi_db.bin
The SYMCLI commands are built on top of SYMAPI library functions, which use system calls that generate low-level SCSI commands to the storage arrays. To reduce the number of inquiries from the host to the storage arrays, configuration and status information is maintained in a Symmetrix host database file (called the Symmetrix configuration database; symapi_db.bin by default). Commands are invoked from the host command line or script. The communications path is through a Gatekeeper devices y Small devices, 6 cylinder minimum configuration for a gatekeeper y Best Practice is to have several gatekeepers Specific devices can be defined as gatekeepers A gatekeeper can be explicitly associated to a specific device group Host-resident symapi database y Contains configuration and status information y Host specific y Groups and/or relationships created on one host are not universal to all hosts in the Symmetrix environment y Created by running symcfg discover command Solutions Enabler includes Group Name Services (GNS). This enable group definitions to be shared and maintained across multiple hosts, including mainframes.
Closing Slide
Module 2: Understanding Symmetrix Configuration
Welcome to the second module of Symmetrix Monitoring and Management. In this module we will be looking closer at the Symmetrix architecture from the prospective of a host using Solutions Architecture. We will start by looking at the physical configuration including frontend, back-end, memory and disk and finish by looking at the logical configuration including logical volumes.
Understanding Symmetrix Configuration - 1
Revision History
Rev Number
1.0
Course Date
August, 2007
Revisions
Complete
Lesson 1 Physical Configuration

Objective: y Map your understanding of the physical Symmetrix configuration to the output of various SYMCLI commands y Understand the various device naming conventions from a host, Symmetrix, and Solutions Enabler perspective y Determine if a front-end director is online and if there is a physical cable connected y Describe the concept of disk groups y Describe the concepts of sparing and determine how the spare pool is configured and if there are spares invoked
2007 EMC Corporation. All rights reserved. Understanding Symmetrix Configuration - 3
Symmetrix Architecture
y Remember this Picture!
Front-end Back-end Memory
y While the host may see it as a Black Box, the Symmetrix is really a complex and sophisticated intelligent storage system
As you look at the configuration of the Symmetrix, keep this block diagram in mind.
Host and Symmetrix Perspectives

y From a host perspective, a Symmetrix array appears as a number of physical disk devices connected to one or more I/O controllers y Host applications address devices using a physical device name
/dev/dsk/c1t1d1 hdisk2 PHYSICALDRIVE4
y Symmetrix devices map to slices of a physical disk called a hyper

Typically a Symmetrix device maps to multiple hypers for redundancy
y Symmetrix configuration, and mapping to host device names is maintained in the SYMAPI DB on the management host y SYMCLI commands are used to query device details from either the SYMAPI database or directly from the Symmetrix
Configuration and status information Back-end information - Disk Directors and Physical Drives Front-add director and channel addresses
A host views a Symmetrix device as a physical disk and will assign it a logical name. In the Symmetrix, a device is a logical abstraction of a disk and maps to slices of physical disks called hypers. On the management host, the SYMAPI database contains a copy of configuration and status information. The SYMNAPI database also maps the host device name to the Symmetrix device number.
Terminology
y Symmetrix Frame y Physical Disk Drive y Hyper-volumes (Splits) y Symmetrix Logical Volume (SLV) Symmetrix Hardware
y Symmetrix ID (sid or symmid) y Physical Device (Pdev) y Logical Device (Ldev) y Symmetrix Device (Dev)
Solutions Enabler
Depending on your point of view, a Symmetrix device can have a number of different names.
Help!
y In addition to the full set of documentation, there is extensive online information
man symcli symcli
Provides the version number
symcli -h
Describes how to use the SYMCLI command.
symcli -v
Displays the list of SYMCLI commands and a brief description of each command.
symcli -env
Displays a list of the environment variables that can be set for a SYMCLI session.
symcli -def
Displays a list of the environment variables that are set for the current SYMCLI session.
For more information about Solutions Enabler symcli commands, reference the Solutions Enabler Symmetrix CLI Command Reference.
symcli
C:\Program C:\Program Files\EMC\SYMCLI\bin> Files\EMC\SYMCLI\bin> symcli symcli -v -v Symmetrix Symmetrix Command Command Line Line Interface Interface (SYMCLI) (SYMCLI) Version Version V6.0.2.0 V6.0.2.0 (Edit (Edit Level:642) Level:642) built built with with SYMAPI SYMAPI Version Version V6.0.2.0 V6.0.2.0 (Edit (Edit Level: Level: 642) 642) SYMCLI SYMCLI BASE BASE Commands Commands ... ... symcfg symcfg - Discover Discover or or display display Symmetrix Symmetrix configuration configuration information. information. Refresh Refresh the the host's host's Symmetrix Symmetrix database database file file or or remove remove Symmetrix Symmetrix info info from from the the file. file. Can Can also also be be used used to to view view or or release release a a 'hanging 'hanging Symmetrix Symmetrix exclusive exclusive lock. lock.
... ... symdev symdev - Perform Perform operations operations on on a a device device given given the the device's device's Symmetrix Symmetrix name. name. Can Can also also be be used used to to view view Symmetrix Symmetrix device device locks. locks.
symdisk symdisk - Display Display information information about about the the disks disks within within a a Symmetrix. Symmetrix. syminq syminq ... ... symlmf symlmf ... ... - Registers Registers SYMAPI SYMAPI license license keys. keys. - Issues Issues a a SCSI SCSI Inquiry Inquiry command command on on one one or or all all devices. devices.
The symcli v command provides the version number and a brief description of the available commands. In this example the output was filtered to show the commands that we will be discussing here. Note: the current release of Solutions Enabler fir 5771 is 6.02.
symcfg -help
C:\Program C:\Program Files\EMC\SYMCLI\bin> Files\EMC\SYMCLI\bin> symcfg symcfg -help -help NAME NAME symcfg symcfg - Discover, Discover, change change or or display display Symmetrix Symmetrix configuration configuration information. information. Refresh Refresh the the host's host's Symmetrix Symmetrix database database file file or or remove remove Symmetrix Symmetrix information information from from the the database database file. file. Can a Can also also be be used used to to view view or or release release a Symmetrix Symmetrix exclusive exclusive lock,or lock,or to to online online or or offline offline RDF RDF (RA) (RA) directors. directors. Scan Scan and and rebuild rebuild the the set set of of devices devices seen seen by by a a host host system. system. Display Display network network service service file file entries. entries. Display Display UNIX UNIX database database and and gatekeeper gatekeeper semaphores. semaphores. Display Display host host and and application application registration registration information. information. Display Display mainframe mainframe CU CU image image information. information. SYNOPSIS SYNOPSIS ... ...
All the SYMCLI commands support the Help option.
Discover and Verify (symcfg discover)

y Build, update, and verify the SYMAPI database
C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg discover discover This This operation operation may may take take up up to to a a few few minutes. minutes. Please Please be be patient... patient... C:\Program C:\Program Files\EMC\SYMCLI>symcfg Files\EMC\SYMCLI>symcfg verify verify -sid -sid 172 172 The The Symmetrix Symmetrix configuration configuration and and the the database database file file are are in in sync. sync. C:\Program C:\Program Files\EMC\SYMCLI> Files\EMC\SYMCLI>
During your first command line session, or if a configuration change has occurred, the configuration database must be built, or rebuilt, with the most complete and current information for all physical devices connected to your host. The following are the discovery syntax options: symcfg discover [-all | -symmetrix | -clariion | -pdev [-sid SymmID] | -sid SymmID] Example: To scan the hardware and rebuild the database, enter: symcfg discover This command scans all SCSI buses, collects information about all the arrays and devices found, and rebuilds the database with the collected device information and parameters from all local and remotely attached devices. You can limit the discovery to just Symmetrix arrays by specifying the -symmetrix or -clariion option, only physical device information by specifying the -pdev option, or just a specific Symmetrix array by specifying the -sid option. Note: symcfg discover scans all SCSI buses on the host, not just those connected to Symmetrix arrays. This can take a significant amount of time to complete. Only use symcfg discover if you have added or removed devices seen by the host. Also, if you had previously run discover and had subsequently removed one or more array(s), a later execution of discover will not remove information from the database relating to the removed Symmetrix array(s). A symcfg -sync is less performance intensive than a full discover operation
Listing Attached Symmetrix (symcfg list)

y symcfg list provides high level details of Symmetrix attached to local host y Will also provide information on remote Symmetrix in SRDF configurations
C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg list list S S Y Y M M M M E E T T R R I I X X Mcode Cache Num Mcode Cache Num Phys Phys Version Version Size Size (MB) (MB) Devices Devices 5771 5771 32768 32768 0 0 Num Num Symm Symm Devices Devices 126 126
SymmID SymmID 000190100172 000190100172
Attachment Attachment Local Local
Model Model DMX3-24 DMX3-24
The first thing that you need to do before using the SYMCLI commands is to initialize the symapi database. This is done using the symcfg discover. If the database has already been initialized, a quick way to verify that it is current and correctly reflects the actual configuration is to run the Verify command. The symcfg list command will provide a list of Symmetrix that are attached to the host. If running from the service processor, you will see the local system and if it is part of a SRDF configuration, it will query across the link and provide information about the remote Symmetrix as well. In the above example, we initialized the database, listed high level information about the attached Symmetrix, and finally verified that the symapi database is in sync with the actual configuration.
Verbose Output (symcfg list -v)

C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg list list -v -v | | more more Symmetrix Symmetrix ID: ID: 000190100172 000190100172 (Local) (Local) Time : Time Zone Zone : Eastern Eastern Standard Standard Time Time Product : Product Model Model : DMX3-24 DMX3-24 Symmetrix : Symmetrix ID ID : 000190100172 000190100172 Microcode : Microcode Version Version (Number) (Number) : 5771 5771 (168B0000) (168B0000) Microcode : Microcode Date Date : 07.21.2005 07.21.2005 Microcode : Microcode Patch Patch Date Date : 07.21.2005 07.21.2005 Microcode : Microcode Patch Patch Level Level : 59 59 Cache Cache Size Size (Mirrored) (Mirrored) : : 16384 16384 (MB) (MB) # : # of of Available Available Cache Cache Slots Slots : # : # of of PermaCache PermaCache Slots Slots In In Use Use : Max Max # # of of System System Write Write Pending Pending Slots Slots : : Max : Max # # of of DA DA Write Write Pending Pending Slots Slots : Max Max # # of of Device Device Write Write Pending Pending Slots Slots : : ... ... Symmetrix Symmetrix Total Total Operating Operating Time Time Number Number of of Configured Configured (Sym) (Sym) Devices Devices Number Number of of Visible Visible (Host) (Host) Devices Devices Number Number of of Configured Configured Actual Actual Disks Disks Number Number of of Configured Configured Hot Hot Spares Spares Number Number of of Unconfigured Unconfigured Disks Disks Maximum Maximum number number of of hypers hypers per per disk disk ... ...
460798 460798 6 6 368998 368998 184498 184498 18440 18440 : : 1 1 days, days, 14:28:55 14:28:55 : : 126 126 : 0 : 0 : : 80 80 : 4 : 4 : 0 : 0 : : 64 64
The v option of the symcfg list command will display verbose information about the configuration. To keep from scrolling the output off the screen, pipe it to more and this will allow you to display the output page at a time. In the example above we see microcode levels, status and information about the configuration. Some of the output was deleted to allow it to fit on a single page.
Environment (symcfg list -env_data)

C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg -sid -sid 097 097 list list -env_data -env_data Symmetrix Symmetrix ID ID : : 000190100097 000190100097 Timestamp Timestamp of of Status Status Data Data : : 09/29/2005 09/29/2005 12:09:54 12:09:54 System System Bay Bay Bay Bay Name Name : : SystemBay SystemBay Number Number of of Fans Fans : : 3 3 Number Number of of Power Power Supplies Supplies : : 8 8 Number Number of of Standby Standby of of Power Power Supplies Supplies : : 8 8 Summary Summary Status Status of of Contained Contained Modules Modules All All Fans Fans : : Normal Normal All All Power Power Supplies Supplies : : Normal Normal All All Standby Standby Power Power Supplies Supplies : : Failed Failed (1) (1) Drive Drive Bays Bays Bay Bay Name Name : : Bay-1A Bay-1A Number Number of of Standby Standby of of Power Power Supplies Supplies : : 8 8 Number Number of of Drive Drive Enclosures Enclosures : : 16 16 Summary Summary Status Status of of Contained Contained Modules Modules All All Enclosures Enclosures : : Normal Normal All All Link Link Control Control Cards Cards : : Normal Normal All All Power Power Supplies Supplies : : Normal Normal All All Standby Standby Power Power Supplies Supplies : : Normal Normal Bay Bay Name Name : : Bay-1B Bay-1B
Environment data including fans and power supplies.
Summary of all Directors (symcfg list dir ALL)

C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg list list -dir -dir ALL ALL Symmetrix Symmetrix ID: ID: 000190100172 000190100172 (Local) (Local) Ident Ident DF-1A DF-1A DF-2A DF-2A FA-7A FA-7A FA-8A FA-8A FA-9A FA-9A FA-10A FA-10A DF-15A DF-15A DF-16A DF-16A DF-1B DF-1B DF-2B DF-2B FA-7B FA-7B FA-8B FA-8B FA-9B FA-9B FA-10B FA-10B DF-15B DF-15B DF-16B DF-16B DF-1C DF-1C DF-2C DF-2C S S Y Y M M M M E E T T R R I I X X Symbolic Symbolic Numeric Numeric 01A 01A 02A 02A 07A 07A 08A 08A 09A 09A 10A 10A 15A 15A 16A 16A 01B 01B 02B 02B 07B 07B 08B 08B 09B 09B 10B 10B 15B 15B 16B 16B 01C 01C 02C 02C 1 1 2 2 7 7 8 8 9 9 10 10 15 15 16 16 17 17 18 18 23 23 24 24 25 25 26 26 31 31 32 32 33 33 34 34 D D I I R R E E C C T T O O R R S S Slot Status Slot Type Type Status 1 1 2 2 7 7 8 8 9 9 10 10 15 15 16 16 1 1 2 2 7 7 8 8 9 9 10 10 15 15 16 16 1 1 2 2 DISK DISK DISK DISK FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel DISK DISK DISK DISK DISK DISK DISK DISK FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel DISK DISK DISK DISK DISK DISK DISK DISK Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online
In this example we are listing the directors in the system and the high level status. Note the different ways to identify a director: Symbolic, Numeric, and Slot number. Also note that the Slot numbers are in decimal starting with one. Whereas Symmetrix slots numbers are in hexadecimal starting with zero.
Back-end Directors (symcfg -DA)

C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg -DA -DA ALL ALL list list Symmetrix Symmetrix ID: ID: 000190100172 000190100172 (Local) (Local) S D S Y Y M M M M E E T T R R I I X X D I I S S K K
D D I I R R E E C C T T O O R R S S
Ident Ident DF-1A DF-1A DF-2A DF-2A DF-15A DF-15A DF-16A DF-16A DF-1B DF-1B DF-2B DF-2B DF-15B DF-15B DF-16B DF-16B DF-1C DF-1C DF-2C DF-2C DF-15C DF-15C DF-16C DF-16C DF-1D DF-1D DF-2D DF-2D
Symbolic Symbolic 01A 01A 02A 02A 15A 15A 16A 16A 01B 01B 02B 02B 15B 15B 16B 16B 01C 01C 02C 02C 15C 15C 16C 16C 01D 01D 02D 02D
Numeric Numeric 1 1 2 2 15 15 16 16 17 17 18 18 31 31 32 32 33 33 34 34 47 47 48 48 49 49 50 50
Slot Slot 1 1 2 2 15 15 16 16 1 1 2 2 15 15 16 16 1 1 2 2 15 15 16 16 1 1 2 2
Type Type DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK DISK
Num Num Of Of Serviced Serviced Hyper Hyper Volumes Volumes 12 12 7 7 8 8 11 11 11 11 8 8 9 9 13 13 12 12 6 6 8 8 10 10 11 11 10 10
Status Status Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online
In this example we are looking only at the back-end directors or the Disk Adapters. Note the output shows the number of hyper volumes or splits serviced by each director.
Open Systems Front-end Directors (symcfg SA)

C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg -SA -SA ALL ALL list list
Symmetrix Symmetrix ID: ID: 000190100172 000190100172 (Local) (Local) S S Y Y M M M M E E T T R R I I X X Ident Ident FA-7A FA-7A FA-8A FA-8A FA-9A FA-9A FA-10A FA-10A FA-7B FA-7B FA-8B FA-8B FA-9B FA-9B FA-10B FA-10B FA-7C FA-7C FA-8C FA-8C FA-9C FA-9C FA-10C FA-10C Symbolic Symbolic 07A 07A 08A 08A 09A 09A 10A 10A 07B 07B 08B 08B 09B 09B 10B 10B 07C 07C 08C 08C 09C 09C 10C 10C Numeric Numeric 7 7 8 8 9 9 10 10 23 23 24 24 25 25 26 26 39 39 40 40 41 41 42 42 D D I I R R E E C C T T O O R R S S Slot Slot 7 7 8 8 9 9 10 10 7 7 8 8 9 9 10 10 7 7 8 8 9 9 10 10 Type Type FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel Status Status Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online Online
In this example we are looking only at the front-end directors. The SA option will display all open systems front end directors. Note the different ways to identify a director: Symbolic, Numeric, and Slot number. Also note that the Slot numbers are in decimal starting with one. Whereas Symmetrix slots numbers are in hexadecimal starting with zero.
Director and Port Status

C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg -fa -fa all all list list -port -port -sid -sid 097 097 Symmetrix Symmetrix ID: ID: 000190100097 000190100097 S S Y Y M M M M E E T T R R I I X X D D I I R R E E C C T T O O R R P P O O R R T T S S Director Director Port Port Status Status Ident Ident FA-3A FA-3A FA-4A FA-4A FA-7A FA-7A FA-10A FA-10A FA-13A FA-13A FA-14A FA-14A FA-3B FA-3B < < . . . . .> .> Type Type FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel Status Status Online Online Online Online Online Online Online Online Online Online Online Online Online Online P0 P0 ON ON ON ON ON ON ON ON ON ON ON ON ON ON P1 P1 ON ON ON ON ON ON ON ON ON ON ON ON ON ON P2 P2 P3 P3 Connection Connection Status Status
P0 P0 P1 P1 P2 P2 P3 P3 X X X X X X -
N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
In this example we are listing the front-end directors, on-line status and if there is a cable connection..
Front-end Channel Director Details (-v)

C:\Program C:\Program Files\EMC\SYMCLI\bin> Files\EMC\SYMCLI\bin> symcfg symcfg -SA -SA 07a 07a -v -v list list ... ... Director Director Identification: Identification: FA-7A FA-7A Director Director Type Type Director Director Status Status Number Number of of Director Director Ports Ports Director Director Ports Ports Status Status Director Director Connection Connection Status Status Director Director Symbolic Symbolic Number Number Director Director Numeric Numeric Number Number Director Director Slot Slot Number Number Director Director Port: Port: 0 0 WWN WWN Node Node Name Name WWN WWN Port Port Name Name Fibre Fibre Channel Channel Loop Loop ID ID Fibre Fibre Adapter Adapter Type Type SCSI SCSI Flags Flags { { Tagged_Commands Tagged_Commands Linked_Commands Linked_Commands Sync_Transfer Sync_Transfer Wide_Transfer Wide_Transfer ... ... : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : FibreChannel FibreChannel Online Online 2 2 [ON,ON,N/A,N/A] [ON,ON,N/A,N/A] [N/A,N/A,N/A,N/A] [N/A,N/A,N/A,N/A] 07A 07A 7 7 7 7 50060482d52cb306 50060482d52cb306 50060482d52cb306 50060482d52cb306 N/A N/A N/A N/A Enabled Enabled Enabled Enabled Enabled Enabled Enabled Enabled
For a more detailed information about a specific director you can use the v option to get verbose output. In this example, we are looking at a fibre Channel director which has two ports per processor. Some of the output was deleted so that it would fit on a single screen. Note: SA identifies SCSI Adapters and includes Parallel SCSI, (SA), Fibre Channel (FA), and iSCSI (SE).
Global Memory (symcfg -memory)

C:\Program C:\Program Files\EMC\SYMCLI\bin>symcfg Files\EMC\SYMCLI\bin>symcfg list list -memory -memory Symmetrix : Symmetrix ID ID : 000190100172 000190100172 Number Number of of Memory Memory Boards Boards : : 4 4 Slot Slot Number Number 0 0 1 1 2 2 3 3 Total Total (Usable (Usable Cache) Cache) Capacity(MB) Capacity(MB) 16384 16384 16384 16384 16384 16384 16384 16384 --------------32768 32768 ---------------
Here we are looking at the memory that is configured in the system. The reason we have (4) 16GB boards but only 32GB of usable capacity is this is a DMX3 with Mirrored Cache.
Physical Disk Drives (symdisk list)

C:\Program C:\Program Files\EMC\SYMCLI\bin>symdisk Files\EMC\SYMCLI\bin>symdisk list list Symmetrix Symmetrix ID ID Disks Disks Selected Selected Ident Ident ----------DF-1A DF-1A DF-1A DF-1A DF-1A DF-1A DF-1A DF-1A DF-1A DF-1A DF-1A DF-1A DF-2A DF-2A DF-2A DF-2A DF-2A DF-2A DF-2A DF-2A DF-15A DF-15A DF-15A DF-15A DF-15A DF-15A DF-15A DF-15A DF-16A DF-16A DF-16A DF-16A DF-16A DF-16A DF-16A DF-16A DF-16A DF-16A : : 000190100172 000190100172 : : 84 84 Capacity(MB) Capacity(MB) Type Hypers Total Free Actual Type Hypers Total Free Actual ------------------- ----------- --------------- --------------- --------------C73LDFX 2 70007 43283 70007 C73LDFX 2 70007 43283 70007 C73LDFX 2 70007 43283 70007 C73LDFX 2 70007 43283 70007 C146LDF 3 140014 133771 140014 C146LDF 3 140014 133771 140014 C146LDF 2 140014 133780 140014 C146LDF 2 140014 133780 140014 C146LDF 3 140014 134885 140014 C146LDF 3 140014 134885 140014 C146LDF 2 140014 134885 140014 C146LDF 2 140014 134885 140014 C146LDF 3 140014 133953 140014 C146LDF 3 140014 133953 140014 C146LDF 2 140014 133953 140014 C146LDF 2 140014 133953 140014 C146LDF 3 140014 132073 140014 C146LDF 3 140014 132073 140014 C146LDF 2 140014 132073 140014 C146LDF 2 140014 132073 140014 C146LDF 3 140014 129598 140014 C146LDF 3 140014 129598 140014 C146LDF 2 140014 133953 140014 C146LDF 2 140014 133953 140014 C146LDF 3 140014 132073 140014 C146LDF 3 140014 132073 140014 C146LDF 2 140014 132073 140014 C146LDF 2 140014 132073 140014 C73LDFX 2 70007 43283 70007 C73LDFX 2 70007 43283 70007 C73LDFX 2 70007 43283 70007 C73LDFX 2 70007 43283 70007 C146LDF 3 140014 136255 140014 C146LDF 3 140014 136255 140014 C146LDF 2 140014 133780 140014 C146LDF 2 140014 133780 140014 C146LDF 3 140014 134885 140014 C146LDF 3 140014 134885 140014
Symb Symb Int Int TID TID Vendor Vendor ------- ----- ----- ------------------01A C 0 01A C 0 SEAGATE SEAGATE 01A C 2 01A C 2 SEAGATE SEAGATE 01A C 01A C 10 10 SEAGATE SEAGATE 01A C 01A C 12 12 SEAGATE SEAGATE 01A D 1 01A D 1 SEAGATE SEAGATE 01A D 3 01A D 3 SEAGATE SEAGATE 02A C 1 02A C 1 SEAGATE SEAGATE 02A C 3 02A C 3 SEAGATE SEAGATE 02A D 0 02A D 0 SEAGATE SEAGATE 02A D 2 02A D 2 SEAGATE SEAGATE 15A C 0 15A C 0 SEAGATE SEAGATE 15A C 2 15A C 2 SEAGATE SEAGATE 15A D 1 15A D 1 SEAGATE SEAGATE 15A D 3 15A D 3 SEAGATE SEAGATE 16A C 1 16A C 1 SEAGATE SEAGATE 16A C 3 16A C 3 SEAGATE SEAGATE 16A C F 16A C F SEAGATE SEAGATE 16A C 16A C 11 11 SEAGATE SEAGATE 16A D 0 16A D 0 SEAGATE SEAGATE
The symdisk command allows access to the configuration information of the physical disks (spindles) that make up a Symmetrix array. It can be used to list all of the disks in a Symmetrix array, or only those that match certain criteria. In this example we are displaying only summary information. Displayed is the Physical disks that are configured in the system, and how much of each drive is actually being used. Note the interface is showing as C & D but with a DMX-3 they are really 0 & 1. The selection criteria allows the user to return only data about the disks on a certain disk director (DA), disk interface (INT), or disk SCSI target ID (TID). The -hypers flag can be used with -v to display additional information about each of the logical hypers on a given disk, including which Symmetrix devices they make up. The -by_diskgroup option will organize the disks by disk group number. The -disk_group option will print only disks within that disk group.
Physical Disk Drive Details (symdisk list v)

C:\Program C:\Program Files\EMC\SYMCLI\bin> Files\EMC\SYMCLI\bin> symdisk symdisk list list v v | | more more ... ... Symmetrix : Symmetrix ID ID : 000190100172 000190100172 Disks : Disks Selected Selected : 84 84 Director Director Interface Interface Target Target ID ID Disk Disk Group Group Number Number Vendor Vendor ID ID Product Product ID ID Product Product Revision Revision Serial Serial ID ID Disk Disk Blocks Blocks Block Block Size Size Actual Actual Disk Disk Blocks Blocks Total Total Disk Disk Capacity Capacity (MB) (MB) Free Free Disk Disk Capacity Capacity (MB) (MB) Actual Actual Disk Disk Capacity Capacity (MB) (MB) Hypers Hypers ... ... : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : DF-1A DF-1A C C 0 0 1 1 SEAGATE SEAGATE SX373307FC SX373307FC C73LDFX C73LDFX 3HZ7FYLM 3HZ7FYLM 143374737 143374737 512 512 143374737 143374737 70007 70007 43283 43283 70007 70007 2 2
Again, the v or verbose option can display much more information about each disk.
Disk Groups
y When the Symmetrix is configured, disk drives are assigned to disk groups
Type Capacity Performance
y Isolate and organize disk drives

Work load isolation Tiered storage
y When Symmetrix Volumes are created, space is allocated from a Disk Group
When physical disk drives are added to the Symmetrix, they can be organized into disk groups. When Symmetrix Logical devices are configured, you can specify a disk group to use.
Disk Groups (symdisk list by_diskgroup)

C:\>symdisk C:\>symdisk list list -by_diskgroup -by_diskgroup Symmetrix : Symmetrix ID ID : 000190102254 000190102254 Disks : Disks Selected Selected : 64 64 Capacity(MB) Capacity(MB) Ident Type Hypers Total Free Actual Ident Symb Symb Int Int TID TID Vendor Vendor Type Hypers Total Free Actual ----------- ------- ----- ----- ------------------- ------------------- ----------- --------------- --------------- --------------Disk Disk Group Group 1: 1: DF-1A C 0 C300LPF 2 286102 272535 286102 DF-1A 01A 01A C 0 SEAGATE SEAGATE C300LPF 2 286102 272535 286102 DF-1A C 2 C300LPF 2 286102 277227 286102 DF-1A 01A 01A C 2 SEAGATE SEAGATE C300LPF 2 286102 277227 286102 DF-1A C 4 C300LPF 2 286102 276583 286102 DF-1A 01A 01A C 4 SEAGATE SEAGATE C300LPF 2 286102 276583 286102 DF-1A C 6 C300LPF 2 286102 281445 286102 DF-1A 01A 01A C 6 SEAGATE SEAGATE C300LPF 2 286102 281445 286102 DF-1A D 1 C300LPF 2 286102 277227 286102 DF-1A 01A 01A D 1 SEAGATE SEAGATE C300LPF 2 286102 277227 286102 DF-1A D 3 C300LPF 2 286102 277227 286102 DF-1A 01A 01A D 3 SEAGATE SEAGATE C300LPF 2 286102 277227 286102 DF-1A D 5 C300LPF 1 286102 285200 286102 DF-1A 01A 01A D 5 SEAGATE SEAGATE C300LPF 1 286102 285200 286102 DF-1A D 7 C300LPF 1 286102 277655 286102 DF-1A 01A 01A D 7 SEAGATE SEAGATE C300LPF 1 286102 277655 286102 DF-16A C 1 C300LPF 2 286102 272535 286102 DF-16A 16A 16A C 1 SEAGATE SEAGATE C300LPF 2 286102 272535 286102 DF-16A C 3 C300LPF 2 286102 277227 286102 DF-16A 16A 16A C 3 SEAGATE SEAGATE C300LPF 2 286102 277227 286102 DF-16A C 5 C300LPF 2 286102 276583 286102 DF-16A 16A 16A C 5 SEAGATE SEAGATE C300LPF 2 286102 276583 286102 DF-16A C 7 C300LPF 2 286102 282383 286102 DF-16A 16A 16A C 7 SEAGATE SEAGATE C300LPF 2 286102 282383 286102 DF-16A D 0 C300LPF 2 286102 272535 286102 DF-16A 16A 16A D 0 SEAGATE SEAGATE C300LPF 2 286102 272535 286102 DF-16A D 2 C300LPF 2 286102 277227 286102 DF-16A 16A 16A D 2 SEAGATE SEAGATE C300LPF 2 286102 277227 286102 DF-16A D 4 C300LPF 2 286102 281445 286102 DF-16A 16A 16A D 4 SEAGATE SEAGATE C300LPF 2 286102 281445 286102 DF-16A D 6 C300LPF 2 286102 273900 286102 DF-16A 16A 16A D 6 SEAGATE SEAGATE C300LPF 2 286102 273900 286102 DF-1B C 1 C300LPF 2 286102 272535 286102 DF-1B 01B 01B C 1 SEAGATE SEAGATE C300LPF 2 286102 272535 286102 DF-1B C 3 C300LPF 2 286102 277227 286102 DF-1B 01B 01B C 3 SEAGATE SEAGATE C300LPF 2 286102 277227 286102
To display how physical disk are organized into disk groups, execute the symdisk list command using the by_diskgroup option.
Data Protection - Sparing

y Spares
When the Symmetrix is setup, a number of disk drives are configured as hot spares
Not User-addressable Member of Spare Pool
y When a drive has failed or the number of soft error indicate that a hard failure is probable, a replacement is initiated
Proactive monitoring through disk scrubbing
y Two sparing policies

Temporary Permanent
The sparing function is automatically initiated by Enginuity when it determines that the number of errors logging on a volume is excessive, and that a hard failure is probable. By default, invoking a spare invokes a call-home. Sparing is designed to be transparent to the user and host level processing. In fact, the only indication a user may have is a EMC Customer Engineer arrives on site to replace the failed drive. Sparing can also be initiated by a script on the service processor.
Spare Pool
y When the binfile is created, a number of drives can be set aside for the Spare pool
Minimum of two spares of each speed & capacity Plus an additional spare for every 100 drives
Spare
Spare
Spare
Spare
y Drives in the pool are used for either temporary or permanent sparing
Spare pool
When the Symmetrix is configured, spare drives are set up through the SymmWin DiskMap wizard. Members in the pool of spares can be used either as temporary spares or permanent spares. While drives are very reliable, the best practice is to configure a minimum of 2 host spares plus one additional spare for every 100 drives of each capacity and speed.
Dynamic Sparing - Temporary Replacement

y Hot spare is invoked against bad drive
Takes up a mirror position. Data is copied to the Hot spare and bad drive is made Not Ready (NR)
Host I/O processed at higher priority
M1
X
M2 Spare
y When bad drive is replaced, data is copied back to new drive

EMC Support initiates replacement
y Hot spare is released and is ready to be used again y Dynamic Sparing

(+) Data Protection (+) Maintains back-end configuration (-) Two Copy operations required
With Dynamic sparing, a hot spare is allocated as a spare and returned to the spare pool after the failing physical is replaced. This functionality is implemented by Enginuity. This process increase data protection in the event of a media failure by minimizing the time that a volume is in a degraded state. In nearly all cases, Symmetrix logical volumes will be configured with either mirrored or parity based protection. While these schemes provide protection in the event of a single drive failure, if a second failure occurs, data could be loss. By configuring Spares, if a failure occurs, or even if there is an increase in the number of soft (recoverable) errors, a Spare drive will be invoked and the data will be copied from the failing driver or the other members in the volume. After a failed drive is repaired, data is resynchronizes to the new disk. The process chooses an available spare drive of equal or larger capacity. There is no restriction preventing a 7.2K/10K/15K spare from being invoked against any other speed failing drive. For example, it is possible that a 7.2K spare will be invoked against a 15K failing drive, which may affect performance. Sparing notes: y Disk Scrubbing is a feature of Enginuity where DAs read disk blocks looking for media defects y Microcode invokes sparing when error threshold exceeded. y Spare becomes the next available mirror y Spare will emulate either CKD or FBA y Copying data (or rebuilding data) is a lower priority background task but priority can be increased using Inlines command
Permanent Sparing
y Dynamic (Temporary) Spare is invoked only for unprotected volumes
Quickly copy data to Dynamic spare before total drive failure Dynamic spare not used if volume is RAID 1, 5, or 6 protected
y Spare is identified and permanent configuration change is made

Spare must be same capacity and speed as failed drive Spare must be on a different director/port than good mirror or RAID member
y Failed is added to the Spare pool as failed

When replaced, will become available as a spare Customer Engineer initiates action
Available spare in in Bad Drive spare pool Spare Pool
After Before
x
M1
M2
Data Flow
Permanent Sparing Advantages: y Reduces the amount of time required to replace the failing drive If the volume being spared has a ready mirror, there is no need for a dynamic spare Reduces the time the system is locked from configuration changes and other operations y Time of exposure to a second drive failure in the RAID group is reduced y Reduces the need for EMC Customer Service to replace the failed drive immediately Disadvantages y More spares are required 2 + (1 per 100 physicals) for every drive type y Must follow the same configuration rules for distributing volumes Same capacity, speed, and block size Cannot be on the same disk director or loop of any mirrors of the failed disk Not all drive failures will be candidates for permanent sparing
Hot Spares Configuration

y Spare configuration information
C:\>symdisk C:\>symdisk list list -hotspares -hotspares Symmetrix Symmetrix ID ID Disks Disks Selected Selected Ident Ident ----------DF-1A DF-1A DF-1B DF-1B DF-16C DF-16C : : 000190102254 000190102254 : : 4 4
Capacity(MB) Capacity(MB) Symb Type Hypers Total Free Actual Symb Int Int TID TID Vendor Vendor Type Hypers Total Free Actual ------- ----- ----- ------------------- ------------------- ----------- --------------- --------------- --------------01A C 4 C300LPF 0 0 0 286102 01A C 4 SEAGATE SEAGATE C300LPF 0 0 0 286102 01B D 4 C300LPF 0 0 0 286102 01B D 4 SEAGATE SEAGATE C300LPF 0 0 0 286102 16C D 4 C300LPF 0 0 0 286102 16C D 4 SEAGATE SEAGATE C300LPF 0 0 0 286102 -v -v DF-1A DF-1A C C 4 4 0 0 True True
C:\>symdisk C:\>symdisk list list -spare_info -spare_info Director : Director : Interface : Interface : Target : Target ID ID : Disk : Disk Group Group Number Number : Hot : Hot Spare Spare : Failed Failed Director Director Failed Failed Interface Interface Failed Failed Target Target ID ID Failed Failed Disk Disk
: : N/A N/A : : N/A N/A : : N/A N/A : : False False

The symdisk list -hotspare command will return information about hot spare devices. If a hot spare is invoked against a failed disk, the symdisk list-v -spare_info option can be used to return information about the failed disk that has been replaced.
Hot Spares & Dynamic Spares

y Determining if hot spare is invoked
C:>/symdev C:>/symdev list list -hotspare -hotspare Symmetrix Symmetrix ID: ID: 000184500120 000184500120 Could Could not not select select any any Symmetrix Symmetrix devices devices to to list. list. C:>/symdev C:>/symdev list list -hotspare -hotspare Symmetrix Symmetrix ID: ID: 000184500120 000184500120 Device Directors Device Device Name Name Directors Device ------------------------------------------------- ----------------------- ------------------------------------------------------------------Cap Cap Sym Physical SA Attribute Sym Physical SA :P :P DA DA :IT :IT Config Config Attribute Sts Sts (MB) (MB) ------------------------------------------------- ----------------------- ------------------------------------------------------------------------0010 Not ???:? N/Grp'd NR 0010 Not Visible Visible ???:? 01A:D0 01A:D0 Unprotected Unprotected N/Grp'd NR 103 103 0098 Not ???:? N/Grp'd NR 0098 Not Visible Visible ???:? 01A:D0 01A:D0 Unprotected Unprotected N/Grp'd NR 47 47 00AE Not 13A:0 N/Grp'd RW 00AE Not Visible Visible 13A:0 01A:D0 01A:D0 2-Way 2-Way Mir Mir N/Grp'd RW 4315 4315 00C7 Not ???:? N/Grp'd NR 00C7 Not Visible Visible ???:? 01A:D0 01A:D0 Unprotected Unprotected N/Grp'd NR 47 47 00DF Not ???:? N/Grp'd NR 00DF Not Visible Visible ???:? 01A:D0 01A:D0 Unprotected Unprotected N/Grp'd NR 47 47 00F5 Not ???:? N/Grp'd NR 00F5 Not Visible Visible ???:? 01A:D0 01A:D0 Unprotected Unprotected N/Grp'd NR 47 47 0108 Not ???:? N/Grp'd RW 0108 Not Visible Visible ???:? 01A:D0 01A:D0 2-Way 2-Way Mir Mir N/Grp'd RW 469 469
When a disk fails, a hot spare is invoked against it. All devices that have mirrors on the failed disk against which the hot spare was invoked, are shown in the symdev list hotspare display as having a hot spare invoked against them. The first example shows that there are no hot spares invoked against failed disks. The second example shows one hot spare invocation against disk 01A:D0. Unprotected devices with a hot spare invoked against them become not ready (NR) because unprotected devices have no mirror to use to rebuild the data on the spare. Mirrored or RAID 5/6 protected devices continue to be ready (RW) when a hot spare is invoked against them.
Lesson 2 Logical Volume Configuration

y Objectives y Describe the relationship between physical disks and logical devices within the Symmetrix, and how they are viewed by and attached host y Briefly describe the different types of logical volumes y Compare and contrast the supported Symmetrix data protection schemes and how they are implemented y Describe how mirror positions are used y Describe the concept of meta devices
The first part of this module was focused on the physical aspects of the Symmetrix. This lesson is focused on the logical volume configuration.
Symmetrix Logical Volumes

y Logical abstraction of a disk drive
Industry term is LUN Logical Unit EMC terms often used are hyper-volume, slice, split, device, or volume Assigned a Volume Identifier
y Enginuity maps Logical Volumes to locations on Physical Disk on back-end

Protection from media failure
RAID 1 Mirroring RAID 5 and RAID 6 parity based protection
Replication
Local Remote
y Symmetrix Logical Volumes are made available to a host

Physical Connectivity Front-end Channel Address Device Masking
One of the most confusing aspects of learning the Symmetrix is the different terms that are used to describe the same thing.
Symmetrix Logical Volume Identifiers

y Symmetrix Device Name (SymDevName)
Unique hex identifier assigned when a volume is created
0000-FFFF
y Physical Device Name (PevName)

Name used by the host to uniquely identify a disk drive Maintained in the SYMAPI Database Will not be assigned if not visible to host
Example: /dev/rdsk/c1t2d0
y Symmetrix Logical Device Name (LdevName)

Assigned name Useful for documenting a system configuration
Example: ldev00c7 or ora_redo_1
y Fibre Channel World Wide Name (WWN)

Used by Fibre Channel to uniquely identify a device Example: 60060480000190100172533030303441
Every Symmetrix Logical Volume has a number of different ways to identify a Symmetrix Logical Volume. Some are used internally by the Symmetrix and others are used by host and applications. In addition a Symmetrix Logical Volume may be assigned a unique label or signature. This is normally done by the host operating system but can also be assigned using the symlabel SMYCLI command for Windows hosts.
Volume Geometry
y A Symmetrix Logical Volume is an emulation of a Physical Disk and uses similar terminology
Sector
(8) 512 byte Blocks (16) 512 byte Block (DMX3/4)
Track (R/W Head)

(8) Sectors
Cylinders
(15) Tracks
y Volume sizes are typically specified in Cylinders
9000 Cyl Device = 8.8GB (9000 X 15 X 8 X 16 X 512 = 8847360000)

Host I/O operation are managed by the Enginuity operating environment, which runs in the Symmetrix I/O subsystem (channel directors and disk directors). Because each of the physical disks are indirectly seen as part of the I/O protocol, Symmetrix devices are presented to the host with the following configuration or emulation attributes: y Each device has N cylinders. The number is configurable (blocks 960) y Each cylinder has 15 tracks (heads) y Each device track in a fixed block architecture (FBA) has 128 blocks of 512 bytes (64K) Note: previous to DMX3, the track sizes for FBA devices 32K y Mainframe hosts use Count Key Devices (CKD) uses variable block sizes. Maximum Volume size that can be configured is 65520 Cylinders. If host applications require larger volumes, multiple volumes can be combined to form a metavolume.
Symmetrix Logical Volume Emulations

y Open Systems hosts use Fixed Block Architecture (FBA)
Each block is a fixed size of 512 bytes A host sees a Symmetrix device as a number of 512 byte blocks
512 Byte Data Block
y Mainframes use Count Key Data (CKD) devices
Count Key
Data
Count field indicates the data records physical location (cylinder and head) record number, key length, and data length Key field is optional and contains information used by the application
For example it could be used as an index
Data field is the area which contains the user data and is variable in length
The Symmetrix supports two emulation modes: FBA and CKD. Open systems use FBA emulation and mainframe uses CKD. CKD data fields are variable in length and can extend beyond the 512 byte blocks used internally in the Symmetrix. The size of a volume is specified by the number of cylinders with one cylinder equaling 15 tracks. A track is the unit of cache allocation. With open systems, the track size was historically 32K and a mainframe track was 57K. With the DMX-3, a track size is now 64K. The reason for this change is more efficient cache utilization.. A notable exception to the 512-byte open systems rule is AS/400 (iSeries). It uses 520 bytes per block. Regardless of the emulation, internally, the Symmetrix stores both FBA and CKD data on physical disk in 512 byte FBA format.
Symmetrix Device Types

Standard Devices Gatekeeper Devices y A Symmetrix device configured for normal Symmetrix operation y Configured using protection method such as RAID 1, RAID-5, RAID 6 y SCSI commands executed by SYMAPI are transferred to the Symmetrix array via a Symmetrix device that is designated as a gatekeeper device y Any device can be used for a gatekeeper but typically a number of vsmall volumes are configured specifically to be used as gatekeepers Metadevices Multiple Standard Devices can be concatenated to create larger devices y Consists of a metahead and its member devices. The metahead is the first device in the metadevice sequence and is responsible for receiving all incoming commands. It also identifies the entire metadevice. When an incoming command for the metahead is processed, the Symmetrix determines which metadevice member should execute the command. y Potential Performance benefits BCV Device y Specialized devices used by the Symmetrix TimeFinder application to create a local copy of data contained in a standard Symmetrix device y Typically used for backup, restore, decision support, and application testing y BCV device have own host address, and is configured as a stand-alone Symmetrix device

SRDF Devices y Type of volumes used by Symmetrix Remote Data Facility (SRDF) to create a devicelevel mirror in a remote Symmetrix y Devices are configured as either RDF1 or RDF2 where during normal operation data is mirrored from the RDF1 to RDF2 devices y RDF1 are the local volume used for production y RDF2 devices are in configured in a remote Symmetrix to support disaster recovery and other applications Dynamic RDF Devices y Enables the dynamic creation, deletion, and swapping of RDF1 and RDF2 relationships online y Dynamic RDF must be enabled via the Configuration Manager and the devices must be designated as Dynamic RDF-capable devices Virtual Devices y Used in TimeFinder/Snap operations y A host-accessible device containing track-level location information (pointers), which indicates where the copy session data is located in the physical storage. y Virtual devices consume minimal physical disk storage, as they store only the address pointers to the data stored on the source device or a pool of save devices.

Save Devices y Special devices (not mapped to the host) that provide physical storage space for preupdate images or changed tracks during a virtual copy session of TimeFinder/Snap and SRDF/A DSE operations y Save devices are a predefined pool of storage devices and must be configured for this purpose. y Save devices are assigned a Symmetrix device number and can be unprotected, mirrored, or parity RAID Device Masking (VCM) Devices) y The device masking database (VCMDB) holds device masking records y Symmetrix devices that have been masked for visibility only to certain hosts.
DRV Devices
y A non-user-addressable Symmetrix device used by the Symmetrix Optimizer in logical volume swapping operations y Temporarily hold user data while reorganization of the devices is being executed
Data Protection Options for DMX3/4

Option RAID 1 Characteristics
y y Write to two separate physical drives Read from single drive DMSP Parity based protection Striped data and parity 3+1 and 7+1 Configurations Two parity drives 6 + 2 and 14 + 2 Data Availability is primary Performance is a secondary consideration New with Enginuity 5772 Not recommended
Protection
Performance Cost Low Cost
Highest
Fastest
RAID 5
y y
High
Fast Read Good Write
Lower Cost
RAID 6
y y y y
Higher
Fast Read Fair write
Lowest Cost
Unprotected
RAID 5 is based on the industry standard algorithm and can be configured with three data and one parity, or seven data and one parity. While the latter will provide more capacity per $, there is a greater performance impact when in degrade mode where a drive has failed and all surviving drives must be read in order to rebuild the missing data. RAID 6 is new with 5772 and is focused on availability. With the new larger capacity disk drives, rebuilt times may take multiple days increasing the exposure to a second disk failure. Works nicely with tiered storage and where you have data that is written once and read occasionally. Other data protection schemes include remote replication using SRDF.
Physical Disk and Hyper Volumes

Physical Disk Hyper Volumes
10 GB 8 GB 9 GB 6 GB
146 GB
8 GB 11 GB 8 GB 36 GB
The software used to split physical disks into volumes is called Hyper Volume Extension. Symmetrix physical disks are split into logical Hyper Volumes. Hyper Volumes (disk slices) are then defined as Symmetrix Logical Volumes (SLV). SLVs are internally labeled with hexadecimal identifiers (0000-FFFF). The maximum number of host addressable logical volumes per Symmetrix configuration is 64,000 using Enginuity 5771+. While Hyper Volume and split refer to the same thing (a portion of a Symmetrix physical disk), a Symmetrix Logical Volume is a slightly different concept. A Symmetrix Logical Volume is an abstraction of a disk drive that is presented to a host via a Symmetrix channel director port. As far as the host is concerned, the Symmetrix Logical Volume is a physical drive. Symmetrix Logical Volumes are defined in the Symmetrix Configuration (BIN File). From the Symmetrix perspective, physical disk drives are partitioned into hyper volumes. A hyper volume could be used as an unprotected Symmetrix logical volume, a mirror of another hyper volume, a Business Continuance Volume (BCV), a member for RAID 5 or RAID 6 volume, a remote mirror using SRDF, and other uses. Volume Table of Contents (VTOC) on disk are used to map logical volumes to physical disks. These data structures are created during initial installation. y Maximum hyper volumes per physical disk varies with software version - currently 255 maximum y Hyper volumes can be of variable size
Mirror Positions
y Internally each Symmetrix Logical Volume is represented by four mirror positions M1, M2, M3, M4
Data structure
y Each mirror positions represents a mirror copy of the volume data protection
Local Replication Remote Replication Dynamic Spare Drive Relocation
Symmetrix Logical Volume
M1
M2
M3
M4
Within the Symmetrix, each logical volume is represented by four mirror positions M1, M2, M3, and M4. These Mirror Positions are actually data structures that point to a physical location of a data mirror and the status of each track of data. Each position either represents a mirror or is unused. For example, an unprotected volume will only use the M1 position to point to the only data copy. A RAID-1 protected volume will use the M1 and M2 positions. If this volume was also protected with SRDF, three mirror positions would be used, and if we add a BCV to this SRDF protected RAID-1 volume, all four mirror positions would be used.
Mirror Position Usage Examples
y Mirror positions may float

M1
Unprotected Raid 1 RAID 5 RAID 6 RAID 1 + BCV RAID 5 + BCV RAID 1 + BCV+ SRDF RAID1 with Dynamic Spare Invoked
M2 Not Used Data Parity Not Used Data Parity Data Data
M3 Not Used Not Used Not Used Not Used BCV BCV BCV Dynamic Spare
M4 Not Used Not Used Not Used Not Used Not Used Not Used RDF Not Used
Data Data Data Data Data Data Data Data
Mirroring: RAID-1
y Two physical copies of the same data
Different disk Different back-end director
Physical Drive LV 04B M1
Cache
Logical Volume 04B
Disk Director
Physical Drive
Data Slot
Host Write
Disk Director
LV 04B M2
Mirroring provides applications both high performance and availability. Mirroring maintains a duplicate copy of a logical volume on two physical drives. The Symmetrix maintains these copies internally by writing all modified data to both physical locations. The mirroring function is transparent to attached hosts, as the hosts view the mirrored pair of hypers as a single logical volume. Key points about RAID-1 Mirroring: y Identical data is stored on a redundant mirrored disk y Mirrored devices by default will reside on different back-end directors y Should either disk fail, the other disk provides continuous availability y Transparent to the host system y Symmetrix will call home if one of the mirrored pair fails y All data on the good mirror is copied to replacement drive y For read intensive applications, mirroring provides even higher performance than nonmirrored disk because there are two possible source for the read and the Dynamic Mirror Service Policy (DMSP) will choose the copy that provides the best performance Mirroring is the best form of data protection available, but it is also the most expensive of all protection options (50% reduction in total storage capacity).
Mirror Service Policies

y Interleave Service Policy y Split Service Policy y Dynamic Mirror Service Policy (DMSP)
Logical Volume 000
Physical Drive
Physical Drive
LV 000 M1 LV 004 M2 LV 008 M1 LV 00C M2
Logical Volume 004
LV 000 M2 LV 004M1
Logical Volume 008
LV 008 M2 LV 00C M1
Logical Volume 00C

During a read operation, the Mirror Service Policy choose the best hyper in the mirrored pair based upon three service policies: y Interleave Service Policy - Shares the read operations of the mirrored pair by reading tracks from both logical hypers in an alternating method: a number of tracks from the primary volume (M1) and a number of tracks from the secondary volume (M2). The interleave policy is designed to achieve maximum throughput. y Split Service Policy - Read operations are assigned to either the M1 or the M2 logical volume, but not to both. Split is designed to minimize head movement. y Dynamic Mirror Service policy (DMSP) - Utilizes both Interleave and split for maximum throughput and minimal head movement. Dynamic Mirror Service policy adjusts the policy for each logical volume dynamically, based on access patterns detected. This is the default mode within the Enginuity operating environment.
Dynamic Mirror Service Policy (DMSP)

y Determines which disk in a set of mirrored disks should service a read request such that:
Workload balancing is optimized Spindle Seek times are minimized Statistically, a disk will seek no more than its maximum
y DMSP collects samples of logical volume activity and stores them in the Enginuity SFS.
Uses this data to make informed performance decisions
The Symmetrix uses one of the three policies when reading from mirrored devices. The M1 and M2 policies tend to be better for random I/O workloads. By their nature, random workloads tend to spread the traffic over many devices, so even with the M1 or M2 policy, many disks are working simultaneously to retrieve data. The Interleave policy tends to be better for sequential I/O workloads. The interleaving allows both mirrors to service reads at the same time. Each device can be permanently locked into using one of the policies, or the Dynamic Mirror Service Policy (DMSP) can be enabled. With DMSP enabled, the Symmetrix analyzes the workload on the DAs, drive interfaces, and drives and runs simulations to predict how each policy would change the situation. It chooses the policy that best balances the workload and also minimizes the disk latency and seek times for each device. After a short delay, the process of choosing a policy is repeated to keep up with dynamically changing workloads. The DMSP algorithm optimizes READ performance in two ways: 1. DMSP balances the load among the physical drives, so that even when the workload is skewed, the loads on the physical drives are as balanced as possible. For example, when a LUN is very busy, the DMSP will use both mirrors to service the read operations of this LUN. 2. DMSP minimizes the seek operations on the disks. Whenever the load balancing allows it, each disk reads from one half of its platters: either from the outermost cylinders, or from the innermost cylinders. Because of this, the seek distances are significantly shortened. Seek minimization is a critical optimization because, as disk capacity increased over the years, the data transfer time was improved much more significantly than seek time.
RAID 1 Layout
y Full Mirror y Best Overall Performance y Best Availability
Disk DA Track 0 1 2 3 4 5 6 7 8 9 A B D1 01b Data Data Data Data Data Data Data Data Data Data Data Data D2 15b Data Data Data Data Data Data Data Data Data Data Data Data D3 16a Data Data Data Data Data Data Data Data Data Data Data Data D4 02a Data Data Data Data Data Data Data Data Data Data Data Data D5 02b Data Data Data Data Data Data Data Data Data Data Data Data D6 01a Data Data Data Data Data Data Data Data Data Data Data Data D7 15a Data Data Data Data Data Data Data Data Data Data Data Data D8 16b Data Data Data Data Data Data Data Data Data Data Data Data
Volume 1
Volume 2
Volume 3
Volume 4
Two copies of each track is maintained on different disks.
RAID 5 Parity Protection
Data: 111
Data: 101
Data: 001
Parity: 011
y Parity is calculated using the Boolean Exclusive OR logic function y Truth Table:
Bit 1 0 0 1 1 Bit 2 0 1 0 1 Result 0 1 1 0
XOR
111 Data 101 Data 010 Partial
XOR
010 Data 001 Data 011 Parity
In the above example we are only using 3 bits to represent a block of data on disk, however, the XOR works the same way whether 3 bits or 512. The advantage of RAID 5 is more of the disk space is available for data storage. With a 4+1 configuration, 75% of the available disk space is used to store data and only 25% is used for data protection. When compared with RAID1, where 50% of the available storage is used for data protection, you can see that this is a cost effective approach. Effective utilization is even higher when using 7+1 configuration where 87.5% of available space is used for data storage.
Data Reconstruction
Data: 111
Data: 101
Data: 001
Parity: 011
y If a track or whole disk is lost, data can be recovered through parity calculations y If the disk containing 001 is lost, the parity is XORed with the surviving members to reconstruct the missing data
XOR
111 Data 101 Data 010 Partial
XOR
010 Data 011 Parity 001 Reconstructed Data
During normal read operations, the performance of RAID 5 is comparable to RAID1. However during degraded mode, where one member drive has failed, performance is impacted as all surviving members must be read in order to reconstruct the data on a failed drive. The performance impact of a drive failure is even greater with a 7+1 configuration.
RAID 5 Layout
y 3 + 1 Striped Parity and Data
Disk DA D1 01b P P P P 0/C 0/D 0/E 1/0 1/9 1/A 1/B 1/C 2/6 2/7 2/8 2/9 D2 15b 0/0 0/1 0/2 0/3 P P P P 1/D 1/E 2/0 2/1 2/A 2/B 2/C 2/D D3 16a 0/4 0/5 0/6 0/7 1/1 1/2 1/3 1/4 P P P P 2/E 3/0 3/1 3/2 D4 02a 0/8 0/9 0/A 0/B 1/5 1/6 1/7 1/8 2/2 2/3 2/4 2/5 P P P P D5 02b P P P P 0/C 0/D 0/E 1/0 1/9 1/A 1/B 1/C 2/6 2/7 2/8 2/9 D6 01a 0/0 0/1 0/2 0/3 P P P P 1/D 1/E 2/0 2/1 2/A 2/B 2/C 2/D D7 15a 0/4 0/5 0/6 0/7 1/1 1/2 1/3 1/4 P P P P 2/E 3/0 3/1 3/2 D8 16b 0/8 0/9 0/A 0/B 1/5 1/6 1/7 1/8 2/2 2/3 2/4 2/5 P P P P
Volume 1
Volume 1
RAID 5 device are made up of four or eight hyper volumes, each containing both data and rotating parity. The stripe width is the amount of data stored on a RAID 5 member. The Symmetrix uses a stripe width of 4 by default, which means 4 tracks are written to a member before writing to the next member. The Table above represents how data might be laid out.
RAID 5 Random Write

y Disk drives assist in parity calculation Cache Data Slot
Host Write
Data XOR
Ol R XO
Write
Data
w Ne & d
ta Da
Parity Slot
XO R
Pa rity
Wr ite
Parity
The added step of XORing the old and new data, and XORing it with the old parity to create new parity, and then writing the new data and new parity is referred to as the RAID 5 write penalty.
RAID 5 Large Sequential Write

y Parity calculation performed in cache Cache Data Slot Data Slot Data Slot
Host Write
it e Wr w ne ta Da
Data
a new D Write
ta
Data
Write new Data
XOR in Cache
Data
Writ e ne w Pa rity
Parity Slot
Parity
Large sequential writes have a performance advantage in that parity can be calculated in cache eliminating an XOR operation.
RAID 6
Data Data Data Data Data Data Data Data
y Similar to Raid 5 but two parity drives

Independent horizontal & diagonal parity 6 + 2 and 14 + 2 configurations Both FBA and CKD emulation supported
y Data Availability is the primary motivation

All other protection schemes cause data lost in dual drive failures Data lost if unrecoverable read error during rebuild Larger capacity drives increase the exposure because of increased rebuild time
y Performance is a secondary
Response time Rebuild times
y Supported with Enginuity 5772

RAID level 6 was not among the original raid levels. It adds an additional parity block to a RAID 5 array. It needs at least four disks (2 disks for the capacity, 2 disks for redundancy). Calculations are much more difficult than the simple XORs used with RAID 5. Like with RAID 5, parity and data are on different disks, for each block. The two parity blocks are also located on different disks. This should be positioned as an availability solution. RAID 6 is slower than RAID 5, but it allows the RAID to continue with any two disks failed. The EMC implementation of RAID 6 yields performance is competitive with the industry. And is ideally suited for archival purposes where data is written once and read occasionally.
RAID 6 Layout
y 6+2 Striped data and parity - Dual Parity
Disk DA D1 01b P P P P 1/9 1/A 1/B 1/C 3/3 3/4 3/5 3/6 9/9 DP D2 15b DP DP DP DP P P P P 3/7 3/8 3/9 3/A 9/D 11/6 D3 16a 0/0 0/1 0/2 0/3 DP DP DP DP P P P P 10/2 11/A D4 02a 0/4 0/5 0/6 0/7 1/D 1/E 2/0 2/1 DP DP DP DP 10/6 11/E D5 02b 0/8 0/9 0/A 0/B 2/2 2/3 2/4 2/5 3/B 3/C 3/D 3/E 10/A 12/3 D6 01a 0/C 0/D 0/E 0/1 2/6 2/7 2/8 2/9 4/0 4/1 4/2 4/3 10/E 12/7 D7 15a 1/1 1/2 1/3 1/4 2/A 2/B 2/C 2/D 4/4 4/5 4/6 4/7 P 12/B D8 16b 1/5 1/6 1/7 1/8 2/E 3/0 3/1 3/2 4/8 4/9 4/A 4/B DP P
Volume
This algorithm calculates two types of parity Horizontal Parity (HP) and Diagonal Parity (DP), in order to provide protection against the loss of two members of the RAID group. The two parities are calculated using different sets of the user data, and are independent of one another. Horizontal Parity is the equivalent of RAID 5 parity, and is calculated from the data across all the data disks. Diagonal Parity (DP) is made up of segments; each DP segment is calculated on a selected group of data segments. Each DP segment skips a different data drive in its calculation, which is key to the ability of this algorithms to reconstruct data after a double drive failure. Another important requirement of this algorithm is that the number of data drives used in the DP calculations be a prime number. Symmetrix offers RAID 6 (6+2) and RAID 6 (14+2) protection (6 data or 14 data, with 2 parities each), neither of which is 'prime'. The Symmetrix implementation of Even-Odd assumes there to be 17 data drives in the RAID group named D1-D17. For each of the protection schemes, the 'real' data drives take up positions D1D6 (for 6+2) or D1-D14 (for 14+2) in the DP calculations; the remaining drives (Dx up to D17) are 'NULL disks'. These NULL disks exist only for the sake of the DP calculations, and the 'data' on these NULL disks is assumed to be all zero when calculating parity. These NULL disks do not consume any system resources (memory, cache, or disks).
RAID-10 CKD Volume

y Mainframe CKD devices support RAID-10
RAID-10 - Mirrored Stripes Each group consists of 8 hypers Data is striped across four hypers and then mirrored
RAID-10 Meta Volume M1

Cyl 1 Cyl 5 Cyl 9 Cyl 13 Cyl 17 Cyl 2 Cyl 6 Cyl 10 Cyl 14 Cyl 18 Cyl 3 Cyl 7 Cyl 11 Cyl 15 Cyl 19 Cyl 4 Cyl 8 Cyl 12 Cyl 16 Cyl 20 Cyl 1 Cyl 5 Cyl 9 Cyl 13 Cyl 17 Cyl 2 Cyl 6 Cyl 10 Cyl 14 Cyl 18
M2
Cyl 3 Cyl 7 Cyl 11 Cyl 15 Cyl 19 Cyl 4 Cyl 8 Cyl 12 Cyl 16 Cyl 20
This is a diagram of a RAID-10 CKD volume. These are sometimes referred to as CKD Meta Volumes. The portion of the logical volume which resides on one physical volume is called a stripe. Each RAID-10 stripe group consists of four stripes distributed across four volumes. These are mirrored to consist of eight total hypers. The stripe group is constructed by alternately placing one cylinder across each of the four volumes. These volumes cannot be on the same disk director. The eight volumes are distributed across the Symmetrix back end for additional availability and improved performance.
Meta Volumes
y Maximum Symmetrix Logical Volume size is 65520 Cylinders
Multiple Logical Volumes can be combined to create Meta Volumes
Capacity Performance
Logical Volume 001 Logical Volume 002 Logical Volume 003 Logical Volume 004 Four individual Symmetrix 16 GB logical volumes
Meta Volume
LV 001 LV 002 LV 003 LV 004
Configured as a Meta Volume, appears to the host as a single 64 GB disk Example: /dev/dsk/c1t1d0
Two or more Symmetrix Logical Volumes can be grouped into a Meta Volume configuration and presented to a host as a single disk. Meta Volumes allow customers to present larger Symmetrix logical volumes than the current maximum hyper volume size(65520 Cylinders) and satisfies requirements for environments where there is a limited number of host addresses or volume labels available. For example Windows only has 26 Drive Letters available. With open systems Meta Volumes, data is either striped or concatenated. Notes: y An Open Systems Meta volume may have a maximum of 255 members (SLVs) y Meta size up to 1.1 Terabyte (up to 7.5TB with RPQ) y Members can be of different size y Does not have to be contiguous devices y Members may be mirrored, RAID-5, or Parity RAID volumes y Members of a Meta Volume may also be configured for remote mirrors using SRDF y Dynamic Spares may be invoked for any member of a Meta Volume y Refer to the following website for further Meta Volume information: http://www.cs.isus.emc.com/config/Configuration/Features/Meta/meta.htm
SYMCLI Logical Volume Info

y Three commands to obtain a list of devices:
syminq
Only devices visible to host Uses SCSI Query command Does not use the SYMAPI Database
sympd
Lists Symmetrix devices that are host-visible Obtains information from the SYMAPI database
symdev
Displays information about all Symmetrix devices, including those that are not visible to the host Information obtained from SYMAPI database
SCSI Query
y syminq uses standard SCSI query commands y Does not use read or update the SYMAPI database
C:>syminq C:>syminq Device Device --------------------------------------------------Name Type Name Type --------------------------------------------------\\.\PHYSICALDRIVE1 \\.\PHYSICALDRIVE1 \\.\PHYSICALDRIVE2 \\.\PHYSICALDRIVE2 \\.\PHYSICALDRIVE3 \\.\PHYSICALDRIVE3 \\.\PHYSICALDRIVE4 \\.\PHYSICALDRIVE4 \\.\PHYSICALDRIVE5 \\.\PHYSICALDRIVE5 \\.\PHYSICALDRIVE6 \\.\PHYSICALDRIVE6 \\.\PHYSICALDRIVE7 \\.\PHYSICALDRIVE7 \\.\PHYSICALDRIVE8 \\.\PHYSICALDRIVE8 \\.\PHYSICALDRIVE9 \\.\PHYSICALDRIVE9 GK GK \\.\PHYSICALDRIVE10 \\.\PHYSICALDRIVE10 GK GK Product Product ----------------------------------------------------Vendor ID Rev Vendor ID Rev ----------------------------------------------------EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 EMC SYMMETRIX 5771 Device Device ----------------------------------------Ser Cap Ser Num Num Cap (KB) (KB) ----------------------------------------5400020080 919680 5400020080 919680 5400021080 919680 5400021080 919680 5400022080 919680 5400022080 919680 5400023080 919680 5400023080 919680 5400024080 919680 5400024080 919680 5400025080 919680 5400025080 919680 5400026080 919680 5400026080 919680 5400027080 919680 5400027080 919680 5400028080 9600 5400028080 9600 5400029080 9600 5400029080 9600
Physical Device (sympd list)

y sympd displays information about Symmetrix Logical Volumes that are visible to the attached host y Queries and update the SYMAPI database
C:\Program C:\Program Files\EMC\SYMCLI\bin>sympd Files\EMC\SYMCLI\bin>sympd list list Symmetrix Symmetrix ID: ID: 000000006196 000000006196 Device Device Name Name Directors Directors Device Device
------------------------------------------------- ------------------------Physical Physical Sym Sym
------------------------------------------------------------Cap Cap SA Attribute Sts SA :P :P DA DA :IT :IT Config Config Attribute Sts (MB) (MB)
----------------------------------------------------- ------------------------- ----------------------------------------------------------------/dev/rdsk/c1t0d0s2 /dev/rdsk/c1t0d0s2 0000 0000 /dev/rdsk/c1t0d1s2 /dev/rdsk/c1t0d1s2 0040 0040 /dev/rdsk/c1t0d2s2 /dev/rdsk/c1t0d2s2 0041 0041 /dev/rdsk/c1t0d3s2 /dev/rdsk/c1t0d3s2 0042 0042 /dev/rdsk/c1t0d4s2 /dev/rdsk/c1t0d4s2 0043 0043
02A:1 02A:1 01C:C6 01C:C6 Unprotected Unprotected N/Grp'd N/Grp'd VCM VCM WD WD 02A:1 02A:1 16A:D4 16A:D4 Unprotected Unprotected N/Grp'd N/Grp'd RW RW 02A:1 02A:1 01D:D4 01D:D4 Unprotected Unprotected N/Grp'd N/Grp'd RW RW 02A:1 02A:1 16B:C4 16B:C4 Unprotected Unprotected N/Grp'd N/Grp'd RW RW 02A:1 02A:1 01A:C4 01A:C4 Unprotected Unprotected N/Grp'd N/Grp'd RW RW
47 47 188 188 188 188 188 188 188 188
Symmetrix Devices (symdev list)

C:\>symdev C:\>symdev list list Symmetrix ID: Symmetrix ID: 000190102254 000190102254 Device Directors Device Device Name Name Directors Device --------------------------------------------- ------------------------- ----------------------------------------------------------------Cap Cap Sym SA Attribute Sts (MB) Sym Physical Physical SA :P :P DA DA :IT :IT Config Config Attribute Sts (MB) ------------------------------------------------------------------------------------------ ------------- ---------------------------------0020 N/Grp'd RW 898 0020 \\.\PHYSICALDRIVE1 \\.\PHYSICALDRIVE1 08A:0 08A:0 16A:C7 16A:C7 2-Way 2-Way Mir Mir N/Grp'd RW 898 0021 N/Grp'd RW 898 0021 \\.\PHYSICALDRIVE2 \\.\PHYSICALDRIVE2 08A:0 08A:0 16B:C6 16B:C6 2-Way 2-Way Mir Mir N/Grp'd RW 898 0022 N/Grp'd RW 898 0022 \\.\PHYSICALDRIVE3 \\.\PHYSICALDRIVE3 08A:0 08A:0 16C:C5 16C:C5 2-Way 2-Way Mir Mir N/Grp'd RW 898 0023 N/Grp'd RW 898 0023 \\.\PHYSICALDRIVE4 \\.\PHYSICALDRIVE4 08A:0 08A:0 16D:C4 16D:C4 2-Way 2-Way Mir Mir N/Grp'd RW 898 0024 N/Grp'd RW 898 0024 \\.\PHYSICALDRIVE5 \\.\PHYSICALDRIVE5 08A:0 08A:0 01B:D4 01B:D4 2-Way 2-Way Mir Mir N/Grp'd RW 898 0025 N/Grp'd RW 898 0025 \\.\PHYSICALDRIVE6 \\.\PHYSICALDRIVE6 08A:0 08A:0 01A:D5 01A:D5 2-Way 2-Way Mir Mir N/Grp'd RW 898 0026 N/Grp'd RW 898 0026 \\.\PHYSICALDRIVE7 \\.\PHYSICALDRIVE7 08A:0 08A:0 01D:D4 01D:D4 2-Way 2-Way Mir Mir N/Grp'd RW 898 0027 N/Grp'd RW 898 0027 \\.\PHYSICALDRIVE8 \\.\PHYSICALDRIVE8 08A:0 08A:0 01C:D5 01C:D5 2-Way 2-Way Mir Mir N/Grp'd RW 898 0028 N/Grp'd RW 9 0028 \\.\PHYSICALDRIVE9 \\.\PHYSICALDRIVE9 08A:0 08A:0 16C:C7 16C:C7 2-Way 2-Way Mir Mir N/Grp'd RW 9 0029 N/Grp'd RW 9 0029 \\.\PHYSICALDRIVE1008A:0 \\.\PHYSICALDRIVE1008A:0 16D:C6 16D:C6 2-Way 2-Way Mir Mir N/Grp'd RW 9 002C ???:? N/Grp'd RW 8438 002C Not Not Visible Visible ???:? 16C:D6 16C:D6 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0033 ???:? N/Grp'd RW 8438 0033 Not Not Visible Visible ???:? 01D:D0 01D:D0 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0034 ???:? N/Grp'd RW 8438 0034 Not Not Visible Visible ???:? 16B:C0 16B:C0 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0035 ???:? N/Grp'd RW 8438 0035 Not Not Visible Visible ???:? 16A:C1 16A:C1 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0036 ???:? N/Grp'd 45000 0036 Not Not Visible Visible ???:? 01A:C2 01A:C2 RAID-5 RAID-5 N/Grp'd (M) (M) RW RW 45000 0037 ???:? N/Grp'd 0037 Not Not Visible Visible ???:? 16A:D2 16A:D2 RAID-5 RAID-5 N/Grp'd (m) (m) RW RW 0038 ???:? N/Grp'd 0038 Not Not Visible Visible ???:? 16A:C3 16A:C3 RAID-5 RAID-5 N/Grp'd (m) (m) RW RW 0039 ???:? N/Grp'd 0039 Not Not Visible Visible ???:? 01A:D1 01A:D1 RAID-5 RAID-5 N/Grp'd (m) (m) RW RW -
The symdev command displays information about Symmetrix Logical Volumes. In this example the volumes are not presented to a front end directors so *** are displayed for the SA:P column and they are not visible to the local host. The symdev command provides similar output as the sympd command, but includes all Symmetrix devices and lists them by Symmetrix device names for all devices on all or a given Symmetrix array(s). The corresponding output shows the Symmetrix device names (Sym), physical device (Physical) names, director information, and device-specific information for all devices on the Symmetrix array. Note: The physical device names are not known for those devices that are not visible to the host making the request. A value of ???:? for the SA means there is no mapping to a front-end director port. A value of ***:* means there are multiple mappings.
Device Details (symdev show)

C:\Program C:\Program Files\EMC\SYMCLI\bin> Files\EMC\SYMCLI\bin> symdev symdev show show 04a 04a Symmetrix Symmetrix ID: ID: 000190100172 000190100172 Device Device Physical Physical Name Name Device Device Symmetrix Symmetrix Name Name Device Device Serial Serial ID ID Symmetrix Symmetrix ID ID Attached Attached BCV BCV Device Device : : Not Not Visible Visible : : : : : : 004A 004A N/A N/A 000190100172 000190100172
: : N/A N/A
Attached Attached VDEV VDEV TGT TGT Device Device : : N/A N/A Vendor : Vendor ID ID : Product : Product ID ID : Product : Product Revision Revision : Device : Device WWN WWN : Device : Device Emulation Emulation Type Type : Device Device Defined Defined Label Label Type: Type: Device : Device Defined Defined Label Label : Device : Device Sub Sub System System Id Id : Device Device Block Block Size Size
EMC EMC SYMMETRIX SYMMETRIX 5771 5771 60060480000190100172533030303441 60060480000190100172533030303441 FBA FBA N/A N/A N/A N/A 0x0001 0x0001
: : 512 512
The symdev show command provides details on a specified volume. The output is continued on the following page.
symdev show (cont.)

Device Device Capacity Capacity { { Cylinders : 12280 Cylinders : 12280 Tracks : 184200 Tracks : 184200 512-byte : 11788800 512-byte Blocks Blocks : 11788800 MegaBytes : 5756 MegaBytes : 5756 KiloBytes : 5894400 KiloBytes : 5894400 } } Device : (Non-Exclusive Device Configuration Configuration : RAID-5 RAID-5 (Non-Exclusive Access) Access) Device : Device is is WORM WORM Enabled Enabled : No No Device Device is is WORM WORM Protected Protected : : No No SCSI-3 SCSI-3 Persistent Persistent Reserve: Reserve: Disabled Disabled Dynamic : Dynamic Spare Spare Invoked Invoked : No No Dynamic : Dynamic RDF RDF Capability Capability : None None Device : Device Service Service State State : Normal Normal Device : (RW) Device Status Status : Ready Ready (RW) Device : (RW) Device SA SA Status Status : Ready Ready (RW) Front Front Director Director Paths Paths (4): (4): ------------------------------------------------------------------------------------------------------------------------------------POWERPATH PORT LUN POWERPATH DIRECTOR DIRECTOR PORT LUN ----------------- ------------------- ------- --------------- ----------------PdevName Type Type Sts PdevName Type Type Num Num Sts VBUS VBUS TID TID SYMM SYMM Host Host ------------------------------------------------------------------------------------------------------------------------------------------Not N/A FA 07A:0 000 Not Visible Visible N/A FA 07A:0 RW RW 000 00 00 00A 00A N/A N/A Not N/A FA 08A:0 000 Not Visible Visible N/A FA 08A:0 RW RW 000 00 00 00A 00A N/A N/A Not N/A FA 09A:0 000 Not Visible Visible N/A FA 09A:0 RW RW 000 00 00 00A 00A N/A N/A Not Visible N/A FA 10A:0 000 N/A FA 10A:0 RW RW 000 00 00 00A 00A N/A N/A } } Not Visible
This is a continuation of the symdev show command. Here we see the capacity and other configuration information as well as the front-end directors that it is presented to. In this case we see it is presented to four different front end directors. Because the local host is not configured to see the volume, the PdevName shows as Not Visible. The out put is continued on the next page. Beginning with Enginuity 5771, track sizes for FBA devices has been increased from 32K to 64K. To make comparing device sizes easier between Enginuity 56xx and 5771 architectures, SYMCLI will present size information in the Enginuity 56xx architecture format, 32K track size, by default. This default can be changed by enabling (default) or disabling the following value in the options file: SYMAPI_TRACK_SIZE_32K_COMPATIBLE = ENABLE | DISABLE Note: In this example this environment variable was NOT set.
symdev show (cont.)

Mirror Mirror Set Set Type Type Mirror Mirror Set Set DA DA Status Status Mirror Mirror Set Set Inv. Inv. Tracks Tracks : : [RAID-5,RAID-5,N/A,N/A] [RAID-5,RAID-5,N/A,N/A] : : [RW,RW,N/A,N/A] [RW,RW,N/A,N/A] : : [0,0,0,0] [0,0,0,0]
Back Back End End Disk Disk Director Director Information Information { { Hyper Hyper Type Type Hyper Hyper Status Status Disk Disk [Director, [Director, Interface, Interface, TID] TID] Disk Disk Director Director Volume Volume Number Number Hyper Hyper Number Number Hyper Hyper Type Type Hyper Hyper Status Status Disk Disk [Director, [Director, Interface, Interface, TID] TID] Disk Disk Director Director Volume Volume Number Number Hyper Hyper Number Number } }
: : : : : : : : : : : : : : : : : : : :
RAID-5 RAID-5 Ready Ready [N/A,N/A,N/A] [N/A,N/A,N/A] N/A N/A N/A N/A RAID-5 RAID-5 Ready Ready [N/A,N/A,N/A] [N/A,N/A,N/A] N/A N/A N/A N/A
(RW) (RW)
(RW) (RW)
This is a continuation of the symdev show command. Here we are looking at the mirror positions, and invalid tracks. The Back end director information for RAID 5 volumes shows as N/A. The back-end for RAID 5 is shown on the next page.
symdev show (cont.)

RAID-5 RAID-5 Device Device Information Information { { Number : Number of of Tracks Tracks in in a a Stripe Stripe : 4 4 Overall : Overall Ready Ready State State of of RAID-5 RAID-5 Device Device : ReadyNoSpare ReadyNoSpare Overall Overall WriteProtect WriteProtect State State of of RAID-5 RAID-5 Device Device : : EnabledNoSpare EnabledNoSpare Member : Member Number Number of of the the Failing Failing Device Device : None None Mirror : Mirror Number Number of of the the Failing Failing Device Device : None None Mirror : Mirror Number Number of of the the Data Data Device Device : 0 0 Member : Member Number Number that that Invoked Invoked the the Spare Spare : None None Mirror : Mirror Number Number of of the the Spare Spare Member Member : None None Disk : Disk Director Director (DA) (DA) that that Owns Owns the the Spare Spare : None None Copy : Copy Direction Direction : N/A N/A RAID-5 RAID-5 Hyper Hyper Devices Devices (3+1): (3+1): { { Device Device : : 004A 004A { { --------------------------------------------------------------------------------------------------------------------Disk DA Hyper Member Disk Disk DA Hyper Member Spare Spare Disk DA Vol# Num DA :IT :IT Vol# Num Cap(MB) Cap(MB) Num Num Status Status Status Status Grp# Grp# Cap(MB) Cap(MB) --------------------------------------------------------------------------------------------------------------------02A:C1 2 2 1921 2 N/A 3 140014 02A:C1 2 2 1921 2 RW RW N/A 3 140014 16A:Cf 5 1 1921 4 N/A 5 140014 16A:Cf 5 1 1921 4 RW RW N/A 5 140014 01B:Cf 5 1 1921 1 N/A 5 140014 01B:Cf 5 1 1921 1 RW RW N/A 5 140014 15B:C1 2 2 1921 3 N/A 3 140014 15B:C1 2 2 1921 3 RW RW N/A 3 140014
This is a continuation of the symdev show command. Here we are looking at the RAID5 specific configuration Note there are 4 hyper volumes associated with the one RAID5 volume.
Device Status and Service State

y Device status:
Ready Not Ready Not available for host read/write operations Write Disabled
y Can be changed using the symdev command y Device service states:

Normal Failed Degraded one or more mirror positions to the protected device is Not Ready but the device is still available to the host for read/write
y The service state of interest can be easily queried: symdev list service_state notnormal
The symdev list and symdev list pd commands provides options that allows the returned data to be limited to a specific service state (-service_state option) . Service states can be degraded, failed, or normal, or all service states except one by preceding the service state value with a not, such as -service_state notfailed.
Device Groups
y Management and control operations are typically performed on a multiple devices in a single operations
Host Operating System volume group Application Dataset
Device Group
y Two types of Device Groups

Device Groups (DG)
Single Symmetrix array Single RDF (RA) group.
Composite Groups (CG)

One or more locally-attached Symmetrix arrays One or more RDF (RA) groups within a Symmetrix
y By default DG information is stored in a hosts SYMAPI DB file

Group Name Services (GNS) Provides the ability to maintain group definitions in a shared repository located on the Symmetrix
A collection of devices can be assigned to a named group to provide a more manageable object to query status and impart control operations. Groups can be used to identify and work with a subset of available Symmetrix devices, obtain configuration, status, and performance statistics on a collection of related devices, or issue control operations that apply to all devices in the specified device group. You can use device groups to identify and work with a subset of available Symmetrix devices, obtain configuration, status, and performance statistics on a collection of related devices, or issue control operations that apply to all devices in the specified device group. Two types of Device Groups: y Device Groups (DG) User-defined group comprised of devices that belong to a single Symmetrix array and a single RDF (RA) group. A control operation can be performed on the group as a whole, or on the individual device pairs that comprise it. By default, a device cannot belong to more than one device group and all of the STD devices in a group must reside on the same Symmetrix array. If the Symmetrix options file parameter SYMAPI_ALLOW_DEV_IN_MULT_GRPS is enabled, a device can be added to multiple groups. y Composite Groups (CG) User-defined group comprised of devices that can belong to one or more locally-attached Symmetrix arrays One or more RDF (RA) groups within a Symmetrix. An RDF consistency group is a CG comprised of RDF devices enabled for RDF consistency. The RDF consistency group acts in unison to preserve dependent write consistency of a database distributed across multiple SRDF systems. It maintains this consistency via PowerPath or Multisession Consistency (MSC), which respects the logical relationships between dependant I/O cycles.
Device Group Example

y Create a Device Group
symdg create type <Regular|RDF1|RDF2> <DgName>
y Add Device using Symmetrix Device name

symld g <DgName> add dev <SymDevName>
y Add Device using Physical Device Name

symld g <DgName> add pd <PDevName>
y Associate Gatekeepers with Device Group

symgate -g prod associate pd c2t0d2
y Display Device Group

symld g <DgName> list symdg show <DgName>
The symdg and symld commands group Symmetrix devices for status, monitoring and control purposes. A device group can be set up to contain all devices used by a particular host or that are used in a particular application. Device groups, as well as the devices in a device group, are assigned names that facilitate reference in a session. You assign a device group name at the time you create it. The name can have up to 31 characters and must be unique for a given configuration database. When you add a device to a device group, it is given a logical name. This name allows you to refer to the device independently of its physical device name or Symmetrix device name. The name can have up to 31 characters and must be unique within the device group. It is known only within the context of the device group to which the device belongs. Three types of device groups: y REGULAR(non-RDF) y RDF1 (RDF source device y RDF2 (RDF target device). When you create a device group in an RDF configuration, you must specify the type of the device group (RDF1or RDF2). Otherwise, the group type defaults to REGULAR when no type is specified. The following device lists can be maintained in device groups: y SRDF device list y TimeFinder/BCV device list y TimeFinder/Snap virtual device list y TimeFinder/Clone target list y Gatekeeper device list
Module Summary
Key points to remember: y A Symmetrix device is a logical emulation of a physical disk
FBA architecture for Open Systems CKD for mainframe systems
y A Symmetrix device has multiple identifiers that are used depending on the perspective
Host Symmetrix Solutions Enabler
y The DMX supports three data protection schemes:

RAID 1 Mirroring RAID 5 Parity based protection RAID 6 Dual parity protection
y There are multiple ways of approaching the Symmetrix configuration

From the physical disk looking at the logical device configuration From the logical device looking the physical storage
Reference Documentation
y PowerLink
Closing Slide
Module 3: Configuring Symmetrix Devices
Welcome to the third module of Symmetrix Monitoring and Management In this module we will examine the symconfigure command and how it can be used to create new mirrored, RAID5, RAID6, and meta devices.
Configuring Symmetrix Devices - 1
Revision History
Rev Number
1.0
Course Date
August, 2007
Revisions
Complete
Configuring Symmetrix Devices

Objective: y Describe how the symconfigure command is used to perform online configuration changes y Describe the purpose and structure of the command file y Use the appropriate command to determine if there is available free disk space to create new devices y Create new mirrored, RAID 5, and RAID 6 devices y Describe the concept of Meta Devices y Use the symconfigure command to form a meta device and add members y Configure hot spare drives y Query the state of configuration changes
2007 EMC Corporation. All rights reserved. Configuring Symmetrix Devices - 3
Device Configuration Overview

y Symmetrix devices are configured a number of different ways
When the Symmetrix IMPL.bin file is first created a number of devices are initially configured Modify the bin file and perform on-line configuration change from service processor EMC ControlCenter Configuration Manager Symmetrix Management Console Solutions Enabler symconfigure command
Executed from management host Create, convert, modify, and delete nearly all types of Symmetrix devices Map, unmap, and mask devices to front-end ports Add or remove drives to the spare pool Configure BCV and SRDF devices and other systems attributes
While historically, all configuration changes were accomplished by EMC Engineers modifying the IMPL.bin file and performing an off-line/online configuration change, for quite sometime now, configuration changes can be performed by the customer. This capability has greatly reduced the time to provision new capacity and has resulted in greater efficiency as customers can provision the appropriate size, protection, and other attributes when they need it.
symconfigure Architecture
y Configuration commands invoked on a management host actually are executed as procedures on the service processor
Symmetrix Configuration Server Initiates change session Service processor performs configuration change on-line
y Change sessions invokes an external configuration lock, allowing only a single change session at a time y Device locks are also placed on specific devices during configuration operations
Prevents control operations (TF &SRDF) To determine if a lock is currently held symdev list lockn 15
GK
The Symmetrix configuration server engages a configuration lock known as lock 15 or lock F on the Symmetrix during the changes session, effectively blocking others from attempting to change the configuration. The lock is released at the end of the session or if the operator chooses to abort the operation. Information about a lock including how long it has been held can be viewed using the command: symcfg list lock15. Symmetrix device level locks are also used to prevent control operations such as TimeFinder establish or split operations from being performed on a device that is being modified by a configuration session. Device locks are engaged by the SYMAPI during the preview stage. Device locks are retained until the configuration change session has completed and the change has been committed on the Symmetrix. SYMCLI provides the ability to release a lock on the Symmetrix. This is not a recommended procedure and is only useful for locks which have been confirmed as stranded. To release a stranded configuration lock: symcfg lockn 15 release Note: It is rarely necessary to perform this operation as the lock is associated with a change session. Releasing a lock in the middle of an active session will cause problems with the processing of the change.
Architecture Details
Host
ECC SYMCLI SYMAPI
Symmetrix
syscalls
Symmetrix
FA RA
Ethernet SYMWIN Scripts SYMWIN Scripts SYMWIN Service Processor
SIL
FC
RA
Ethernet
SYMWIN Service Processor

The Symmetrix Configuration Manager deploys a a client-server architecture where the SYMAPI on the management host is the client and SYMWIN on the service processor is the server. Note the communication paths also include remote Symmetrix if part of a SRDF environment.
Prerequisites
y Requires Configuration Manager License key
Installed using symlmf
y Verify that the current Symmetrix environment supports host-initiated configuration change
C:\>symconfigure C:\>symconfigure verify verify
A A Configuration Configuration Change Change Verification Verification is is in in progress. progress. Please Please wait... wait... Establishing Establishing a a configuration configuration change change session...............Established. session...............Established. Verifying Verifying configuration...................................Verified. configuration...................................Verified. Terminating Terminating the the configuration configuration change change session..............Done. session..............Done.
The The configuration configuration verification verification session session has has succeeded. succeeded.
Before attempting to perform a configuration change using Solutions Enabler, execute the symconfigure verify command to verify the feature is licensed, there is a communications path to the Symmetrix, the service processor is on-line, and there are no other configuration sessions active.
Configuration Change Session

y Configuration changes progress through three stages
Preview
Verifies syntax and command structure
Preview
Prepare
Runs preview checks and verifies appropriateness of configuration change against current configuration
Prepare
Commit
Performs preview, prepare checks and then executes the change on the Symmetrix
y Stages can be executed individually
Commit
Configuration changes progress through three stages: preview, prepare, commit. While it is possible to execute the commit directly and have it progress through the preview and prepared. It is often appropriate to create the change session, preview and prepare to verify that the change is appropriate and then commit the change later during a scheduled maintenance window where there is less activity on the Symmetrix.
Command Files Define Configuration Changes

y Command file provides specific details of configuration change to be processed y Multiple operations can be defined in a single file
Executed sequentially
Example: changes.cmd
#Sample command file create dev; convert dev; form meta ; map dev ; create spare; delete dev ;
y Example: symconfigure commit file changes.cmd y In Unix, may redirect standard in symconfigure commit <<EOF create dev; EOF
Command files are used to define the operations to be performed during a the configuration session. This file is created using an editor such as vi or notepad. Multiple operations can defined in a single file with each operation delimitated with a semicolon Optionally, for UNIX, you can redirect the command as standard input and not create a command file.
Before Creating New Devices

y Determine the available space on the physical disks
C:\>symdev C:\>symdev list list -da -da all all -space -space Symmetrix Symmetrix ID: ID: 000190102254 000190102254 DA DA ------------------------Ident Ident Int Int TID TID --------- ----- ----01A 01A 01A 01A 01A 01A 01A 01A 01A 01A 01A 01A 16D 16D 16D 16D 16D 16D 16D 16D 16D 16D 16D 16D 16D 16D Total Total C C C C C C C C D D D D C C C C C C D D D D D D D D 0 0 2 2 4 4 6 6 1 1 3 3 2 2 4 4 6 6 1 1 3 3 5 5 7 7 Capacity Capacity (MegaBytes) (MegaBytes) --------------------------------------------------------------------------Total Configured Unconfigured Total Configured Unconfigured ----------------------- ----------------------- ----------------------286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 286102 13558 13558 8874 8874 9510 9510 4652 4652 8874 8874 8874 8874 8874 8874 4652 4652 3763 3763 8874 8874 8874 8874 898 898 8438 8438 272535 272535 277227 277227 276583 276583 281445 281445 277227 277227 277227 277227 277227 277227 281445 281445 282335 282335 277227 277227 277227 277227 285200 285200 277655 277655 Details Details ----------------------Hypers Hypers Forma Forma ----------- --------2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA FBA
----------------------- ----------------------- ----------------------- ----------- --------18310546 531464 17778747 120 18310546 531464 17778747 120 FBA FBA
There are a number of ways to determine the amount of free disk space that is available for use in configuring new devices. symdev list da all space above displays configured and unconfigured (free) disk space on a disk by disk basis as well as a summary. symconfigure list -freespace provides a summary of available disk space.
Creating New Devices

y Command file includes:
create dev count= <number of devices> size= <size in cylinders>
1Cyl = 1920 512-byte blocks or .98MB
Example command file: create dev count=100, size=9000, emulation=FBA, config=2-WAY-Mir, disk_group=1;
emulation=<emulation type>
FBA, Celera FBA, VME512FBA CKD-3380 or CKD-3390
config=<Protection type>
2-WAY-Mir RAID 5
datamember_count-3 or 7
RAID 6
datamember_count=6 or 14
disk_group_number=
Syntax for command file: y Hatch marks # are used for comments y White space is ignored y Each statement must end with a semicolon ; y Commas , are optional y Command file is case insensitive y While you can have multiple statements in a single command file, they must be in the appropriate order
Creating RAID 5 and RAID 6 Devices

y Example command files create dev count=24, size=16000, emulation=CKD-3390, config=RAID5, datamember_count=3, disk_group=1; create dev count=100, size=35000, emulation=FBA, config=RAID6, datamember_count=6, disk_group=1;
RAID 5 and RAID 6 devices are configured in a similar way. Note, you must also specify the number of data members. For example in a RAID 5 3+1 configuration, there are three data members.
symconfigure Example
C:\>symconfigure C:\>symconfigure -file -file new_devices.cmd new_devices.cmd -v -v commit commit Establishing Establishing a a configuration configuration change change session...............Established. session...............Established. Processing Processing symmetrix symmetrix 000190102254 000190102254 { { create create dev dev count=10, count=10, size=9000, size=9000, emulation=FBA, emulation=FBA, config=2-Way config=2-Way Mir, Mir, mvs_ssid=0000, mvs_ssid=0000, disk_group=0; disk_group=0; } } Performing Performing Access Access checks..................................Allowed. checks..................................Allowed. Checking Checking Device Device Reservations..............................Allowed. Reservations..............................Allowed. Submitting Submitting configuration configuration changes..........................Submitted changes..........................Submitted Validating Validating configuration configuration changes..........................Validated. changes..........................Validated. New New symdevs: symdevs: 0041:004A 0041:004A Initiating Initiating PREPARE PREPARE of of configuration configuration changes...............Queued. changes...............Queued. PREPARE PREPARE requesting requesting required required resources.....................Obtained. resources.....................Obtained. Step Step 013 013 of of 017 017 steps.....................................Executing. steps.....................................Executing. Local: Local: PREPARE...........................................Done. PREPARE...........................................Done. Initiating Initiating COMMIT COMMIT of of configuration configuration changes................Queued. changes................Queued. COMMIT COMMIT requesting requesting required required resources......................Obtained. resources......................Obtained. Step Step 019 019 of of 116 116 steps.....................................Executing. steps.....................................Executing. Step Step 209 209 of of 255 255 steps.....................................Executing. steps.....................................Executing. Step Step 209 209 of of 255 255 steps.....................................Executing. steps.....................................Executing. Step Step 212 212 of of 255 255 steps.....................................Executing. steps.....................................Executing. Step Step 212 212 of of 255 255 steps.....................................Executing. steps.....................................Executing. Local: Local: COMMIT............................................Done. COMMIT............................................Done. Terminating Terminating the the configuration configuration change change session..............Done. session..............Done. The The configuration configuration change change session session has has successfully successfully completed. completed.
In the example above, the verbose option was used and the processing is shown in detail. Much of the output was removed to fit it on the slide. The processing details is also logged in the SYMAPI log file.
Meta Devices A.K.A Meta Volumes

y Maximum Symmetrix device size is 65520 Cylinders
Multiple devices can be combined to create Meta devices Presented to host as single device
Capacity Performance
Device 001
Meta Device
Dev 001 Dev 002
Device 002
Device 003
Dev 003 Dev 004
Device 004
Four individual Symmetrix 16 GB logical volumes
Configured as a Meta Volume, appears to the host as a single 64 GB disk

Two or more Symmetrix devices can be grouped to form a meta device configuration and presented to a host as a single disk. Meta devices allow customers to present larger Symmetrix devices than the current maximum hyper volume size (65520 Cylinders). With Meta devices, data is organized either striped or concatenated. Notes: y An Open Systems Meta volume may have a maximum of 255 members y Meta size up to 3.825 Terabyte Larger with RPQ y Members can be of different size y Does not have to be contiguous devices y Members may be mirrored, RAID-5, or RAID-6 y Members of a Meta Volume may also be configured for remote mirrors using SRDF y Dynamic Spares may be invoked for any member of a Meta Volume
Meta Device Data Layout

y Meta Device organization determines how a host accesses information
Concatenated
Meta Head Meta Member Meta Member Meta Tail
Striped
Potential increased performance for sequential writes Default and recommended stripe size is 2 cylinders
Meta Head Meta Member Meta Member Meta Tail
A Meta Device is two or more Symmetrix volumes presented to a host as a single addressable device. It consists of a meta head device, some number of member devices, and a meta tail device. The meta head is the first device in the meta device and receives commands from the host. When a incoming command is processed, Enginuity software determines which meta device should execute the command. With Open Systems, Meta device data is either striped or concatenated. Concatenated devices are organized with the first byte at the beginning of the first device. Addressing continues to the end of the first device before data on the next device is referenced. When writing to a concatenated device, the first meta device member receives all the data until it is full, then data is directed to the next member and so on. Striping divides each meta device into a series of stripes, addressing a stripe from each device before advancing to the next stripe on the first device. When writing to a striped volume, equal size stripes of data from each member are written alternately to each member of the set. Striping benefits sequential writes by avoiding stacking multiple writes to a single spindle and disk director, This scheme creates large volumes, but additionally spreads the workload across multiple disk drives and directors on the back-end and can potentially have very good performance impact.
Creating Meta Devices

y Create member devices as usual
Mirrored RAID-5 RAID-6
Example command file:

form meta from dev 0c9 config=concatenated count=4; ---or--form meta from dev 0DE config=striped stripe_size=2 cyl; add dev 0DF:0E1 to meta 0DE;
y Form the meta head and add members

Two Approaches:
Specify a count Form Meta head and add members as separate operations
Specify config=
Concatenated Striped
Stripe_size
12920 (512) byte blocks 2 Cylinders
You can form or dissolve a meta device, or add or remove members. To initially create a meta device, the head and members are specified in a command file. You can add or remove meta members from an existing concatenated meta devices or add members to a striped meta device. If using the count option, no members need to be specified. Members will automatically be selected from the pool of unmapped devices that match the size, emulation, and configuration type of the meta head. Historically, stripe size can be expressed in blocks or cylinders. Today, the default and only supported value is (1920) 512-byte blocks or 2 Cylinders. Restrictions: y The member device must be unmapped (not visible to a host) y The meta must contain at least one meta member. When a meta device is formed, at least one member must be added. y To create a striped meta device, all members must be the same size and have the same type of protection y Meta devices must be composed of devices that are the same emulation type y Mainframe meta devices are created using the create dev command, not the form meta command. You can also dissolve a meta device which releases all its members making them available as regular devices.
Expanding Meta Devices

y The capacity of a meta device can be expanded while the device is on-line y With concatenated meta device expansion, the new capacity is added to the end
add dev 113 to meta 0CE
y With striped meta device expansion, the data is re-striped across all members
Protected creates a BCV copy to preserve the data add dev 0113:0014 to meta 0CE, protect_data=true, bcv_meta_head=0DE; Non-protected DATA IS LOST protect_date=False
Meta device can be reconfigured to add additional capacity while the device is online and available for host I/O. An administrator can: y Expand both concatenated and striped meta devices y Convert an unused device to a concatenate or striped meta device y Convert a populated device to a concatenated or striped meta device When performing a protected expansion, a BCV_meta must have been previously created and be the same size, configuration, and member count as the original meta device.
Converting Meta Devices

y Convert from:
Concatenated to striped Striped to concatenated Example: convert meta 0c2, config=striped, protect_data=TRUE, bcv_meta_head=113;
y Conversion can be done while the meta device is on-line y For protected operations, the BCV meta device must be identical to the original meta device in capacity, member count, and stripe size
Meta Devices can be reconfigured while the device is online to the host.
Removing Members and Dissolving Meta Devices

y Removing a member from a meta device effectively makes the device smaller to the host
For concatenated meta devices:
Data is preserved up to where the member is removed Host must be able to handle this geometry change
Removing a member of a striped meta device will result in data loss Example: remove dev 113 from meta 0c2;
y Dissolving a meta device

Data is preserved on the member devices but would be inaccessible to the host Example: dissolve meta dev 0ce;
While it is possible to reduce the size of a meta device by removing a member, what we are effectively doing is truncating the device. Many host systems will not be able to handle this type of change.
Managing Spare Pool

y Physical drives can be reserved as hot spares and added to the spare pool
Disk cannot contain any hypers Example: create spare count=2, format=512
y To remove a hot spare and make it available for normal use

Can only remove a hot spare that is currently not invoked To view existing hot spares: symdisk list hotspare Specify the director, Interface, and disk id Example delete spare disk=[16A,D,8]
The spare pool contains disk drives that can be invoked to either temporarily or permanently replace a failing or failed drive.
Managing Configuration Change Sessions

y Configuration changes may take a while to execute
Depending on the activity on the Symmetrix Extent of the changes
y Session status can be monitored

Example: symconfigure query i 10
y Aborting a configuration is possible but not recommended

May not be possible if the session is passes a point of no return Frees configuration session lock Example: symconfigure sid 254 abort
y When config changes complete, update the SYMAPI DB symcfg discover

Depending on the Symmetrix workload and the extend of the changes, Configuration change sessions can take quite some time to complete. Progress caqn be monitored using the query option.
SYMAPI Log
y symconfigure progress is logged in the file: ../emc/SYMAPI/log/symapiyyymmmddd.log
C:/>cd C:/>cd program program files/emc/symapi/log files/emc/symapi/log C:/>More C:/>More symapi2070817.log symapi2070817.log 08/16/2007 2624 3336 08/16/2007 06:32:58.546 06:32:58.546 2624 3336 EMC:SYMCONFIGURE EMC:SYMCONFIGURE iCfgChgSessionStart iCfgChgSessionStart Starting Starting a a local local CfgChg CfgChg session session for for SID SID 000190102254, 000190102254, symapi symapi V6.3-771 V6.3-771 (0.0) (0.0) ucode ucode 5771 5771 08/16/2007 2624 3336 08/16/2007 06:33:02.031 06:33:02.031 2624 3336 srvr srvr (000190102254)...Established. (000190102254)...Established. Establishing Establishing session session with with Local Local cfg cfg
08/16/2007 2624 3336 Verifying 08/16/2007 06:33:08.687 06:33:08.687 2624 3336 Verifying configuration...................................Verified. configuration...................................Verified. 08/16/2007 2624 08/16/2007 06:33:10.921 06:33:10.921 2624 server.............Done. server.............Done. 3336 3336 Terminating Terminating session session with with configuration configuration
08/16/2007 3748 176 08/16/2007 11:25:04.843 11:25:04.843 3748 176 EMC:SRVdaemon EMC:SRVdaemon srvShowReadyMessage srvShowReadyMessage ANR0020I ANR0020I SYMAPI SYMAPI server server listening listening on on port port 2707 2707 over over IPv4 IPv4 only only
Details of a configuration change session is logged in the SYMAPI log.
Module Summary
Key points to remember: y Solutions Enabler provides a CLI that allow most Symmetrix configuration changes to be made on-line y The symconfigure command uses a command file to define the operations to be performed in a session y Three stages of progression when processing a config change:
Preview Prepare Commit
y Meta devices allow the creation of logical devices that are larger the largest Symmetrix device
Additional benefit is potential improved performance.
y PowerLink
Closing Slide
Module 4: Symmetrix Management Console
While this course is focused on using the Solutions Enabler CLI to monitor and manage the Symmetrix, it is appropriate that we briefly discuss the Symmetrix Management Console SMC) which is often described as a GUI front-end to the CLI.
Symmetrix Management Console - 1
Revision History
Rev Number
1.0
Course Date
August, 2007
Revisions
Complete
Copyright 2007 EMC Corporation. All rights reserved. These materials may not be copied without EMC's written consent. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. All other trademarks used herein are the property of their respective owners.
Objectives
Upon completion of this module, you will be able to: y Describe the SMC Architecture y Install and configure the SMC Server y Using a supported web-browser, connect to the SMC console y Use SMC to monitor and manage a Symmetrix
SMC is designed to be self-documenting. Other than the Installation notes, and the on-line help text, there is no other documentation available (or required).
Symmetrix Management Console (SMC)

y Device manager for the Symmetrix
Light-weight, web-based application Intuitive browser interface
y Monitor, configuration and operational control of one or more arrays

Supports all configuration capabilities as Solutions Enabler CLI Real-time reporting No historical data No SAN or Host monitoring and control Manage one Symmetrix at a time
y Bundled with EMC Control Center Symmetrix Manager

2007 EMC Corporation. All rights reserved. Symmetrix Management Console - 4
The Symmetrix Manager Console is a light-weight, web-based graphical user interface for performing Symmetrix monitoring and management tasks. Almost anything you can do with the CLI can be done using SMC GUI. SMC is supported on all Symmetrix starting with Enginuity 5568 and newer. SMC is an independent application that includes a server that runs on a Linux or windows server. The client is a web-browser, and is therefore supported on a wide variety of client systems with network connectivity to the SMC server. SMC GUI features closely match Solutions Enabler in both terminology and the information presented. Even die-hard CLU users will find the filtering features useful and the interface for performing common tasks such as configuring new devices is extremely intuitive.
SMC Management Capabilities

Discovery
y Array and internals config y Logical and physical devices
Configuration
y Device creation, configuration, masking y Supports Solutions Enabler code for Open Systems and Mainframe y Dynamic Cache Partitioning y Symmetrix Priority Controls
Monitoring
y Properties y Status y Alerts
Administration
y User-level security, logging, auditing y Access-control management
Replication
y Configuration and control y Discovery of objects and status y SRDF monitoring y Command Line Generator
Symmetrix Management Console can be used to perform all the typical management and configuration tasks such as device creation and configuration, along with volume masking and front-end port configuration. In addition, SMC provides an easy to use interface for performing local and remote replication setup and control.
Client Server Architecture

y Browser-based client y SMC Server operates within Solutions Enabler environment
Connectivity to the Symmetrix
Gatekeeper
SMC Server SMC Server storapid SYMAPI DB storsrvd storapid SYMAPI DB GK
Shares the SYMAPI DB Feature Licenses

SMC License
y Two deployment options:
SMC Server on the Solutions Enabler host Stand-alone SMC server remotely communicates to Solutions Enabler host using storapisrv daemon
The SMC Client runs on a supported web-browser with network connectivity to the SMC server. y Internet Explorer 6.0+ and Firefox 1.0.5+ are supported y Java RTE The SMC Server y Windows 2000 Server SP4+, Windows 2003 Standard or Enterprise Edition y Linux Redhat 4, 32 bit or SUSE 9, 32 bit y Requires Solutions enabler environment y Requires at least the SMC and Base License keys. Other and key for all SE functionality as appropriate
Installation Prerequisites
y SMC Server requirements
Solutions Enabler V6.2+ must be installed on the same system as the SMC Server
Use symlmf to enter SMC License key The storevntd (SYMAPI Event Daemon) is required for SMC to receive Symmetrix alerts
Server host must have at least 512MB of memory for the SMC application
y Client browser requirements

Javascript must be enabled Pop-up blockers must be disabled
Or set to not block all pop-ups from the SMC server address
Java browser plug-in is required

Version 1.5 or above is highly recommended
Minimum browser resolution is 1024 x 768

SMC is designed to require a light footprint on both the Server and client systems.
Installation Process
1. Click the SMC install executable 2. Select destination locations 3. Enter Super User Name and Connection Type
y y Super User can create initial additional accounts Local Symmetrix connection or Remote using storsrvd
4. Enter http and https ports (default 7070 and 8443) 5. Start Copying Files verification screen 6. Start Symmetrix Management Console Service query 7. Installation Complete screen, click Finish
Installation Process details are available in the Installation Guide, Chapter 1: Installing the Symmetrix Management Console. y Super User Name The initial user that will have rights to do all things in SMC including creating additional user accounts and setting permissions and LDAP-SSL Default is user lowercase "smc" with password lowercase "smc" SECURITY WARNING: After installation has completed, you are strongly advised to change this default password as a security precaution! Non-default user must be an authorized user on the SMC server host y Connection Type Local is the default and requires no additional information Remote provides a means to specify the remote symapisrv location Node Name (or IP address) Net Port, defaults to 2707, the default port for symapisrv start Must match port that symapisrv was started on (4.)
Invoking the Client Browser

y Open browser and point to hostname/IP address of SMC Server
https://<host>:port Example: https://localhost:8443
To start the SMC client, open a web browser, and point to the host name (SMC server) and the port entered (or accepted) during installation y If the installation defaults were accepted and running client local to the SMC Server, enter http://localhost:7070 y If the installation defaults were accepted, but running client remote from the SMC Server, enter http://<host>:7070 y If the port was changed from the default specify the <http_port> instead of 7070 y For a secure connection: https://<host>:<https_port> y The first time the client browser is invoked there may be a lengthened display of a progress bar as the client is loaded. At the Web Console login screen, enter the Super Username and Password y Enter the Super User Username, default smc, or as specified during the installation y Enter the Password, default smc y If a correct Username/Password is entered, the SMC Web Console will appear Reminder: pop-up blockers must be disabled or set to allow all pop-ups from the specified host in order to display the SMC Web Console
SMC Interface
Menu Menu Bar Bar View View Bar Bar
(Properties) (Properties) View View (Details) (Details)
Navigation Tree
Menu Bar y Contains multiple sub-menus: File, Control, Administration and Help y Contains actions: Export, Logout, and Refresh View (also in File menu) y Alert counter if clicked will select Alert View (also in View Bar) Navigation Tree y Contains folders made up of sub-folders and objects y Each folder can be expanded or collapsed y Select a single object for monitoring and control operations Selecting a folder containing multiple objects is the only way to "select" multiple objects in the Navigation Tree View Bar y Toggle View Pane between different views: Properties, Config Session, Alerts, and Command History (Properties) View y Color-coded to match current View with the View Bar, e.g Blue for Properties View, as shown on this slide y Display will depend on Tree Selection, View Bar selection, and Object selection within the View area y View can split horizontally 1 or 2 times to display additional detail
Symmetrix Properties
y Similar to output of CLI commands
symcfg list -v
Note the similarities in the properties view of the Symmetrix with the output of the symcfg list v command
Device Properties
y Similar to output of CLI commands
symdev list & symdev show
The Properties view of devices is similar to the output of symdev list. When you select a specific device and display the properties, it has similar information as the symdev show command.
Create Device
y Creates configuration session
Same parameters as symconfigure
Again, the SMC provides similar options as the CLI but in an intuitive GUI.
Task List
y Configuration session added to task list
Actions: Preview, Commit, Deactivate
Configuration sessions are created and displayed in the Task list for execution when appropriate
Alerts View and Acknowledge
The Alerts View displays a list of alerts for the selected Symmetrix array y Severity: the alerts severity, as defined by SYMAPI. y Object: the object to which the alert is related. y Message: a description of the alert. y Created: timestamp for when the alert happened. y Last Modified: timestamp. y Acknowledged By: a user name. y Category: SYMAPI category. y Code: SYMAPI error code. Alerts will only appear if first enabled via the Administration menu.
Command History View
The Command History View displays a list of actions taken by the users using SMC. y Command History displays for the selected Symmetrix array y Command History information is view-only, there are no actions
Device Groups
y All device groups in the local SYMAPI database or GNS (if active) are available in SMC y Additional Device Groups can be created y Add one device type to the DG at a time y No logical device name support
SMC Device Group creation is a multi-step process y First, the Device Group is created and populated with devices y Using the Create Device Group dialog box, add or remove devices to the group with the Add/Remove function This dialog box can only accept changes to a device group one type at a time. To add multiple types of devices to a device group, find and accept all devices of one type, by clicking OK Open the dialog box again, select a different type, and continue to find devices of that type
SMC Local User Accounts

y The installation dialog includes the specification of the SMC Super User account
Default user "smc" with password "smc"
Recommended to create other Administrators in place of Super User
Can specify different Super User name in installation dialog
y Menu: Administration Local User Accounts

List existing usernames in a table Add new user information (name and password) Edit user information (enter a new password) Delete selected user account(s)
Accounts only apply to SMC and do not require any user credentials outside SMC, with the exception of an initial Super User and default "smc password.
Permissions and Roles

y To Modify a Users permissions:
Menu: Administration Permissions List Symmetrix ID, Username, and Role entries Add new permission entry Edit permission entry (change role) Delete permission entry Administrator - StorageAdmin and SecurityAdmin combined StorageAdmin - full Symmetrix control Monitor - view Symmetrix only, no manipulation of array SecurityAdmin - add users and set permissions, no Symmetrix access None - no access other than login (the default)
y Five Roles
y Super User
Created during installation Only user with permission to set Symmetrix Preferences and LDAP-SSL smc/smc username/password is a potential security hole if unchanged
Permissions must be explicitly added for any newly defined users, default permission is none.
SMC and SYMCLI Differences

y SMC and SYMCLI are very similar in functionality
Both have the same code base and provide the same feature set Similar output and terminology
y Symmetrix configuration is simpler in SMC

Pull down menus and context sensitive actions
y Device filtering simplifies device grouping y SMC allows acknowledgement and tracking of alerts
No similar feature in CLI
y SYMCLI operations default to online mode, always refreshing the latest Symmetrix data unless -offline is specified
SMC views do not automatically refresh when selected in the tree SMC views refresh at the end of a control operation or upon explicitly selecting Refresh View
Potential Common Issues

y Installation Configuration Errors
Incorrect information for remote symapisrv
symapisrv not running at remote site
Incorrect HTTP or HTTPS port assignments Super User name specified cannot be authenticated in Windows
y Pop-up blockers preventing SMC Console from displaying y Firewall blocking ports used for HTTP, HTTPS or symapisrv y SMC not licensed
SMC is a simple product to install and operate, so there are few potential problems. The installation verifies key information, such as the presence of prerequisite software and minimum memory requirements. Error message will clearly point out if one of these are the problems. For example, the message for the symapisrv not detected cannot tell if the host or port assignment is wrong or if symapisrv simply has not been started, but the message should be sufficient to lead the user to make the appropriate checks and to ensure it is running on the correct host and port. Pop-up blockers will prevent the SMC Console from displaying. This setting is controlled in a drop-down menu in your browser utility (e.g. Internet Explorer). y Some pop-up blockers will provide an error message, making the user aware of the issue y Some pop-up blockers tested provide no message whereby the screen just vanishes.
Module Summary
Key points covered in this module: y SMC was designed to be a lightweight device manager y Easy to install and minimal infrastructure requirements y Provides similar functionality as Solutions Enabler
Requires appropriate SE license key
y Local or remote deployment options

SMC Server running on a host that is attached to the Symmetrix SMC Communicates via the storapisrv to host attached to the Symmetrix
y Intuitive browser-based client

Closing Slide
Module 5: Host Connectivity
Welcome to the fourth module of Symmetrix Monitoring and Management In this module we will take a look at what is involved with making Symmetrix devices available to an attached host.
Host Connectivity - 1
Revision History
Rev Number
1.0
Course Date
August, 2007
Revisions
Complete
Host Connectivity
y Upon completion of this module, you will be able to: y Describe requirements for connecting a Open Systems host to a Symmetrix y Using the appropriate host, switch and Solutions Enabler commands, determine if a Fibre Channel fabric and port logins have occurred y Describe concept of port flags, determine how they should be set for a specific host, and use the appropriate Solutions Enabler commands to set them y Make a Symmetrix device visible to a host by setting the channel address using Solutions Enabler y Describe the concept of device masking and how masking is configured on the Symmetrix
2007 EMC Corporation. All rights reserved. Host Connectivity - 3
There is more to connecting a host to a Symmetrix than simply plugging in the cables. This module starts by reviewing the I/O stack and all the components between the host application and the Symmetrix. To fully understand what is happening under the covers, we will spend a bit of time reviewing the Fibre Channel initialization process and then explore in detail what it takes to configure a Symmetrix for host access. Included will be configuring the operational characteristics of a front-end port, device mapping, and device masking.
IO Stack Open Systems

y Host
Operating System Application and/or Database Logical Volume Manager PowerPath HBA
App/Database OS/ LVM PowerPath HBA HBA
Host
y Storage Area Network (SAN)

Switches & Cables Fibre Channel or TCP/IP
FA/SE Adapter Cache
Back-end Directors
SAN
y Symmetrix
Front-end Director
Symmetrix
The diagram above illustrates the IO stack of a typical open systems host. Starting for the top, the host initiates a read or write operation. The I/O operation is passed by the I/O handlers in the operating system to the Logical Volume Management layer. PowerPath is between the Host Bus adapters and the upper layers of the IO stack. It is an optional component but is part of most open systems environments. It performs two functions: load balancing and path failover. While it is possible to connect a HBA directly to a Symmetrix frond-end director port, more likely, you will be connected through a Storage Area Network or a SAN. A SAN consists of one or more interconnected switches. Originally SANs were based on Fibre Channel protocols but today we are starting to see TCP/IP based SANs for applications that dont require the performance and availability of Fibre Channel. SANs provide greater connectivity by allowing more than one host to share the same front-end port on the Symmetrix. When compared to legacy SCSI connectivity which used parallel cables, SANs provide greater distances between the host and the Symmetrix. Depending on the SAN design, distances of 10s of kilometers are possible. This allows consolidation and management efficiency. In the Symmetrix, devices are presented to either a Fibre Channel or iSCSI front-end director and assigned a channel addresses. When more than one host is connected to the same front-end port (most-likely) Volume Logix is configured to restrict which host has access to which specific volumes. The front-end directors were designed to support a large number of SCSI variants, therefore each port must be configured to support a subset of the SCSI and Fibre Channel protocol required for specific hosts.
Connectivity Requirements
y Requirements for connecting an Open Systems host to a Symmetrix:
HBA and device drivers must be installed and configured Path management software (PowerPath, DMP, MPIO, etc) SAN connection between the Host Bus Adapter (HBA) and the Symmetrix front-end director port
Physical cable connection Logical connection zoning
On the Symmetrix:
1.Director flag settings 2.Channel address assignment on multiple Front-end ports
Availability Performance
3.Device masking
There are three things that must be configured when connecting a open systems host to the Symmetrix. HBA, SAN, and Symmetrix. We will look briefly at the HBA and the SAN but the focus of this discussion is the Symmetrix.
Host Perspective
y The typical host configuration includes two or more Host Bus Adapters (HBA)
HBAs are SCSI Initiator devices Unique identifier
Fibre Channel WorldWide Name (WWN) iSCSI Qualified Name (iqn)
y If more than one HBA, PowerPath or other path management software is installed to provides:
Availability Path failover Performance Load balancing
y HBA connect to target devices and discover Logical Units (LUN)

Symmetrix Devices Identified by
LUN Addresses WWN
y Reference Host Connectivity Guides on Powerlink

Typically a host will be configured with two or more HBAs, with each HBA configured to see the same set of devices on the Symmetrix. The Symmetrix has an Active-Active architecture and the same devices may be visible on more than one front-end port. When you add EMC PowerPath or other path management software to coordinate access to the devices, you get a higher level of availability and performance by spreading the workload across multiple HBAs and multiple front-end ports. In a Fibre Channel world all devices are identified by a unique identifier called a WWN. The picture to the left is taken from the Emulex HBAnywhere utility and you can see on the host WIN1, there are two HBAs and each HBA sees four different front-end ports. On each port, seven devices called LUNs are mapped. Again, in environments where the same LUNs are visible on different front-end ports and HBAs, path management software such as PowerPath is required. Reference Host Connectivity Guides on Powerlink. The HBAnyware is an Emulex tool that allows easy verification of the FC environment on a host.
PowerPath
y Monitors path availability and workload y If multiple paths, PowerPath dynamically selects the optimal path y Adjusts for spikes and changes in workload y Transparently redirects IO in the event of a path failure
Without PowerPath
Apps Apps Apps Apps Apps Apps Apps Apps
With PowerPath
Apps Apps Apps Apps
PowerPath
PowerPath
PowerPath
FA
FA
FA
FA
Symmetrix
Symmetrix
PowerPath is an EMC host-resident software product that works with any storage system to deliver intelligent I/O path management, and intelligent I/O balancing. PowerPath software improves performance and availability by intelligently sending I/O down the least busy path. This ensures consistently channel utilization across all paths. PowerPath also automatically detects and restores server-to-storage path failures. Without PowerPath, applications direct I/O to devices as needed. In the illustration, some of the devices have more I/Os in their queues, leading to longer delays on those devices. If devices that have more I/O are crowded on to one HBA, that channel will be utilized more than the others. PowerPath balances I/O over multiple paths to the same device. Host applications write to devices usual and PowerPath drivers intercept the I/O, and directs it to the device through the least busy path.
Fibre Channel Initialization

y Physical cable connection
FA Port and the Switch HBA Ports and the switch
Host
HBA FLOGI HBA FLOGI
y When the HBA and FA ports are initialized, they will log in to the switch
Identifies itself using the World Wide Name (WWN) Updates the FC Name Server Fabric Login (FLOGI)
PLOGI
F-port F-port
Switch
F-port F-port FLOGI FA FLOGI FA
PLOGI
y The Initiator (HBA) will query the switch for a list of Target devices (FA ports) y The HBA logs in to the FA ports
Port Login (PLOGI) Device discovery
Symmetrix
The first step in connectivity is to have a connections from the Host Bus Adapter to the FA port. This requires physical cabling, and a logical path controlled by Fabric Zoning. When the HBA and FA ports are initialized , or when the cables are connected, a Fabric Log In (FLOGI) occurs. This is a standard part of the Fibre Channel protocol where devices identify themselves to the fabric using their World Wide Name (WWN), a unique 128 bit hex identifier. The switch will update the Name Server database and assign the device a Fibre Channel Address. After the Fabric Log In, the HBA will ask the switch for the address of target devices (FA ports) and then attempt to log into each port. This is called a Port Log In or PLOGI. After the HBA has logged into the FA port, it will do a device discovery. Note: The FA port maintains a persistent login tables on the FA ports so you can identify what HBAs have logged in in the past and the current state of the connection.
HBA Status (syminq hba)

y The status of the HBA can be queried using syminq
Port WWN Online state
# # syminq syminq hba hba -fibre -fibre Host Host Name Name : : diamond diamond HBA HBA Type Type : : FibreChannel FibreChannel HBA HBA Name Name : : Emulex-LP8000-1 Emulex-LP8000-1 Vendor Vendor : : Emulex Emulex Corporation Corporation Model Model : : LP8000 LP8000 Serial Serial Number Number : : 0000c9238053 0000c9238053 Firmware Firmware Version Version : : 3.20 3.20 (D2D3.20X3) (D2D3.20X3) Driver Driver Version Version : : 5.00f; 5.00f; HBAAPI HBAAPI v1.3, v1.3, 3-29-02 3-29-02 Host Host WWN WWN : : 20000000c9238053 20000000c9238053 Number Number of of Ports Ports : : 1 1 Port Port WWN WWN : : 10000000c9238053 10000000c9238053 Port Port name name : : /devices/pci@1f,4000/fibre-channel@2 /devices/pci@1f,4000/fibre-channel@2 Port Port type type : : NLPort NLPort Port Port FCID FCID : : 2493971 2493971 Port Port speed speed : : 1gbit 1gbit Supported Supported speed speed : : 1gbit 1gbit Port Port state state : : Online Online
HBA vendors provide tools to view the HBA. In addition, the Solutions Enabler syming command can be used to query the configuration and login status of an HBA. Note that in Fibre channel, all devices are identified by the World Wide Name. Actually there are two World Wide Names; the Node WWN and the Port WWN. The port WWN is what is typically used when configuring Fabric zoning and device masking. In the example above, the Port state of Online indicates that the FLOGI process was successful and the HBA has logged into the switch.
Front-end Director Status

y Determine if the FA port is on-line and a cable connection
Port status Connection status
# # symcfg symcfg -fa -fa all all list list -port -port -sid -sid 097 097 Symmetrix Symmetrix ID: ID: 000190100097 000190100097 S S Y Y M M M M E E T T R R I I X X Ident Ident FA-3A FA-3A FA-4A FA-4A FA-7A FA-7A FA-10A FA-10A FA-13A FA-13A FA-14A FA-14A FA-3B FA-3B Type Type FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel FibreChannel D D I I R R E E C C T T O O R R P P O O R R T T S S Port Connection Port Status Status Connection Status Status P0 P0 P0 P1 P1 P2 P2 P3 P3 P0 P1 P1 P2 P2 P3 P3 ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A X X X X X X -
Director Director Status Status Online Online Online Online Online Online Online Online Online Online Online Online Online Online
Legend Legend for for Connection Connection Status: Status: (X) (X) : : Fibre Fibre Port Port is is Connected Connected to to a a Fibre Fibre Port Port (HBA, (HBA, Switch Switch or or RF RF Director) Director) (-) (-) : : Fibre Fibre Port Port is is Not Not Connected. Connected.
On the Symmetrix, you can also determine the status of the FA port. The port status tells us if the Port is online and the connection status tells us if there is a cable connected. If the Connection status shows an X under a port, host likely it is connected to a switch or directly to an HBA.
Front-end Director Details

y FA Port details
WWN Port and connection status
# # symcfg symcfg -SA -SA 07a 07a -v -v list list ... ... Director Director Identification: Identification: FA-7A FA-7A Director Director Type Type Director Director Status Status Number Number of of Director Director Ports Ports Director Director Ports Ports Status Status Director Director Connection Connection Status Status Director Director Symbolic Symbolic Number Number Director Director Numeric Numeric Number Number Director Director Slot Slot Number Number Director Director Port: Port: 0 0 WWN WWN Node Node Name Name WWN WWN Port Port Name Name Fibre Fibre Channel Channel Loop Loop ID ID Fibre Fibre Adapter Adapter Type Type SCSI SCSI Flags Flags ... ...
: : : : : : : : : : : : : : : : : : : : : : : :
FibreChannel FibreChannel Online Online 2 2 [ON,ON,N/A,N/A] [ON,ON,N/A,N/A] [N/A,N/A,N/A,N/A] [N/A,N/A,N/A,N/A] 07A 07A 7 7 7 7 50060482d52cb306 50060482d52cb306 50060482d52cb306 50060482d52cb306 N/A N/A N/A N/A
The verbose option provides similar information on the port and connection status. Note. If a director is not on-line, you can use the symcfg command to place a port on-line or off-line. Example: symcfg -SA 07a p 0 online
Switch Perspective - Fabric Login (FLOGI)

y Online indicates the HBA and FA Ports have logged into the switch
y switchshow (Brocade)
y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y
switch118:admin> switch118:admin> switchshow switchshow Port Port Media Media Speed Speed State State ========================= ========================= 0 id N2 Online F-Port 0 id N2 Online F-Port 1 id N2 Online F-Port 1 id N2 Online F-Port 2 id N2 Online F-Port 2 id N2 Online F-Port 3 id N2 Online F-Port 3 id N2 Online F-Port 4 id N2 No_Light 4 id N2 No_Light 5 id N2 No_Light 5 id N2 No_Light 6 id N2 No_Light 6 id N2 No_Light 7 id N2 No_Light 7 id N2 No_Light 8 id N2 Online F-Port 8 id N2 Online F-Port 9 id N2 Online F-Port 9 id N2 Online F-Port 10 id N2 Online F-Port 10 id N2 Online F-Port 11 id N2 Online F-Port 11 id N2 Online F-Port 12 id N2 Online F-Port 12 id N2 Online F-Port 13 id N2 Online F-Port 13 id N2 Online F-Port 14 id N2 Online F-Port 14 id N2 Online F-Port 15 id N2 Online F-Port 15 id N2 Online F-Port
50:06:04:61:00:60:02:42 50:06:04:61:00:60:02:42 50:06:04:60:00:60:02:42 50:06:04:60:00:60:02:42 50:06:04:68:00:60:02:42 50:06:04:68:00:60:02:42 50:06:04:69:00:60:02:42 50:06:04:69:00:60:02:42
10:00:01:69:30:60:2e:d2 10:00:01:69:30:60:2e:d2 10:00:01:60:30:60:2e:d2 10:00:01:60:30:60:2e:d2 10:00:01:68:30:60:2e:d2 10:00:01:68:30:60:2e:d2 10:00:01:61:30:60:2e:d2 10:00:01:61:30:60:2e:d2 10:00:01:68:30:60:2f:3b 10:00:01:68:30:60:2f:3b 10:00:01:60:30:60:2f:3b 10:00:01:60:30:60:2f:3b 10:00:01:61:30:60:2f:3b 10:00:01:61:30:60:2f:3b 10:00:01:69:30:60:2f:3b 10:00:01:69:30:60:2f:3b
You can take a number of different approaches to verifying connectivity status; starting at the host and working toward the Symmetrix or starting at the Symmetrix and look toward the host. Another approach is to start in the middle. In Fibre Channel SANs, the switch provides a login service. When a HBA or FA port is connected to the switch, it identifies itself by its WWN, specifies operating parameters such as buffer credits, link speed, and class of services and is assigned a Fibre Channel address. In the example above, we see that both the HBA and FA ports have logged into the switch. Note, the example is for a Brocade switch but other vendors have commands that provide similar information.
Switch Zoning
y Switch zoning determines which FA ports an HBA sees
y zoneshow (Brocade)
y y Switch118:admin> Switch118:admin> zoneshow zoneshow ... ... Effective Effective configuration: configuration: cfg: cfg: zone: zone: Production_cfg Production_cfg HBA0_FA07A_Port0 HBA0_FA07A_Port0 10:00:01:60:30:60:2f:3b 10:00:01:60:30:60:2f:3b 50:06:04:60:00:60:02:42 50:06:04:60:00:60:02:42 HBA0_FA10A_Port0 HBA0_FA10A_Port0 10:00:01:60:30:60:2f:3b 10:00:01:60:30:60:2f:3b 50:06:04:68:00:60:02:42 50:06:04:68:00:60:02:42 HBA1_FA07A_Port0 HBA1_FA07A_Port0 10:00:01:61:30:60:2f:3b 10:00:01:61:30:60:2f:3b 50:06:04:61:00:60:02:42 50:06:04:61:00:60:02:42 HBA1_FA10A_Port0 HBA1_FA10A_Port0 10:00:01:61:30:60:2f:3b 10:00:01:61:30:60:2f:3b 50:06:04:69:00:60:02:42 50:06:04:69:00:60:02:42
zone: zone:
zone: zone:
zone: zone:

Zoning controls access in a Fibre Channel network by allowing or restricting access between HBAs and front-end ports. There are several ways of defining zones, but the most common is using the World Wide Port name which uniquely identifies a device in a Fibre Channel network. In the example above, we see several different zones defined. Each zone includes a single HBA and a FA port. While it is possible to have a big zone with multiple members, our best practice is to configure single initiator zoning which is a single HBA per zone. This approach eliminates unnecessary interactions between devices.
FA Port Login
y HBA > FA Log in
FLOGI
y y # # symmask symmask list list logins logins dir dir 02a 02a p p 1 1 y y Symmetrix Symmetrix ID ID : : 000000006196 000000006196 y y Director Director Identification Identification : : FA-2A FA-2A y y Director Director Port Port : : 1 1 y y Identifier Identifier Type Type User-generated User-generated Node Port Node Name Name Port Name Name FCID FCID Logged Logged On On In Fabric In Fabric
y y ------------------------------- --------- ----------------------------------------------------------------- ----------- ----------- --------y y 10000000c9238053 10000000c9238053 Fibre Fibre diamond diamond 10000000FEEBDAED 10000000FEEBDAED Fibre Fibre diamond diamond i@1f,4000,@2 i@1f,4000,@2 i@1f,4000,@2 i@1f,4000,@2 260e13 260e13 Yes Yes 260e13 260e13 No No Yes Yes No No
If the both the HBA and FA ports have logged into the switch, and the switch is zoned correctly, the HBA should log in to the FA. The identifier field indicates which HBA is communicating with the Symmetrix array. Usergenerated node and port names are identified as the AWWN or AISCSI alias associated with it. Columns labeled On Fabric and Logged In indicate whether the HBA is connected to a fabric and whether it is logged in to the Symmetrix system You can use the verbose (-v) option to view the last active login information.
Symmetrix Connectivity Requirements

y Once there is connectivity between the HBA and the front-end director port, there are three things that need to be configured on the Symmetrix:
0. Assessment of host requirements & planning - Critical! 1. Director port characteristics
y y y y y y y SCSI and Fibre Channel operating parameters Host Operating Systems Specific Assign a channel address to a device on a SP port Logical Unit Number A single FA port can be shared my many hosts Controls access so specific hosts see specific devices Masking information is maintained in Volume Logix Database
2. Device mapping
3. Device masking
There are three things that must be configured on the Symmetrix in order to connect a open systems host: port flags, device mapping, and device masking However one of the most important thing is to have a plan. y What host is going to connect to what port y What operating systems and versions y Number, type, and firmware levels for Host Bus Adapters (HBA) y Is PowerPath or other multi-pathing failover software used y How many, what protection, what size volumes are required y Performance considerations, special Symmetrix features Normally this is done as part of the pre-site survey and is again validated prior to installation. In Engineering labs, you may not have a formal site planning document but you still need a plan.
Front-end Port Characteristics

y Front-end directors can be configured to support a wide variety of host # # symcfg symcfg -SA -SA 07a 07a -v -v list list ... ... operating Director Director Identification: Identification: FA-7A FA-7A systems y Called flags y Parameters for:
Fibre Channel SCSI Gigbit Ethernet (iSCSI)
Director Director Type Type Director Director Status Status Number Number of of Director Director Ports Ports Director Director Ports Ports Status Status Director Director Connection Connection Status Status Director Director Symbolic Symbolic Number Number Director Director Numeric Numeric Number Number Director Director Slot Slot Number Number Director Director Port: Port: 0 0 WWN WWN Node Node Name Name WWN WWN Port Port Name Name Fibre Fibre Channel Channel Loop Loop ID ID Fibre Fibre Adapter Adapter Type Type SCSI SCSI Flags Flags { { Tagged_Commands Tagged_Commands Linked_Commands Linked_Commands Sync_Transfer Sync_Transfer Wide_Transfer Wide_Transfer ... ... : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : FibreChannel FibreChannel Online Online 2 2 [ON,ON,N/A,N/A] [ON,ON,N/A,N/A] [N/A,N/A,N/A,N/A] [N/A,N/A,N/A,N/A] 07A 07A 7 7 7 7 50060482d52cb306 50060482d52cb306 50060482d52cb306 50060482d52cb306 N/A N/A N/A N/A Enabled Enabled Enabled Enabled Enabled Enabled Enabled Enabled
Protocols used to communicate between hosts and storage systems are designed to support a wide variety of applications and operating system requirements. SCSI is a standards-based protocol that has been around for over twenty years and the command set and nexus is flexible enough to support many different types of storage devices and host operating systems. Nearly every server vendor supports SCSI, unfortunately not every vendor implements SCSI in exactly the same way. For example, while both HP-UX and IBM AIX support the SCSI protocol, they support a different subset of the operational parameters. Fibre Channel and IP are transport protocol used with the SCSI protocol and they too has a number of configurable protocol and link parameters. The good news is that the emulation code in the Symmetrix front-end director port is implemented in software and we have the flexibility to configure the front-end port on to support diversity of host configurations. These settings are called port flags.
Setting Front-end Port Flags

y Port operating characteristics, called flags, are configured in Example command file: the binfile using SymmWin on # This is a command file for the Service Processor # setting FA port flags y Also configured using Solutions Enabler symconfigure command y Reference the EMC Support Matrix for specific settings for a host environment
set port 07A:0 SCSI_3=enabled, Volume_Set_Addressing=enabled Unique_WWN=enabled, Auto_Negothiate=enabled; set port 10A:0 copying port 07A:0;
You cannot simply plug a cable into a free FA port and have it work correctly. Various host operating systems have different protocol requirements. Specific port flags are configured in the binfile when the Symmetrix is first configured but can also be configured using the symconfigure command. This allows a storage administrator to dynamically reconfigure the operating characteristics of a port without having to depend on EMC to change the binfile. It is important to understand what the individual flags do before changing them. An incorrect setting can cause errors or performance problems. Reference the EMC Support matrix or the eLab Navigator for specific configuration requirements for each operating system type and configuration.
Heterogeneous Environments
y If there multiple host sharing a front-end port and they have different operational parameter requirements, the port setting can be overwritten using the symmask command
Specify the host operating system Specify the specific flag setting (SE 6.4)
# # symmask symmask -wwn -wwn 1234567891234567 1234567891234567 set set heterogeneous heterogeneous on on IBM_AIX_DMP IBM_AIX_DMP -dir -dir 7a 7a -p -p 0 0
# # symmask symmask -wwn -wwn 1234567891234567 1234567891234567 set set hba_flags hba_flags on on C,SC3 C,SC3 dir dir 7a 7a -p -p 0 0
Setting the flags for a specific FA port is a good approach if all host that share the port have the same requirements. If you have a heterogeneous environment and different hosts have different requirements, the flag requirements can be set in the Volume Logix database and they are enforced on a connection basis. Two ways to do this: By setting the Operating system type or setting the specific flags that will override the settings on the port. Setting the specific flags is the preferred way starting with Solutions Enabler 6.4 Volume Logix and the VCM database are discussed later in this module. Note: Setting the heterogeneous host configuration has been superseded by Setting the HBA port flags. The heterogeneous host configuration types continue to be valid, but will not be expanded. To switch to setting HBA port flags, the heterogeneous host configuration must be disabled on the array, and all flags must be reset.
E-Lab Configuration Information
The E-lab Navigator is accessible from the Powerlink and/or other EMC websites. Host specific director flag settings information can be found here. The support matrix refers to these flags as bit settings as they were originally set by toggling specific bits
Director Bit Information Table
This is an excerpt from the full Director Bit Information table. The table includes all operating systems, both clustered and non-clustered, and all supported Enginuity levels. There are also references to extensive footnotes.
Mapping a Device to a Port

y Symmetrix Devices are made available on a FA port by assigning them a channel address
Referred to as mapping Configure in the binfile but also can be configured using Solutions Enabler
Host
HBA HBA
C#
y Typically the same devices are mapped to two or more FA ports

Availability Load balancing Requires host-based path management software
FA FA
T#
03
00 04
01
02 FF
D#
y The channel address is used by an attached host access the device

Often reflected in the c#t#d# device naming convention Example channel address 0003 may be seen as c1t0d3
Symmetrix
A Symmetrix can have over 64000 devices configured. Not all devices are accessed by every front-end port. Instead, specific devices are mapped to specific ports by assigning a channel address. Host systems discover and access Symmetrix devices using these Channel Addresses. For open systems hosts, the Channel address is the SCSI ID. Normally a host uses a combination of the Controller, Target, and Logical Unit Number to address a disk device. The Controller number is the Host Bus Adapter, the Target is the port on the Storage System and the Logical Unit Number is the Channel Address we assign.
Mapping Requirements
1. Identify available devices
Devices that are currently not mapped to other hosts Normally only a single host will have access to device Exception would be clustered environments
2. Identify available channel address

Valid addresses: 000-FFF (Maximum of 4096 addressable devices per port)
Not all addresses may be valid for a host
Address must be unique on a port
Before performing a mapping operation, some research and planning is required. First what is a valid address range for a host. Secondly, what addresses are currently available. Possible host operating system addressing issues: y Some hosts only support addresses 00-FF y Some hosts require range of addresses to start with 00 y Some hosts do not allow holes in address range Example devices 00-10, 20-2F Later in this module we will discuss an alternative way of assigning addresses that can allow multiple hosts to use the same address for different devices on the same FA port.
Mapping - Identify Available Devices

C:\>symdev C:\>symdev list list -noport -noport Symmetrix Symmetrix ID: ID: 000190102254 000190102254 Device Directors Device Device Name Name Directors Device ----------------------------------------------- ------------------------- ------------------------------------------------------------------------Cap Cap Sym SA Attribute Sts (MB) Sym Physical Physical SA :P :P DA DA :IT :IT Config Config Attribute Sts (MB) ----------------------------------------------- ------------------------- ------------------------------------------------------------------------002C ???:? N/Grp'd RW 8438 002C Not Not Visible Visible ???:? 16C:D6 16C:D6 2-Way 2-Way Mir Mir N/Grp'd RW 8438 002D ???:? N/Grp'd RW 8438 002D Not Not Visible Visible ???:? 16A:D6 16A:D6 2-Way 2-Way Mir Mir N/Grp'd RW 8438 002E ???:? N/Grp'd RW 8438 002E Not Not Visible Visible ???:? 16B:D7 16B:D7 2-Way 2-Way Mir Mir N/Grp'd RW 8438 002F ???:? N/Grp'd RW 8438 002F Not Not Visible Visible ???:? 16D:D7 16D:D7 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0030 ???:? N/Grp'd RW 8438 0030 Not Not Visible Visible ???:? 16C:C1 16C:C1 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0031 ???:? N/Grp'd RW 8438 0031 Not Not Visible Visible ???:? 16D:C0 16D:C0 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0032 ???:? N/Grp'd RW 8438 0032 Not Not Visible Visible ???:? 01B:D0 01B:D0 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0033 ???:? N/Grp'd RW 8438 0033 Not Not Visible Visible ???:? 01D:D0 01D:D0 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0034 ???:? N/Grp'd RW 8438 0034 Not Not Visible Visible ???:? 16B:C0 16B:C0 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0035 ???:? N/Grp'd RW 8438 0035 Not Not Visible Visible ???:? 16A:C1 16A:C1 2-Way 2-Way Mir Mir N/Grp'd RW 8438 0036 ???:? N/Grp'd 45000 0036 Not Not Visible Visible ???:? 01A:C2 01A:C2 RAID-5 RAID-5 N/Grp'd (M) (M) RW RW 45000 003A ???:? N/Grp'd RW 11250 003A Not Not Visible Visible ???:? 01A:C4 01A:C4 RAID-5 RAID-5 N/Grp'd RW 11250 003B ???:? N/Grp'd RW 11250 003B Not Not Visible Visible ???:? 16A:D4 16A:D4 RAID-5 RAID-5 N/Grp'd RW 11250 003C ???:? N/Grp'd RW 11250 003C Not Not Visible Visible ???:? 16A:C5 16A:C5 RAID-5 RAID-5 N/Grp'd RW 11250
Identifying unmapped devices can be easily done using the symdev list noport command which identifies devices that are not currently assigned to any port.
Mapping - Identify Available Channel Address

C:\>symcfg C:\>symcfg list list -FA -FA 8a 8a -address -address -available -available Symmetrix Symmetrix ID: ID: 000190102254 000190102254 (Local) (Local) Director Device Attr Address Director Device Name Name Attr Address ------------------------------------------- --------------------------------------------------------- ------- --------------------------Ident Physical VBUS Ident Symbolic Symbolic Port Port Sym Sym Physical VBUS TID TID LUN LUN ------------ --------------- ------- ------- ------------------------------------------------ ----- ----FA-8A 08A 0 AVAILABLE 0 00 FA-8A 08A 0 AVAILABLE 0 00 000* 000* 0 00 0020 0 00 050 050 0020 \\.\PHYSICALDRIVE1 \\.\PHYSICALDRIVE1 0 00 0021 0 00 051 051 0021 \\.\PHYSICALDRIVE2 \\.\PHYSICALDRIVE2 0022 0 00 0022 \\.\PHYSICALDRIVE3 \\.\PHYSICALDRIVE3 0 00 052 052 0 00 0023 0 00 053 053 0023 \\.\PHYSICALDRIVE4 \\.\PHYSICALDRIVE4 0024 0 00 0024 \\.\PHYSICALDRIVE5 \\.\PHYSICALDRIVE5 0 00 054 054 0025 0 00 0025 \\.\PHYSICALDRIVE6 \\.\PHYSICALDRIVE6 0 00 055 055 0026 0 00 0026 \\.\PHYSICALDRIVE7 \\.\PHYSICALDRIVE7 0 00 056 056 0027 0 00 0027 \\.\PHYSICALDRIVE8 \\.\PHYSICALDRIVE8 0 00 057 057 0028 0 00 0028 \\.\PHYSICALDRIVE9 \\.\PHYSICALDRIVE9 0 00 058 058 0029 0 00 0029 \\.\PHYSICALDRIVE10 \\.\PHYSICALDRIVE10 0 00 059 059 AVAILABLE 0 00 AVAILABLE 0 00 05A* 05A* Total ---Total ---Mapped 10 Mapped Devices: Devices: 10 Including 10 Including Metamembers: Metamembers: 10 Available 4086 Available Addresses: Addresses: 4086 (s) (s)
Before performing a mapping operation, some research and planning is required. First what is a valid address range for a host. Secondly, what addresses are currently available. (*)indicates a gap in the address assignments or are the next available address in the run . In this example address 000 -4F and 05A and above are available addresses.
Mapping a Device to a Port

y Configure in the binfile but also configured using Solutions Enabler symconfigure command y In most environments, the same devices are mapped to more than one FA port
Availability Performance
Example command file:

# This is a command file for # mapping a device to a port
y Example: symconfigure -file mycmdfile commit y Also possible to perform masking in same operation
map dev 00cd to dir 14A:0 target=0, lun=7; map dev 00cd to dir 3A:0 target=0, lun=7; map dev 00e0:00e7 to dir 14A:0 starting target= 1, lun=0; map dev 00e0:00e7 to dir 3A:0 starting target= 1, lun=0
Mapping can be performed when the bin files is created or can be modified by editing the binfile and doing an on-line configuration change but it is often easier to perform mapping using the symconfigure command. When configuring mapping, you specify the LUN address and optionally the target address. You can assign a specific address to specific LUNS or specify a range of devices and a starting address. It is a best practice to specify the same channel address when configuring a LUN on multiple FA ports.
Displaying Device Mapping

y Device address from the device perspective
C:\>symdev C:\>symdev show show 21 21 Symmetrix Symmetrix ID: ID: 000190102254 000190102254 Device Device Physical Physical Name Name Device Device Symmetrix Symmetrix Name Name Front Front Director Director Paths Paths (2): (2): { { ------------------------------------------------------------------------------------------------------------------------------------------POWERPATH PORT LUN POWERPATH DIRECTOR DIRECTOR PORT LUN ------------------- ------------------- ------- --------------- ----------------PdevName Type Type Sts PdevName Type Type Num Num Sts VBUS VBUS TID TID SYMM SYMM Host Host ------------------------------------------------------------------------------------------------------------------------------------------\\.\PHYSICALDRIVE2 N/A FA 08A:0 000 \\.\PHYSICALDRIVE2 N/A FA 08A:0 RW RW 000 00 00 051 051 000 000 Not N/A FA 09A:0 000 Not Visible Visible N/A FA 09A:0 RW RW 000 00 00 051 051 N/A N/A } }
: : \\.\PHYSICALDRIVE2 \\.\PHYSICALDRIVE2 : : 0021 0021
The symdev show command can be used to determine what devices are mapped to what port. In most environments, high availability is important and mapping the same device to multiple ports is a requirement.
Host Addressing Considerations

y An FA port supports a maximum Host A Host B Host C of 4096 device addresses y Typically, multiple host systems share the same port y Masking will effectively allow exclusive assess to a subset of devices
00 HBA HBA HBA HBA HBA HBA
FA
FA
01 04
02 FF
03
y May be desirable to have each host address range start with 00 y Solution: Configuring Dynamic LUN Addressing
Symmetrix
The channel address assigned to a logical is used by the host when it discovers and accesses a Symmetrix Logical Volume. Typically a disk is addresses through a Controller instance, a Target device, and Device on that target. This is often represented as a c#t#d#. For example /dev/rdsk/c1t0d3 references Host Bus Adapter 1, Target 0 (first port accesses through the HBA), and device 3 (channel address 3). LUN offset is an enhanced visibility feature that allows any host type to adjust host visibility by offsetting (renumbering) LUN addresses. This is useful for host types that need to see LUN 0000 or transform a noncontiguous LUN sequence to a contiguous sequence. In a case where two hosts access the same Symmetrix director port and need to see a LUN 0000 but not the same device, you can use LUN offset so that one host sees the devices mapped from LUN x as starting from LUN 0000, and the other host sees devices from LUN y as starting from LUN 0000. To account for noncontiguous device LUN addresses, specify a LUN base and offset as hexadecimal values to adjust for the break in the LUN sequence. The base hex value represents the first LUN in a renumbered LUN sequence. The offset hex value added to the base value determines where to begin renumbering. For example, if a host needs to detect LUN 0000 but you want your host to detect only LUNs 0005 through 0008, you can specify a LUN base address of 0000 and an offset of 0005 The use of the Solutions Enabler symmask command is discussed later in this module but below is an example of a command to renumbers LUNs 0005 through 0008 as LUNs 0000 through 0003: : symmask set lunoffset on 0005 0000 dir 16A -p 0 wwn 10000000c920b484
Device Masking
Host Host y Device masking A B allows multiple hosts to effectively share the same front-end ports
HBA HBA HBA HBA
Host C
HBA HBA
Host D
HBA HBA
Host X
HBA HBA
Host Y
HBA HBA
Host Z
HBA HBA
FA port can see up to 256 HBAs
Switch
y Restrict access to specific host and/or host clusters y Implemented in the Symmetrix with Volume Logix
Fibre Channel iSCSI
FA or SE
p0 p1
VCMDB
FA or SE
p0 p1
Symmetrix
Storage Area Networks provide a fan-out capability were it is likely that more than one host is connected to the same Symmetrix port. The actual number of HBAs that can be configured to a single port is operating system and configuration dependent but fan-out ratios as high as 256:1 are currently supported. Reference the support matrix for specific configuration limitations. Each port may have as many as 4096 addressable volumes presented. When several hosts connect to a single Symmetrix port, an access control conflict can occur because all hosts have the potential to discover and use the same storage devices. However, by creating entries in the Symmetrixs device masking database (VCMDB), you can control which host sees which volume. Device Masking is independent from zoning but they are typically used together in an environment. Zoning provides access control at the port level and restricts which host bus adapter sees which port on the storage system and device masking restricts which host sees which specific volumes presented on a port. With Fibre Channel, Device Masking uses the UWWN (Unique Worldwide Name) of Host Bus Adapters and a VCM database device. In iSCSI, the iSCSI Qualified Name (IQN) is used. Regardless of the protocol, the concepts are the same. The device-masking database (VCMDB) on each Symmetrix unit specifies the devices that a particular WWN or IQN can access through a specific Fibre port.
Device Masking
y HBA-to-FA connection records are created and maintained in the Device Masking Database (VCMDB)
HBA0 WWN -> FA03a:0 - dev 000-010 HBA0 WWN -> FA14a:0 - dev 000-010 HBA1 WWN -> FA03a:0 - dev 000-010 HBA1 WWN -> FA14a:0 - dev 000-010
FA3a:0 FA14a:0
Host A
HBA0 HBA1 WWN WWN
y Entries in the VCMDB define relationship between masked connections and devices
FA consults VCMDB to resolve access rights
00
01
02
VCMDB
y Same approach for both FC and iSCSI

Symmetrix
Device Masking controls host access to a set of devices by maintaining a set of entries in the VCMDB on the array that defines the relationship between masked connections and devices. These entries are sometimes called initiator records. Each entry includes a host's HBA identity (WWN or iSCSI Qualified Name), its associated FA port, and a range of devices mapped to the FA port that should be visible only to the corresponding HBA. Once you make this VCMDB entry and activate the configuration, the Symmetrix makes visible to a host those devices that the VCMDB indicates are available to that host's initiator through that FA port. Volume Logix is the brand name for the software in the Symmetrix that performs the device masking function. The capability is built into Enginuity but its use is optional.
Volume Logix Database - VCMDB

y Database types are Symmetrix model specific and sized to handle the maximum number of logical volumes
Originally a separate Symmetrix Logical Volume Starting with the DMX-3, it is stored in the Symmetrix File System
y VCM DB is maintained using Solutions Enabler symmask and symmaskdb commands y Solutions Enabler accesses the VCM database using a VCM gatekeeper device
VCM gatekeeper is host accessible The SFS is not directly accessible VCM GK only needs to be mapped to the management host
Management Host
VCMDB
Symmetrix File System
VCM VCM GK GK
The Volume Logix Database persistently maintains the device masking information. Originally the database was located directly on a Symmetrix Logical Volume. On DMX-3 it is maintained in the Symmetrix File System (SFS). Rather than create the actual VCMDB device, today we create a VCM Gatekeeper device which is used by the Solutions Enabler to access the database on the SFS, as the SFS volumes are not host addressable. The VCM Gatekeeper is a 6-cyl device. Solutions Enabler 5.3+ is required for DMX-3 running 5771 code. By default, the device masking VCMDB is accessible to all HBAs that log into the director port where the database is configured. Thus, any host with access privileges can effectively modify the contents of the database if it has device masking commands are installed. Beginning with Enginuity Version 5670, the VCMDB can be unmapped from any director that is not being used for masking control.
Setting up Device Masking

y There are four steps in setting up device masking on the Symmetrix
1. Configure a VCM gatekeeper device 2. Set the VCM attribute for the device 3. Set the VCM flag for front-end director port 4. Assign the VCM gatekeeper to front-end port
Typically performed when the bin file is initially created
After the VCM is setup, the database is maintained using the SE symmask and symmaskdb commands
Add and remove masking entries Initialize, query, backup, and restore database May also be configured with EMC ControlCenter or SMC External configuration locks are used to coordinate updates
To set up device masking, you must create a database volume (VCMDB), and set flags on the Fibre Channel or iSCSI ports to enable the uses. Once the database is setup and enabled, the Solutions Enabler symmask command can be used to configure entries granting specific hosts access to specific volumes. During the execution of the symmask or symmaskdb commands, the SYMCLI sets a Symmetrix External Lock on the Symmetrix where the device masking database (VCMDB) resides. This lock ensures that only one host can make changes to the database at any one point in time. If during the processing of a symmask or symmaskdb command, the host fails, or a Ctrl/C is performed in the middle of the command, the lock might not release and could lock out further needed changes or control actions. If a device masking command is interrupted and the lock is not released, future invocations of a device masking command will display the following error message: The operation failed because another process has an exclusive lock on the local Symmetrix. To further examine the presence of this lock, use the following form: symcfg -sid SymmID list -lock -lockn ALL The command will list Symmetrix external locks being held.. To release this lock, use the following form: symcfg -sid SymmID -lockn # release
Discovery
y The symmask discover hba command
Verifies VCM is enabled for port Identifies paths to the VCMDB Assigns alias names to the HBAs AWWN
C:\>symmask C:\>symmask discover discover hba hba FA3a:0 FA14a:0
Host A
HBA0 HBA1 WWN WWN
Symmetrix Symmetrix ID ID
: : 000190102254 000190102254
VCM GK
Device Device Masking Masking Status Status : : Success Success
VCMDB
Identifier Identifier ------------------------------10000000c93da44a 10000000c93da44a 10000000c93da45c 10000000c93da45c
Type Type --------Fibre Fibre Fibre Fibre
User-generated User-generated Name Name ------------------------------------------------------------W2K3-39-106/10000000c93da44a W2K3-39-106/10000000c93da44a W2K3-39-106/10000000c93da45c W2K3-39-106/10000000c93da45c
00
01
02
Symmetrix
The symmask discover command can be run on the management host and or other attached hosts. The symmask discover identifies paths to the device masking database (VCMDB) and assigns alias names (AWWN/AISCSI) to the HBAs residing on the host on where the command was executed. When the symmask discover finds a host HBA, it reads the login history table and performs the following: 1) Checks whether an alias exists in the device masking VCMDB. If one does, this command writes it to the login history table. If there is no alias in the device masking VCMDB record, or the login history table, it creates an ASCII alias and writes it to the login history table. 2) Outputs the initiator identifier (WWN/iSCSI) of the HBAs that are connected to the masked channel and Symmetrix array. Alias names can be used in the command line, replacing the cumbersome numeric identifiers. These names, which are stored in the Symmetrix arrays login history table, identify the HBAs connected to the network interface. Alias names can be shorter in length and much more recognizable than the cryptic WWNs/iSCSIs. ASCII alias names generated by the discover action consists of two parts: the name of the host and the name of the HBA. For example: the AWWN for a host whose TCP/IP hostname is john4554b, on adapter 10000000c920cf87, would be john4554b/10000000c920cf87.
Initializing and Formatting the VCMDB Database

# # symmaskdb symmaskdb list list database database The The Symmetrix Symmetrix device device masking masking database database has has not not been been initialized initialized
C:>symmaskdb C:>symmaskdb init init -file -file VCMdbBackup070707 VCMdbBackup070707 Initialize Initialize Symmetrix Symmetrix SymMask SymMask database database on on Symmetrix Symmetrix 000190100172 000190100172 (y/[n])? (y/[n])? y y Symmetrix Symmetrix SymMask SymMask database database on on Symmetrix Symmetrix 000190100172 000190100172 initialized initialized
During the initial setup of any device masking environment, you must initialize the database. In the process of formatting the VCMDB, the current data is cleared. In most cases, you do not want to clear the data of an existing VCMDB. If you are unsure whether a VCMDB currently exists, issue the following command:: symmaskdb list database To initialize and clear the VCMDB database, you must specify a backup file name to safeguard against clearing data in the database that should not be lost. For example, the following command creates a file called MyInitBackup and attempts to write any current data to it prior to initializing and formatting the VCMDB: symmaskdb init file MyInitBackup Note: The Solutions Enabler device masking function requires a license key. This is installed using the symlmf command.
Adding HBA Access to Symmetrix Devices

C:> C:> symmask symmask list list hba hba Identifier Type Identifier Type ------------------------------- --------10000000c9274156 Fibre 10000000c9274156 Fibre Adapter Adapter ------------------------------i@1f,4000,@2 i@1f,4000,@2 Physical Physical Device Device Path Path ----------------------------------------------Physicaldrive0 Physicaldrive0 Physicaldrive1 Physicaldrive1 Physicaldrive2 Physicaldrive2 Physicaldrive3 Physicaldrive3 C:> C:> symcfg symcfg list list -FA -FA 7a 7a -addr -addr Symmetrix Symmetrix ID: ID: 000190100172 000190100172 (Local) (Local) Director Device Attr Address Director Device Name Name Attr Address ------------------------------------------- --------------------------------------------------------- ------- --------------------------Ident Ident ----------FA-7A FA-7A Symbolic Symbolic --------------07A 07A Port Port Sym Sym ------- ------0 0 0040 0040 0041 0041 0042 0042 0043 0043 0044 0044 Physical Physical --------------------------------------------Not Not Not Not Not Not Not Not Not Not Visible Visible Visible Visible Visible Visible Visible Visible Visible Visible VBUS VBUS ------0 0 0 0 0 0 0 0 0 0 TID TID ----00 00 00 00 00 00 00 00 00 00 LUN LUN ----000 000 001 001 002 002 003 003 004 004 Dir:P Dir:P --------7A:0 7A:0 7A:0 7A:0 8A:0 8A:0 8A:0 8A:0
10000000c92741a1 10000000c92741a1 Fibre Fibre i@1f,4000,@4 i@1f,4000,@4
When configuring device masking, there are three pieces of information that are needed. First, you need to know the World Wide Port Number for the Host Bus Adapter(s). Second, you need to know how the HBA is connected to the Symmetrix; specifically you need to know the frontend director number and port. Finally, you need to know what Symmetrix devices that are mapped to the port. Available devices are those that have a channel address assigned. There are a number of ways to identify the WWN of a HBA. One way is to use the symmask list hba command. Not only will this display the WWN for each HBA in the host, it also displays the director and port of the connection to the Symmetrix and the device file for the gatekeeper device. In the example above we see the host actually has two HBAs: one with the WWN of 10000000c9274156, and the other 10000000c92741a1.We also see that one is connected through director 7a port0 and the other through 8a por0. To get a list of Symmetrix Logical Volumes on a port you could use the Solutions Enabler symcfg list command as shown in the example above. He we see there are a number of volumes presented to Director7A Port 0 starting with Symmetrix Logical Volume number 0040.
Adding HBA Access to Symmetrix Devices

C:> C:> symmask symmask add add dev dev 40,41,42 40,41,42 -wwn -wwn 10000000c9274156 10000000c9274156 -dir -dir 7a 7a -p -p 0 0 C:> C:> symmask symmask add add dev dev 40,41,42 40,41,42 -wwn -wwn 10000000c92741a1 10000000c92741a1 -dir -dir 8a 8a -p -p 0 0
C:> C:> symmask symmask refresh refresh
Refresh Refresh Symmetrix Symmetrix FA FA directors directors with with contents contents of of SymMask SymMask database database 000190100172 000190100172 (y/[n]) (y/[n]) ? ? y y
Symmetrix Symmetrix FA FA directors directors updated updated with with contents contents of of SymMask SymMask Database Database 000190100172 000190100172 C:> C:>
To make an entry for the HBA-to-FA connection in the VCMDB and specifying devices that the HBA can access, use the symmask command shown above. On the first line we are specifying that volumes 0040, 0041, and 0042 are accessible to the first HBA through FA 7A port0. The second command enables access to the same volumes through the other HBA and the other Symmetrix port. After making changes to the VCM database, you must tell the Symmetrix to refresh the access control tables in the director. This is done using the symmask refresh command. Note: In addition to explicitly listing the devices to be added. If you are using devices that have been reserved, you must supply the device reservation ID. For example, to add reserved device 0014 for access to Host3b using director 16a, port 0, enter: symmask -wwn 1234567890123456 add dev 0014 -dir 16a -p 0 -reserve_id 5
You can also use a previously defined device group to specify the devices to mask to a host. For example: symmask wwn 1234567890123456 add -g prodB -std -dir 16a -p 0
Dynamic Addressing
y When devices are mapped to frontend ports, a channel address is assigned y The LUN address can be specified in the connection record overriding the mapped address y Allows each host sharing a port to used the same addresses to access different devices
00 FA FA
Host A Host B Host C

HBA HBA HBA HBA HBA HBA
01 04
02 FF
03
y Example: symmask add devs 15,18,20 -lun 0 -wwn 20000000c920b484 -dir 14C -p 1
Symmetrix
When adding devices you can specify the starting LUN address for each device using the -lun option. Or have SYMAPI assign the LUN address using the -dynamic_lun option. -lun = Specifies starting LUN addresses. You can specify a single starting LUN or multiple LUNs to match the given ranges. For example: symmask add devs 15,18 -lun 0 -wwn 20000000c920b484 -dir 4C -p 1 -dynamic_lun = Specifies the use of dynamic LUN addressing. The application assigns the addresses based on what is already in use for the host HBA.symmask add devs 2C,2E,30 -dynamic_lun -wwn 20000000c920b484 -dir 2a -p 1
Displaying the Contents of the VCMDB

C:> C:> cd cd \Program \Program Files\EMC\SYMCLI\bin Files\EMC\SYMCLI\bin C:> C:> symmaskdb symmaskdb list list database database
Symmetrix Symmetrix ID ID
: : 000190100172 000190100172
Database Database Type Type Last Last updated updated at at
: : Type6 Type6 : : 01:40:29 01:40:29 PM PM on on Thu Thu Sep Sep 01,2007 01,2007
Director Director Identification Identification : : FA-7A FA-7A Director Director Port Port : : 0 0
User-generated User-generated Identifier Identifier ------------------------------10000000c92741a1 10000000c92741a1 Type Type --------Fibre Fibre Node Node Name Name Port Port Name Name Devices Devices ----------------0040:0042 0040:0042
----------------------------------------------------------------10000000c92741a1 10000000c92741a1 10000000c92741a1 10000000c92741a1
You can display the entire contents of the VCMDB or use options to restrict the display to your area of interest. In the example above we are displaying access control records for the entries we previously added. Note: the entire output is not displayed. You can restrict the output to a specific HBA. For example: symmaskdb -list devs -wwn 10000000c9238053 You can also view which HBAs have been assigned to specific devices. For example: symmaskdb list assignment -dev 0040:0043
VCM Database Maintenance

y Access control is tied to the WWN of an HBA y If a HBA fails and must be replaced, the VCMDB must be updated to reflect the change
Example: symmask wwn 20000000c920b484 replace 20000000c920b393 Updates all masking records
y Backup the VCM database

Periodically and before making changes Example: symmaskdb sid 123 file VCM_backup073007 backup
Because device masking is tied to the WWN of an HBA, if it must be replaced, the VCMDB bust e updated to reflect the new WWN.
Module Summary
Key points covered in this module: y Making a Symmetrix Device available to a host involves simply connecting a cable!
Host
Install HBA and Drivers Path management software
Switch
Zoning
Symmetrix
Port flags Device mapping Device masking
y Planning is critical!
Capacity, availability, & performance
Connecting a host to the Symmetrix involves more than connecting a cable between the host and the Symmetrix. On the host, it is necessary to physically install a supported HBA and load the appropriate device driver. The authorative list of what we support can be The EMC Support Matrix.. In most environments, a switch will be used. This is what provides the fan-out capability and allows a single FA port to be shared among many HBAs. The switch is zoned to allow or restrict access between specific HBAs and specific FA ports. Zoning is normally done using Fibre Channel World Wide Names. On the Symmetrix, Three things need to be configured; port flags, mapping, and masking. Port flags define th4 SCSI and Fibre channel operating characteristics. Mapping assigns channel addresses to devices on FA ports. Masking grants access for specific devices to specific HBAs. While the focus of this module has been on Fibre Channel connectivity in a Open Systems environment, the same concepts apply to iSCSI environments as well.
y PowerLink
Closing Slide
Module 6: Monitoring Symmetrix Activity
In this module we will take a look at the Solutions Enabler commands that allow you to monitor performance activity on the Symmetrix.
Monitoring Symmetrix Activity - 1
Revision History
Rev Number
1.0
Course Date
August, 2007
Revisions
Complete
Monitoring Symmetrix Activity

Upon completion of this module, you will be able to: y List objectives for performance analysis y Discuss the relationships between IO size, IO Operations per Second, and data throughput y Describe performance metrics that can be queried on the Symmetrix y Use the symstat command to display activity on the Symmetrix
Performance Monitoring
y Performance monitoring is the process used to identify what is normal and to identify bottlenecks in a system y Optimal performance is achieved when workload is spread evenly across all components y Factors outside the Symmetrix significantly impact application performance
Cache
BE
FE Symmetrix
2007 EMC Corporation. All rights reserved. Monitoring Symmetrix Activity - 4
Performance tuning is all about identifying bottlenecks in the systems so they can be eliminated. Bottlenecks occur when one or more components are over utilized and new work becomes queued up and waits before being processed. A physical disk drive is a good example of a potential bottleneck. Because of mechanical and electrical limitations, a physical disk drive is only capable of processing a certain number of IOs in a given timeframe. If there is more work than can be processed, some work will be held up. Performance analysis is about analyzing the workload to identify these bottlenecks or hotspots. Once a bottleneck is identified, it can be removed by rebalancing the workload. Quite often, after a bottleneck is removed, another one will appear. Therefore performance analysis is a iterative process of monitor, identify, resolve. While our focus is on the Symmetrix, there are many other factors that can be the root cause of perceived performance problems. Often times, we monitor to qualify that the problem is not with the Symmetrix. Other times real bottlenecks can be identified and tuning is necessary to resolve. Performance can be defined by raw numbers, it is usually the user experience in real-world environments that will prompt the investigation of performance.
IO Operation - End to End

y Different prospective: Application, Host, SAN, Storage System y I/O might be broken into smaller units, or re-combined depending on the protocol of the end points
1 File (File System) 6 Blocks (HBA) 12 Frames (Fibre) 6 Blocks (FA) 2 Slots (Cache) 4 Writes (DA->Disk)
An I/O is a unit of work, or a complete transfer, between two end points. There are many end points in a single pathway of the Enterprise storage environment. Each one uses a different protocol which may break the received I/O into multiple I/Os before passing it on to the next end point. When considering an I/O at a particular level, always keep in mind that more than one complete transfer might be taking place at a different level to transmit that same data. The illustration shows how a single file might be broken up during the process of storing it on a Symmetrix. The file will frequently be broken up into multiple I/Os (or blocks) when the File System Manager transfers it to the Host Bus Adapter. The HBA perceives the activity as a series of smaller work units. The HBA will conform to Fibre Channel protocol, and transfer each I/O it receives as a series of frames of up to 2 KB to the next connectivity device. The frames are routed through the fabric to the fibre adapter (FA) on a Symmetrix, where they are re-assembled into the original I/O form seen by the HBA. The FA will transfer the data into Symmetrix cache slots. For this example, lets assume that the file data fits smoothly into two slots. The cache slots will be treated as single units from this point; a disk adapter (DA) will transfer each slot to the physical disk (called destaging) in one write operation. However, if the data is mirrored, twice as many writes will be needed to commit the data to disk.
I/O Components
Neg
Header
Data
Ack
time
All I/O operations include four stages:

1. Negotiation Communications used by both endpoints to agree to and manage the operation. 2. Header Identifying or addressing information. 3. Data Actual transferred data. 4. Acknowledgement Final communication to terminate the action.
Time required for negotiation, header, and acknowledgement typically fixed in size
Larger I/Os have the same overhead as smaller I/Os
Transferring a single I/O between any points involves more than just transferring the data. In every data transfer protocol, several other tasks must be performed when sending an I/O. These other tasks add time to the overall process and increase the amount of data on the transmission channel. They can be thought of as additional parts or components of the I/O transfer process. The components are: Negotiation, Acknowledgement Both endpoints must agree to the transfer and manage the operation. This includes any handshaking tasks that processors use to schedule and organize the activity. Most protocols require some setup negotiation to start the I/O and a final finish message to terminate it. Header Some identifier or address value that the receiving endpoint uses to properly utilize the data. Any CRC or parity bits used to error check the data can be considered header also, since this overhead adds to the information load on the channel. Data The actual data. All of the other components are added to the I/O during transfer and removed before the data can be utilized. The Negotiation, Header, and Acknowledgement components are largely fixed in size regardless of I/O size. A 2 KB I/O requires about the same negotiation and header information as a 32 KB I/O. A change in the I/O size typically means a change in the Data component only. The illustration graphically shows the parts of an I/O on a time graph. At the start of the I/O, some time is taken for the Negotiation, then time is taken for the I/O header, etc.
I/O Per Second

y I/O Per Second (IOPS)
Number of transfers per second An important measure of I/O, especially when large numbers must be transferred to meet service levels, e.g. database.
Time to completely transmit I/O, or channel cost of transmission
I/O Per Second
I/O per second (IOPS) measures the number of transfers per second between end points. The most common end points considered for this measurement are a host and its storage. As data is broken into blocks and transferred to and from disk, each is counted as an I/O. IOPS can also be measured at more focused points in the data transfer, such as between a caching system and the physical disks. IOPS is a good measure when the I/O sizes are small, especially when large numbers must be transferred to meet service levels. This is often true in OLTP (On Line Transaction Processing) environments: a high IOPS measure indicates that many database transactions (typically one or more I/O each) can be serviced per second.
Throughput
y Throughput
Volume of Data transferred per second
Overhead for negotiation, header, etc ignored Larger I/Os , provide higher throughput
Frequently used as the capacity rating of channel

e.g. 200 MBps Fibre, 100 MBps iSCSI Theoretical maximum for all traffic on the channel, including headers and negotiation
Measured Throughput Actual cost of data transfer

Throughput measures the volume of data transferred per second through an I/O channel. All commonly used performance measurement tools report only the data transferred when measuring throughput, since this is the useful part of the I/O. The header and negotiation overhead are ignored even though bytes of content must be transferred across the channel to complete these tasks as well. Throughput is a useful measure when I/O is large, especially when the time taken to transfer the overall volume of data needs to be minimized. Backup is a good example of this sort of activity. The total number of individual I/Os is not a concern, the total time to move the backup data archive is. A throughput value is often used to rate the speed of a channel: 200 MBps Fibre, 100 MBps iSCSI, etc. This is more accurately termed bandwidth. This number is a theoretical maximum for all traffic on the channel, including headers and negotiation. Since performance measurement tools report the data volume and ignore header and negotiation, measured throughput can not reach the rated maximums in practice. Additionally, processors at the endpoints will often require some calculation time between requests, further ensuring that the maximum ratings can not be achieved under real conditions.
IOPS, Throughput, and I/O Size Relationship

y IOPS, and throughput are inversely related
Block size is a key factor
Small I/O IOPS: 5 Through: Large I/O IOPS: 2 Through:
high
IOPS
Throughput
y As I/O size increases, IOPS decreases y As I/O size increases, Throughput increases
low I/O Size

The I/O per second, throughput, and I/O size measures are closely related. When small I/Os are transferred less time is taken on each I/O, increasing the number that can be moved in a given time period. The IOPS measure is typically large for small I/Os. However, the data payload is small when compared to the header and negotiation factors. So while many I/Os are being exchanged, the total data throughput is small. When large I/Os are transferred more time is taken on each I/O, decreasing the IOPS measure. However, a larger percentage of the channel resources are spent on data rather than header and negotiation. This increases the overall throughput. This is illustrated in the block diagrams above. The combined data parts of the large I/O example is roughly 50% larger than the small I/O example. The bottom graph in the illustration shows this relationship. As the I/O size being transferred increases the IOPS measure drops. If the number of transfers per second is the important measure of performance in this environment, then smaller I/O size is beneficial. Databases typically set a size for their I/O transfers that maximizes this benefit. The graph also illustrates how throughput will increase as the I/O size increases, even as the total number of I/Os goes down. File copy and backup operations will take advantage of this relationship by increasing the I/O size.
Response Time
y Length of time taken to process an I/O y Often the only real measure of performance problems
If response time increases to noticeable levels, you have a problem
y Symmetrix response time measured by front-end director y Host response time measured by host operating system
App
OS
Host queue
HBA
Connectivity
FA
Cache
DA
Disk
Host measures
Symmetrix measures
Response time is another measure of performance that is often used in parallel with measures like I/O per second and throughput. Response time measures the total time taken to process an I/O. This is often very useful in detecting application slowdowns. The speed of an application that relies on disk access is defined by the access response time, so when it increases, the application necessarily must slow down. Symmetrix response time is measured by the front end directors. The time of the start of the I/O negotiation by the director to the time of the final acknowledgement is the Symmetrix response time. Therefore, these measures omit any time taken in SAN connectivity devices, HBA delays, or host queuing. Host response times are measured by the host operating system. The time of the start of the applications request to the time of the operating systems final acknowledgement to the application is the host response time. This measure includes queuing, HBA and connectivity delays, as well as the entirety of the Symmetrix time.
Symmetrix Statistics
y The Symmetrix maintains counters that provide statistics on system activity
IO per second and throughput Front-end directors and ports Back-end directors and ports Disks drives Cache hits and misses Write Pendings Other metrics
y Solutions Enabler symstat command querying these counters allow real-time reporting of activity y Counters are leveraged by other tools such as Work Load Analyzer to provide historical view of activity
The Symmetrix array maintains statistical counters for performance-critical operations. Using the symstat command, you can view the performance statistics derived from these counters. Real-time vs. historical: y Real Time Whats the current workload on the Symmetrix? y Historical When do peaks occur? Will peaks overlap? Will peaks be distributed
Reporting Objects
y symstat reports statistics on how object perform in an environment:
Devices
Throughput and I/O statistical on a device or set of device within a device group Read/Write Operations Cache statistics including hits and misses Other statistics including dynamic mirroring service policy (DMSP)
Directors
Similar statistics but on director, director port, or type of director basis Prefetching activity
Disk
Reports Back-end I/O requests and throughput for selected disks
SRDF/A sessions
CYCLE, CACHE, and REQUESTS
The statistics command (symstat) performs the following: y Queries Symmetrix devices to capture raw performance counts and store them in memory. y Retrieves the performance counts for the Symmetrix array as a whole. y Retrieves the performance counts for a director or director port. y Retrieves the performance counts for one or more Symmetrix devices. y Retrieves the performance counts for one or more Symmetrix device groups, composite groups, or RDF groups. y Retrieves the performance counts for a selection of, or all, Symmetrix disks. y Retrieves the timestamp of the performance count sample. y Retrieves and displays replication session statistics for SRDF/A. y Retrieves GigE iSCSI network statistics.
Types of Statistics
y REQUESTS
Reports I/O requests and throughput for selected devices, directors, or SRDF/A sessions Default type if no type is specified
y BACKEND
Reports back-end I/O requests and throughput for selected devices
y PORT
Reports performance statistics for a director port
y ISCSI
Report GigE network statistics
y CACHE
Reports cache activity for selected front-end or remote link directors, or SRDF/A sessions
y MEMIO
Reports cache memory to disk activity for selected devices
y DISK
Reports back-end I/O requests and throughput for selected disks
y PREFETCH
Reports track prefetch disk activity for selected back-end directors only
y DMSP
Reports dynamic mirroring service policy (DMSP) statistics for the selected device(s)
Other symstat statistics include: CYCLE y Report cycle summary information for SRDF-A sessions. RDF y Reports SRDF statistics PATH y Report R-Copy path information for non incremental sessions.
REQUESTS Front-end Activity

y Most common type of statistic Default for symstat command
I/O requests I/O Operations per second Throughput - Data transferred
y Real-time statistical information for:

Entire storage system Device Device group, SRDF/A sessions
y Specify the interval and the number of samples (counts) y Example: symstat -i 5 -c 5 -sid 054
The default type of statistics for the for the symstat command is requests, therefore if a type is not specified with the type option, requests are presented. The -c argument defines the number of samples. The default for this argument is continuous sampling. If you do not specify this argument, but you specify an -i value, the command produces a continuous statistical output, requiring a cancel (Ctrl-C) to stop the process. If no sample interval is specified, the default sample interval is 10 seconds; the minimum is 5 seconds. It is recommended that 30 seconds or greater be used for effective statistical sampling. To filter the information to display only information of interest, you can use the g and specify a device group. symstat -i 5 -c 3 -g production_dg
I/O Requests - symstat type request

C:\>symstat C:\>symstat -i -i 5 5 c c 5 5 DEVICE DEVICE 07:07:01 07:07:01 07:07:06 07:07:06 0021 0021 (DRIVE2) (DRIVE2) 0022 0022 (DRIVE3) (DRIVE3) 0023 0023 (DRIVE4) (DRIVE4) 0024 0024 (DRIVE5) (DRIVE5) Total Total 07:07:12 07:07:12 0021 0021 (DRIVE2) (DRIVE2) 0022 0022 (DRIVE3) (DRIVE3) 0023 0023 (DRIVE4) (DRIVE4) 0024 0024 (DRIVE5) (DRIVE5) Total Total IO/sec IO/sec READ READ 538 538 488 488 566 566 195 195 1787 1787 527 527 500 500 538 538 224 224 1789 1789 WRITE WRITE 255 255 241 241 265 265 98 98 859 859 253 253 242 242 268 268 108 108 871 871 KB/sec KB/sec READ READ 8617 8617 7808 7808 9062 9062 3139 3139 28626 28626 8441 8441 8006 8006 8608 8608 3593 3593 28648 28648 WRITE WRITE 3865 3865 1571 1571 13765 13765 % % Hits Hits %Seq %Seq Num Num WP WP RD RD WRT WRT READ READ Tracks Tracks 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20810 20810 19086 19086 20002 20002 1860 1860 61758 61758 20610 20610 19020 19020 20470 20470 2072 2072 62172 62172 98 98 100 100 77 77 100 100 97 97 100 100
4080 4080 100 100 100 100 4249 4249 100 100 100 100
----------- ----------- ------------- ------------- ----- -----
----- -----------
4051 4051 100 100 100 100 3872 3872 1731 1731 13948 13948 99 99 100 100 80 80 100 100 97 97 100 100 4294 4294 100 100 100 100
----------- ----------- ------------- ------------- ----- -----
----- -----------
The default type of statistics for the for the symstat command is requests, therefore if a type is not specified with the type option, requests are presented.
Understanding Write Pending

y Global Memory is not an infinite resource y Some memory is used by the system, the rest is available for cache slots
With DMX3 Cache is allocated in 64K slots Read Cache Write Cache
No more than 80% of total cache slots
Global Memory
Cache Slots
y Cache is a shared resource

All I/O operations goes through cache Allocated with fairness to prevent a few very active devices from consuming all available cache
Global Data Track Tables
Cache is the heart of a Symmetrix. Every I/O must go through cache, whether it is a read or a write. From a users data perspective, cache serves two primary functions. y Maintains recently accessed data making it readily available. Statistically, users will access data they most recently used. The longer data resides untouched, the less likely it is to be accessed again. The system uses Least Recently Used (LRU) algorithms to determine which cache slots to replace and which ones to keep. y Buffer writes until they can be destaged to a persistent location on disk.. When a host writes data to the DMX, it is stored in cache, and the host is notified the data is saved. Data is then destaged to disk as a background process while the Symmetrix systems is less utilized. The total size of cache is determined by the number and size of the memory boards in the Symmetrix. DMX systems can have a maximum of 8 boards that are up to 64GB each for a total of TB., The IMPL.bin that defines how cache is laid out and allocates three area; the Common Area, Device Table Area and Cache Slot area. There is one Device Table for every configured logical in the Symmetrix. The tables are used as a look up space for each track in the system. The aim of the tables is to provide direct access to a given block of data, either in cache (if it is there) or on the drives, in the shortest time possible. The table space starts at the end of the common area of global memory and does not end until each configured logical volume is mapped. This part of global memory is directly mapped, and the structures are built during the IMPL of the Symmetrix. As we have just mentioned the logical volumes are mapped into the direct mapping area of cache.
Write Pending Limits

y Three types of write pending limits:
System DA Device
symcfg symcfg list list -v -v <...> <...> Symmetrix Symmetrix ID: ID: 000190300180 000190300180 Time Time Zone Zone : : EDT EDT Product Product Model Model : : DMX3-6 DMX3-6 Symmetrix Symmetrix ID ID : : 000190300180 000190300180 Microcode Microcode Version Version (Number) (Number) : : 5771 5771 (168B0000) (168B0000) Microcode Microcode Date Date : : 06.22.2006 06.22.2006 Microcode Microcode Patch Patch Date Date : : 06.22.2006 06.22.2006 Microcode Microcode Patch Patch Level Level : : 86 86 Cache Cache Size Size (Mirrored) (Mirrored) : : 16384 16384 (MB) (MB) # # of of Available Available Cache Cache Slots Slots : : 460798 460798 # # of of PermaCache PermaCache Slots Slots In In Use Use : : 6 6 Max Max # # of of System System Write Write Pending Pending Slots Slots : : 368998 368998 Max Max # # of of DA DA Write Write Pending Pending Slots Slots : : 184498 184498 Max Max # # of of Device Device Write Write Pending Pending Slots Slots : : 18440 18440
When a user writes to any device in the Symmetrix system, the data is write pending (WP) until it is destaged from the cache to the physical disk. This acts as a buffer for the writes, and enables the system to accept writes faster then disk speeds. Under normal circumstances, the Symmetrix system will not destage data to disk immediately after its written to cache. This enables time for the intelligent algorithms to optimize the order of writes to the disk. Since most applications write with a high locality of reference, it can dramatically reduce the I/O traffic to the back-end disks. When write requests coming into the Symmetrix arrive faster than they can be destaged on the back-end, performance problems can occur. This is especially true when there are a large number of writes to a few volumes. The system imposes a limits on the: y Logical Volume Write-pending (LVWP), about 5% of available write cache. When a device reaches the LVWP ceiling, each new write to a track that is not already write pending for the device will trigger a special task. This special task waits for write pending data to be destaged to disk and a slot to become free before the new write is accepted. y System-wide write-pending (80% of usable cache). When the Symmetrix is at the systemwide write-pending limit, and you write to a track that is not already write pending, each new write will trigger a special task. This task waits for write pending data to be destaged to disk and a slot to become free before the new write is completed.
Type Memory
y symstat -i 5 -c 3 -type memio
C:\>symstat C:\>symstat -i -i 5 5 -c -c 3 3 -type -type memio memio DEVICE DEVICE 07:17:12 07:17:12 07:17:17 07:17:17 0021 0021 (DRIVE2) (DRIVE2) 0022 0022 (DRIVE3) (DRIVE3) 0023 (DRIVE4) 0023 (DRIVE4) 0024 (DRIVE5) 0024 (DRIVE5) 0025 0025 (DRIVE6) (DRIVE6) 0026 0026 (DRIVE7) (DRIVE7) 0027 (DRIVE8) 0027 (DRIVE8) WP WP Tracks Tracks 4490 4490 4402 4402 4548 4548 4416 4416 1406 1406 1042 1042 1032 1032 ----------21336 21336 5680 5680 5650 5650 5716 5716 5646 5646 1570 1570 1158 1158 1250 1250 ----------26670 26670 Tracks/sec Tracks/sec Prefchd Prefchd Destgd Destgd % % Dev Dev WPmax WPmax
0 0 11 0 0 11 0 0 11 0 0 11 0 0 11 0 0 11 0 0 11 0 0 11 4 98 3 4 98 3 2 86 2 2 86 2 8 93 2 8 93 2 --------------------- ------ -----14 277 7 14 277 7 0 0 14 0 0 14 0 0 14 0 0 14 0 0 14 0 0 14 0 0 14 0 0 14 2 93 4 2 93 4 4 80 3 4 80 3 8 77 3 8 77 3 --------------------- ------ -----14 250 9 14 250 9
07:17:23 07:17:23 0021 0021 (DRIVE2) (DRIVE2) 0022 0022 (DRIVE3) (DRIVE3) 0023 (DRIVE4) 0023 (DRIVE4) 0024 (DRIVE5) 0024 (DRIVE5) 0025 0025 (DRIVE6) (DRIVE6) 0026 0026 (DRIVE7) (DRIVE7) 0027 (DRIVE8) 0027 (DRIVE8)
Type Memory IO shows cache statistics including Write pendings, and prefetch activities
Type Cache
y symstat -i 5 -sa all -type cache
C:\>symstat C:\>symstat -i -i 5 5 -sa -sa DIRECTOR DIRECTOR 07:19:48 07:19:48 07:19:54 07:19:54 FA-8A FA-8A 07:19:59 07:19:59 FA-8A FA-8A 07:20:04 07:20:04 DIRECTOR DIRECTOR 07:20:09 07:20:09 07:20:09 07:20:09 FA-8A FA-8A 07:20:15 07:20:15 07:20:20 07:20:20 FA-8A FA-8A 07:20:25 07:20:25 07:20:30 07:20:30 FA-8A FA-8A DIRECTOR DIRECTOR 07:20:35 07:20:35 07:20:35 07:20:35 FA-8A FA-8A 07:20:41 07:20:41 07:20:48 07:20:48 FA-8A FA-8A 07:20:53 07:20:53 FA-8A FA-8A
all all -type -type cache cache Misses/sec Disconnects/sec Misses/sec Disconnects/sec Cache Read System Device Cache Read System Device 217 0 0 217 0 0 207 0 0 207 0 0 Misses/sec Misses/sec Cache Cache Read Read 196 196 184 184 174 174 Misses/sec Misses/sec Cache Cache Read Read 164 164 106 106 133 133 Disconnects/sec Disconnects/sec System Device System Device 0 0 0 0 0 0 0 0
0 0 0 0 Disconnects/sec Disconnects/sec System Device System Device 0 0 0 0 0 0 0 0 0 0 0 0

Cache misses often occur with random workload and require the date be staged from disk befor the I/O operation completes.
Front-end Director
y Front-end director board has four independent processors
Loaded with specific emulation code
ESCON EA FICON EF Fibre Channel FA ISCSI SE 8D Direct access channels to cache 8C 8B 8A 9D 9C 9B 9A
Each processor has one or more ports Every director has a direct connection to every global memory board
y Performance activity can be measured at the Processor level or the port level
Front-end directors, also called channel directors are normally installed in pairs, providing redundancy and continuous availability in the event of repair or replacement. Each front-end director has multiple microprocessors and supports multiple independent data paths to the global memory to and from the host system.
Director Request
y symstat -i 10 -c 3 -dir all
C:\>symstat C:\>symstat -i -i 10 10 -c -c 3 3 -dir -dir all all DIRECTOR IO/sec Cache DIRECTOR IO/sec Cache Requests/sec Requests/sec 07:25:42 READ WRITE RW 07:25:42 READ WRITE RW 07:25:52 07:25:52 DF-1A DF-1A FA-8A FA-8A DF-16A DF-16A DF-1B DF-1B DF-16B DF-16B DF-1C DF-1C DF-16C DF-16C DF-1D DF-1D DF-16D DF-16D Total Total 07:26:03 07:26:03 DF-1A DF-1A FA-8A FA-8A DF-16A DF-16A DF-1B DF-1B DF-16B DF-16B DF-1C DF-1C DF-16C DF-16C DF-1D DF-1D DF-16D DF-16D Total Total
% % RW RW Hits Hits 0 0 100 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ----72 72 0 0 100 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ----75 75

264 264 3435 3435 140 140 171 171 150 150 272 272 145 145 303 303 284 284 ----------5164 5164 178 178 3568 3568 68 68 81 81 349 349 190 190 324 324 178 178 187 187 ----------5123 5123
0 0 2133 2133 0 0 0 0 0 0 2 2 1 1 2 2 2 2 ----------2140 2140 0 0 2224 2224 0 0 0 0 1 1 0 0 2 2 1 1 1 1 ----------2229 2229
254 254 2120 2120 133 133 164 164 143 143 257 257 136 136 285 285 269 269 ----------3761 3761 171 171 2206 2206 65 65 76 76 333 333 181 181 306 306 170 170 178 178 ----------3686 3686
255 255 4254 4254 133 133 164 164 144 144 259 259 138 138 288 288 271 271 ----------5906 5906 172 172 4431 4431 65 65 76 76 335 335 182 182 308 308 171 171 179 179 ----------5919 5919
Front-end and back-end requests
Port
y symstat -type PORT -dir -port
C:\>symstat C:\>symstat -i -i 10 10 -type -type PORT PORT -dir -dir 8a 8a -port -port 0 0 07:39:44 07:39:44 07:39:44 07:39:44 07:39:54 07:39:54 07:40:05 07:40:05 07:40:15 07:40:15 07:40:25 07:40:25 07:40:35 07:40:35 07:40:35 07:40:35 07:40:46 07:40:46 07:40:56 07:40:56 07:41:06 07:41:06 07:41:17 07:41:17 DIRECTOR DIRECTOR FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A DIRECTOR DIRECTOR FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A FA-8A PORT PORT 0 0 0 0 0 0 0 0 0 0 PORT PORT 0 0 0 0 0 0 0 0 0 0 IO/sec IO/sec 4675 4675 4612 4612 3244 3244 4715 4715 3293 3293 IO/sec IO/sec 4643 4643 3215 3215 4643 4643 3067 3067 4810 4810 Kbytes/sec Kbytes/sec 74770 74770 73640 73640 51802 51802 75404 75404 52668 52668 Kbytes/sec Kbytes/sec 74248 74248 51412 51412 74260 74260 48822 48822 76934 76934
Individual port statistics are valuable when analyzing response time issues.
Back-end Components
y Logical Device
Traffic routed to that device, no matter how it is physically stored
device
y Disk Director
Traffic handled by the processor for all disks, including protection writes and business continuity synchronization
DA-1A
DA
directors
DA-16A
DA
ports disks
y Disk
Traffic to/from all hyper volumes on the disk, including protection, BC, and internal scrubbing
hyper volumes
There are three components involved with processing an I/O from the back-end prespective: The logical device For the most part, I/Os from the host to a particular logical device are observed and measured at the device level. So a standard measure like I/O per second indicates the number of I/O generated by the hosts accessing the device. However, some device measures show the back end traffic associated with the device. These measures are typically prefixed with DA. The Disk Director Traffic measured at the disk director level shows all traffic to/from all of the physical disks controlled by the director. This includes host I/O, protection writes to mirrored and RAID devices, and internal synchronizations like those performed by TimeFinder and Optimizer. The Disk Physical disk traffic is measured at the disk level. All traffic for the hyper volumes stored on a physical disk are measured here. As with the disk adapter level, traffic due to protection and internal synchronizations will also be visible in these metrics. Additionally, traffic performed by internal scrubbing (error detection and correction) will also be apparent here.
BACKEND Back-end Activity

y symstat -type backend
C:\>symstat C:\>symstat -type -type backend backend -i -i 10 10 DEVICE IO/sec DEVICE IO/sec 07:43:50 READ 07:43:50 READ WRITE WRITE 07:44:00 07:44:00 0021 0021 0022 0022 0023 0023 0024 0024 0025 0025 0026 0026 0027 0027 0036 0036 0037 0037 0038 0038 0039 0039 003A 003A 003B 003B 003D 003D 003F 003F Total Total
KB/sec KB/sec READ WRITE READ WRITE
Prefetched Prefetched Tracks Tracks Tracks Used Tracks Used 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ----------0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ----------0 0
(DRIVE2) (DRIVE2) (DRIVE3) (DRIVE3) (DRIVE4) (DRIVE4) (DRIVE5) (DRIVE5) (DRIVE6) (DRIVE6) (DRIVE7) (DRIVE7) (DRIVE8) (DRIVE8) (Not (Not Visible Visible (Not (Not Visible Visible (Not (Not Visible Visible (Not (Not Visible Visible (Not (Not Visible Visible (Not (Not Visible Visible (Not (Not Visible Visible (Not (Not Visible Visible
) ) ) ) ) ) ) ) ) ) ) ) ) ) ) )
0 57 0 5435 0 57 0 5435 0 66 0 5700 0 66 0 5700 0 85 0 5928 0 85 0 5928 0 91 0 6076 0 91 0 6076 0 92 0 5871 0 92 0 5871 0 65 0 5679 0 65 0 5679 0 100 0 7829 0 100 0 7829 0 0 51 0 0 0 51 0 0 0 76 0 0 0 76 0 0 0 25 0 0 0 25 0 0 0 57 0 0 0 57 0 0 0 76 0 0 0 76 0 0 0 128 0 0 0 128 0 0 0 57 0 0 0 57 0 0 0 76 0 0 0 76 0 ----------- ----------- ------------- ------------0 556 546 42518 0 556 546 42518
Displays performance statistics at the back-end DA
Symmetrix Prefetch
y DA detects two sequential reads, considers probability of sequence continuing, begins prefetch task for device y Can have many simultaneous prefetch tasks y Two tracks are read into cache ahead of read stream y As sequential stream continues, more read ahead: up to 32 tracks y Prefetch task remains for an additional 120 seconds after sequential I/O stops to avoid thrashing
Enginuitys Prefetch algorithm collects statistics about the sequential patterns of each logical volume. Prefetch calculates the probabilities that the host will read subsequent tracks (those that follow the track that the system just read) in the very near future. Based on these probabilities and system utilization factors, Enginuity decides whether to prefetch more data, how many tracks to prefetch, when to prefetch, and for how long to keep the prefetched data in cache. The system collects statistics on an ongoing basis so that the algorithm can adjust itself to rapid changes in the workload. The Symmetrix prefetching algorithm is carried out by the Disk Directors. When two sequential reads are detected, a prefetch task is started for the device. Depending on the Enginuity level, eight or more prefetch tasks can be active on a single device at a time. Considering the number of devices in a modern Symmetrix array, this means that the disk directors can be monitoring a considerable number of prefetch tasks at once. Once the task begins, the disk directors try to prefetch data into cache at least two tracks in advance of the current read address. As the sequential stream continues, more read ahead will be performed up to a maximum of 32 tracks. Once the sequential read stream stops, the prefetch task will remain monitoring the activity for another 120 seconds before terminating. If the sequence begins again before the 120 seconds has elapsed, the same task will continue the prefetch work. This prevents tasks from being a performance issue by starting and stopping frequently.
Prefetch
symstat -type prefetch
C:\>symstat C:\>symstat -i -i 5 5 -c -c 4 4 -sid -sid 254 254 -da -da 2a 2a -type -type prefetch prefetch
DIRECTOR DIRECTOR 12:08:20 12:08:20
Preftch Preftch Trk/sec Trk/sec
Preftch Preftch Trk/sec Trk/sec Used Used Unused Unused
12:08:25 12:08:25
DA-2A DA-2A
1 1
1 1
0 0
12:08:30 12:08:30
DA-2A DA-2A
7 7
6 6
1 1
12:08:36 12:08:36
DA-2A DA-2A
19 19
19 19
0 0
Physical Disks
y Physical disk are the slowest link in the I/O Chain
Positional delay Rotational delay
y Physics limits the speed of a drive

Parallelism overcome physical limits Best performance is achieved using more smaller drives
y Key to performance is balance

Utilize as many as possible
y But the best performance is achieved by servicing I/O request from memory rather than disk
Benefits of cache
Any given worlkoad is going to put some level of stress on the backend. Understanding the backend activity and the workload characteristics is critical in determining the number of spindles necessary.
Disk
symstat -type disk
C:\>symstat C:\>symstat -type -type disk disk -i -i 10 10 -sid -sid 254 254 DISK IO/sec KB/sec DISK IO/sec KB/sec 08:02:40 READ READ WRITE 08:02:40 READ WRITE WRITE READ WRITE 08:02:50 08:02:50 01A:C2 1 0 76 0 01A:C2 1 0 76 0 01A:C4 0 0 51 0 01A:C4 0 0 51 0 01A:C6 0 151 38 5864 01A:C6 0 151 38 5864 01A:D1 0 0 32 0 01A:D1 0 0 32 0 01A:D3 0 0 25 0 01A:D3 0 0 25 0 01A:D5 0 149 0 5749 01A:D5 0 149 0 5749 16A:C3 1 0 76 0 16A:C3 1 0 76 0 16A:C5 0 0 51 0 16A:C5 0 0 51 0 16A:D2 0 0 32 0 16A:D2 0 0 32 0 16A:D4 0 120 6 4574 16A:D4 0 120 6 4574 01C:C4 0 163 6 6302 01C:C4 0 163 6 6302 01C:C6 0 0 25 0 01C:C6 0 0 25 0 01C:D1 1 0 76 0 01C:D1 1 0 76 0 01C:D3 0 0 51 0 01C:D3 0 0 51 0 01C:D5 0 151 0 5744 01C:D5 0 151 0 5744 16C:C3 0 0 32 0 16C:C3 0 0 32 0 16C:C5 0 139 25 5320 16C:C5 0 139 25 5320 16C:D2 1 0 76 0 16C:D2 1 0 76 0 16D:D5 0 166 0 6332 16D:D5 0 166 0 6332 Total -----Total ------ ----------- ------------- ------------8 1994 1648 76750 8 1994 1648 76750
The type DISK displays I/O requests and throughput on a physical disk. The drive is identified by the DA director number, interface ID, and SCSI ID of the disk drive
Module Summary
Key points covered in this module: y The Symmetrix maintains statistical counters of activity in the array
Querying these counters allow real-time reporting of activity
Solutions Enabler symstat command
Allows real-time monitoring of activity on the Symmetrix Complements other tools such as Workload Analyzer
y Keep in mind
Individual statistics are of little value when taken alone
Analysis Relationships
What is good in one environment could be a problem in another There is more in the I/O stack than just the Symmetrix that can impact performance
y PowerLink
Closing Slide
Module 7: Symmetrix Security
Welcome to the last module of Symmetrix Monitoring and Management In this module we will take a look at the Solutions Enabler security features including user-based and host-based access controls, and auditing facilities.
Symmetrix Security - 1
Revision History
Rev Number
1.0
Course Date
August, 2007
Revisions
Complete
Symmetrix Approach to Security

y Three prong approach:
Physical Security
Restricted data center access Passwords Secure Service Credentials (SSC) on the Service Processor
Access controls for configuration and control

Host based using symacl User based using symauth
Monitoring - Who did what when?

symaudit symevent
y In addition to access controls provided by Volume Logix and fabric zoning

2007 EMC Corporation. All rights reserved. Symmetrix Security - 3
The first defense in security is to control physical access to the systems. All the access control and auditing will not help if the Symmetrix is open to physical threat. Nearly all enterprise customers restrict access to the data center and today, with Secure Service Credentials on the service processor, even if a person could physically get to the Symmetrix, they couldnt log into the Symmetrix and invoke configuration changes or control operations. Not all threats require physical access to the Symmetrix. An end user on a host attached to the Symmetrix could invoke commands to cause accidental or malicious data loss or unauthorized access. The Symmetrix includes host-based and user-based access control that enables only the required level of access but not more access than required. Also important to security is understanding who is doing (or attempting) to do what operations on the system. These protections are in addition to the host access controls provided by device masking using Access Logix and fabric zoning.
Host-based Access Control - symacl

y Caution! Anyone with host access to a can execute any configuration and control function on any Symmetrix device
Fibre Channel connection to the Symmetrix At least on device visible to serve as a Gatekeeper Solutions Enabler installed and appropriate license
y Solution:
Initialize and manage Symmetrix Access Control Restrict management access so only specific hosts can perform specific configuration and operations on specific devices Does not limit or restrict normal I/O Operations
y Results:
Secure sharing of a system Eliminate accidental or intentional tampering Support change control process by restricting ad hoc operations
Anyone with access to Symmetrix-based management software can execute any function on any Symmetrix device. Applications can issue management commands through any device in a Symmetrix and invoke configuration or control commands. Open Systems hosts can manipulate mainframe devices, Windows hosts can manipulate UNIX volumes, and vice versa. Shared storage systems, are vulnerable to either accidentally or intentionally tampering. To prevent this, the symacl command can be used by an Symmetrix administrator to restrict host access to defined sets of devices and specific operations When configured and an unauthorized host attempts to run a control operation, the SYMCLI command will return a message similar to the following: Symmetrix Access Control Denied the Request, to the consol and log the unauthorized use in the SYMAPI log, and the Symmetrix Audit log. SYMACL is a feature of the Symmetrix and Enginuity operating environment beginning with Version 5x67 and Solutions Enabler 4.3 and later. Note: Once Access Control have been implemented, additional steps must be added to customers Change Management Process. Provisioning operations require access control changes!
symacl Key Components

y Unique host identifier (Access ID)
Encrypted hexadecimal number
Host
Access ID:
12345678-ABCDEF12-87654321
PIN Derived from the physical hardware and software configuration Created using the symacl unique command
y PIN number known only to authorized Access Control Administrators y Access Control database in the Symmetrix
Maintains the definitions of what hosts can do what operations on what devices Resides in the Symmetrix File System
No direct user access
Access Access Control Control Database Database
Managed only through an authorized host and by a administrator that has the PIN
Symmetrix
Hosts with management rights to the Symmetrix are identified with an Access ID, an encrypted hexadecimal number. These unique identifiers are derived from the system configuration. An example of an Access ID is: 46712F2A-AF54C32A-FE365D1C. By uniquely identifying a host, it is possible to target specific rights to specific host on a system-by-system basis. If a host cannot produce a unique ID, it will be considered Unknown and will be provided limited or no configuration and control access. Because this unique ID is derived from the physical configuration, any significant changes to host hardware may result in a change in ID, requiring the regeneration of the ID and updating the access control entries. When Access Controls is first configured, an access pin is created. When an Administrator attempts to execute an access control configuration command, the user will be prompted to enter the PIN. The PIN can also be set in the environment variable SYMCLI_ACCESS_PIN. All access control data is maintained in a Access Control database that resides in the Symmetrix File System.
Configuration Objects
Access Control Group
Hosts Hosts identified identified by by unique unique ID ID
Access Pool
Set Set of of Symmetrix Symmetrix Devices Devices
Control
Host A Host B Host C ...
111 121 131 121 131 111 112 122 132 112 122 132 113 123 133 113 123 133 114 124 114 114 124 114 115 125 135 115 125 135
y Access Control Group

Group of one or more host Similar access requirements
Access Control List (ACL)

y y y y y y ACE ACE ACE ACE ACE ACE
y Access Pool
List of Devices
Each host identified by unique Access ID y Access Control List Access Control Entries (ACE) Level of Access (Permissions)
Allowing hosts in Access Control Group to control devices defined in Access Pools
When configuring access control, there are three objects that are created: Access Control Groups, Access Pools, and Access Control Entries.\ y Access Control Groups Are logical grouping of hosts with similar needs (determined by an Administrator) identified by Unique access IDs and names. Access groups are allowed to act on Access Pools based on permissions (access types) granted by the Administrator. The unique host ID for open systems can be viewed by running symacl -unique. y Access pools A set of Symmetrix devices. Permissions (or level of access), are assigned to allow a host to perform certain Solutions Enabler functions on a specified set of devices. These sets of devices are referred to as Access Pools or accpool. y Access Control Entries (ACEs) Once the group and access pool mechanisms are established, the access control entries (ACEs) are created, which grant permissions to these pools. The ACEs along with Access Control Groups and Access Control Pools are managed using the symacl command and stored in the access control database. Access Control Lists (ACLs) consist of Access Control Entries (ACEs
Access Control Database

y Resides in the Symmetrix File System
No direct user access Initial configured on the Service Processor Configured and maintained using symacl commands
From authorized hosts Only by administrators with the Access PIN
FA FA
Management Host
GK GK
ACL DB
Symmetrix Symmetrix File File System System
Symmetrix
The size of the SymACL database is currently 32KB. This is large enough for 320 entries based on an average entry size of 100bytes. This is large enough for most environments. The actual space used can be seen with the symacl list v command. The entries that are high lighted are what is normally created as part of the initial default set up.
Access Controls Setup

y Initial setup is performed by an EMC Engineer on site
1. Enable access control in the binfile (5771 and prior) 2. Initialize the Access Control database in the Symmetrix File System
Creates a default Access Groups (AdminGrp and UnknwGrp) Creates default Access Pools (ALL-DEVS and !INPOOLS) Add a minimum of one administrative host (two or more would be better) Add Host Access Control ID to AdminGrp Access Group Add User Access Control ID
3. Generate a unique host identifier for the administration host(s)
4. Create an access ID PIN
SymACL requires an EMC CE using SymmWin to enable SymACL to set up the first Administrator Group. The following procedure must be completed on every Symmetrix. To implement a minimal SymACL configuration: 1. For Enginuity code 5x71 and earlier, perform an online configuration change to the BIN file with the 'Enable Access Control - Yes' initialization setting. Note: This step is not required for Enginuity releases 5x72 and later where this setting is enabled by default. 2. Initialize the SymACL database on the Symmetrix system via the SymmWin Procedure Wizard. 3. Obtain a unique identifier from a host (note two admin hosts recommended) that will be the primary and/or secondary admin host(s). This is done by running the symacl unique command on the admin host. 4. Create an administrative user from the SymmWin Procedure Wizard. This procedure configures the admin host and creates a PIN. At this point you've enabled ACL and configured an ACL admin host. By default, all other hosts have full access (via the Unknown group), unless expressly restricted via access controls.
Enable Access Control

y Must be done in the binfile with 5771 and prior
Enabled by default with 5772 Change requires Online Configuration Change
The initial setup is normally performed by EMC Engineers onsite but is shown here for reference only. Starting with 5772, the step of enabling access control is not required as it is enabled by default.
Initialize Access Control

y From SymWin Procedure Wizard
Initializes the ACL database Creates default Access Control Groups and Access Control Pools
The initial setup is normally performed by EMC Engineers onsite but is shown here for reference only. This steps creates the ACL database in the Symmetrix File system and initializes it with the default access control groups and Access Control pools.
Add Administration Host Access ID

y Identify Access ID for Administration host
C:\>symacl C:\>symacl -unique -unique The The unique unique id id for for this this host host is: is: 2C5F00A6-54408FC0-92397573 2C5F00A6-54408FC0-92397573
y Using symwin on the Service Processor
The initial setup is normally performed by EMC Engineers onsite but is shown here for reference only. At least one host must be identified as the administration host and added to the Admin Access Control Group.
Add Admin User Access ID (PIN)

y Create a User entry and PIN
User Based Host Based Quit
The initial setup is normally performed by EMC Engineers onsite but is shown here for reference only. An administrator Personal Identification Number (PIN) is also created during initialization.
Verify Access Control Status

y Access control must be enabled in the bin file
On-line Config Change Cannot use symconfigure
C:\>symacl C:\>symacl list list v v SYMMETRIX SYMMETRIX ACCESS ACCESS CONTROL CONTROL STATUS STATUS Symmetrix Symmetrix ID: ID: 000190102254 000190102254 Access Access Control Control Session Session Locked Locked Time Time Held Held in in Seconds Seconds Lock Lock Identifier Identifier Time Time Enabled Enabled Time Time Disabled Disabled Time Time Updated Updated ADMIN ADMIN priv priv ADMINRD ADMINRD priv priv DB DB space space remaining remaining C:\>symcfg C:\>symcfg list list v v Symmetrix Symmetrix ID: ID: 000190102254 000190102254 (Local) (Local) Access Access Control Control Configuration Configuration State State
: : : : : : : : : : : : : : : : : : : :
Enabled Enabled No No 0 0 None None Fri Fri Aug Aug 10 10 08:31:17 08:31:17 2007 2007 Thu Thu Jan Jan 01 01 00:00:00 00:00:00 2004 2004 Fri Fri Aug Aug 10 10 08:31:17 08:31:17 2007 2007 Yes Yes N/A N/A 31756 31756 bytes bytes
: : Enabled Enabled
symacl v is a quick way of determining if ACL is enabled. You can also see this attribute by performing a symcfg list v command.
Default Access Control Environment

y On the administration host verify that default Access Control configuration has been enabled and the host has administrative authority
AdminGrp
Admin permissions to All Devices All Permissions to All Devices
C:\>symacl C:\>symacl list list -acl -acl Symmetrix Symmetrix ID: ID: 000190102254 000190102254 Group Group Name Name --------------------------AdminGrp AdminGrp AdminGrp AdminGrp UnknwGrp UnknwGrp UnknwGrp UnknwGrp Pool Pool Name Name ----------------------ALL_DEVS ALL_DEVS ALL_DEVS ALL_DEVS ALL_DEVS ALL_DEVS !INPOOLS !INPOOLS Access Access Type Type --------------------ADMIN ADMIN ALL ALL BASE BASE ALL ALL
Unknown Group
Base Permissions to All Devices All Permissions to All Devices that are not currently in Access Pools
With the default Access Control configuration, until other access controls are configured, all hosts would continue to have full permissions to perform all operations on all devices. The recommended implementation strategy is to gradually phase in Access Controls one application at a time and continue to add appropriate access controls until the Symmetrix is locked down appropriately. Not all devices need to be placed under control of Access Control.
Securing the Symmetrix

y Planning is critical!
Full assessment of Host and Application environment required
y Best Practices is to gradually phase in implementation

1. Create Access Control Groups
Add one or more hosts
2. Create Access Control Pools

Add devices to pool
3. Create Access Control Entries

Grant BASE rights to the Access Group for All Devices Grand specific feature access rights to the Access Group for the Access Pool
y When access controls are full configured, remove the Unknown Group preventing access by undefined hosts
It cannot be over stated: Planning is the key to a successful implementation of Access controls. If not thought out completely and implemented precisely, applications will fail and customer disaster positions may be compromised. The last step of removing the Unknown group may not be necessary as it might be appropriate to only provide access controls on critical application devices and all other devices would not be members of any access pool and thus open for command and control operations.
Configuring Access Control Data

y The symacl command is used to manage access control database entries Command File y Uses a command file to define specific operations
Similar to symconfigure Command file is processed to manage access control entries
Preview Prepare Commit
Create Delete Remove Add Remove
Access Group Access ID Access Pools Devices ACE

Example: symacl commit file myacl.cmd

The symacl command works like the symconfigure command in that it uses a command file to define the management function to perform. When executing a symacl command it performs three progressive operations on the command file. y Preview operation verifies the syntax and correctness of the contents of the entries in the command file. y Prepare operation (also before you commit) performs the preview checks, but also verifies the appropriateness of the requested access control modifications against the current state of the access control database in the Symmetrix array. y Commit operation performs both the preview and prepare checks and then commits the contents of the command file to the Symmetrix Access Control database. A lock is taken out by the specified Symmetrix during an access control change session. Only one access control session can be active within a Symmetrix at a time.
Create Access Control Group

1. Generate unique Access IDs for each host
symacl unique
mycmd
create accgroup OraClstr; add host accid 12345678-87654321-ABCDEFAB name node1 to accgroup OraClstr; add host accid 87654321-12345678-ABCDEFAB name node2 to accgroup OraClstr; add host accid ABCDEFAB-12345678-87654321 name node3 to accgroup OraClstr; add host accid 12345678-87654321-ABCDEFAB name node4 to accgroup OraClstr;
2. Create Access Group 3. Add hosts to Access Group y Example:

symacl commit file mycmd
Note: You will be prompted for the access PIN if environment variable SYMCLI_ACCESS_PIN is not set
The typical approach is to implement access control by host. To do this, the first step is to generate unique access ID for each host using the command symacl unique. Next, create an access group and add all host that share similar access requirements. In the example above, in a command file, we added the commands to created a group for a cluster of Oracle servers and called the group OraClster. In the same file we added command entries to add four servers called node 1-4. Once the command file has been written and saved we can run the symacl -commit command to process the command file and add the entries to the access control database. All commit operations must be run from a host that is a member of the AdminGrp, and the user must enter the correct Access PIN that was generated when the system was configured for access control. Once a group is created additional hosts can be added, remover, or moved from one group to another. A group can also be deleted.
Example:
C:\>symacl C:\>symacl commit commit -f -f diamond.cmd diamond.cmd Enter Enter Access Access PIN: PIN: Command Command file: file: (diamond.cmd) (diamond.cmd) PREVIEW......................................................Started. PREVIEW......................................................Started. PREVIEW......................................................Done. PREVIEW......................................................Done. PREPARE......................................................Started. PREPARE......................................................Started. Creating Creating group group OraClstr......................................Done. OraClstr......................................Done. Adding Adding Host Host access access id id node1 node1 to to group group OraClstr................Done. OraClstr................Done. Adding Adding Host Host access access id id node2 node2 to to group group OraClstr................Done. OraClstr................Done. Adding Adding Host Host access access id id node3 node3 to to group group OraClstr................Done. OraClstr................Done. Adding Adding Host Host access access id id node4 node4 to to group group OraClstr................Done. OraClstr................Done. PREPARE......................................................Done. PREPARE......................................................Done. Starting Starting COMMIT..............................................Done. COMMIT..............................................Done. C:\>symacl C:\>symacl list list -accgroup -accgroup Symmetrix Symmetrix ID: ID: 000190102254 000190102254 Group Group Name Name ------------------------------------------------------------------AdminGrp AdminGrp UnknwGrp UnknwGrp OraClstr OraClstr Number Number of of Access Access IDs IDs ------------------2 2 1 1 4 4 Number Number of of ACLs ACLs ----------------2 2 2 2 0 0
Example above creates an access control group and adds the unique Host IDs of four systems.
Creating Access Pools

y
Access pools are logical groupings of Symmetrix devices

Permissions are assigned to allow a host to perform functions on a sets of devices defined in access pools
1. Identify devices related to hosts and applications that require the same access control 2. Create a Access Pool 3. Add devices to the pool
myfile.cmd
create accpool Orapool; add dev 003a:03f to accpool Orapool;
Example:
Access pools are logical groupings of Symmetrix devices. Permissions are assigned to allow a host to perform functions on a sets of devices defined in access pools. The first step is to plan access control by identifying devices that are related to an application and or set of hosts. Create a access pool and add devices using a command file. Again, when the commit is executed, the user will be prompted for the PIN if the environment variable SYMCLI_ACCESS_PIN is not set appropriately. Devices can be added or removed from the access pool and the pool can be deleted if no longer needed.
ACL Permissions (A.KA. Access Types)

y ADMIN* y ADMINRD* y ALL* y BASE y BASECTRL y BCV y CACHCTRL y CFGDEV y CFGSYM* y CHECKSUM y CREATEDV* y DIRCTRL* y ECC y OPTMZR y POWRPATH y QOS y RCOPY y SDDF y SDR y SNAP y VLOGIX*
* Access Control Entries associated with all Devices.

All others are directed at Device Pools
After creating access groups and access pools, permissions are set by creating Access Control Entries (ACEs) in the Access Control List (ACL) that grant permission to execute specific Solutions Enabler functions on a group of devices defined in access pools, by specific host defined in access groups. Reference pages 91-93 of Solutions Enabler Symmetrix Array Management CLI version 6.4 Product Guide for full description of the type of controls each ACL allows and a list of the commands effected. Note, most ACL permissions are associated with a Device Pool except those in bold which are associated with ALL DEVICES.
Access Control Entries (ACE)

y
Permissions are set by creating Access Control Entries

Grant permission to control specific Solutions Enabler functionality
On groups of devices defined in Access Pools By specific host defined in Access Groups
1. Identify Solutions Enabler functions to be controlled 2. Create a Access Control Entry

y y y Grant permissions to devices in Access Pool Grant permissions to all devices Grant permissions to devices not in a device pool
myfile.cmd
grant access=BASE to accgroup OraClstr for ALL devs; grant access=BCV to accgroup OraClstr for accpool OraPool; grant access=VLOGIX, SDR to accgroup OraClstr for NON_POOLED devs; remove access=rdf from accgroup OraClstr for accpool OraPool;
Example:
Note: Before you added permissions for specific Solutions Enabler functionality, it is first required that BASE permissions be granted.
Service Processor Considerations

y The Service Processor on the Symmetrix is often used to execute SYMCLI command y Access Control must be configured like any other host
Create Access Group containing only the Service Processor Grant:
BASE rights to ALL DEVICES OPTIMIZER to ALL DEVICES ADMINRD to ALL DEVICES Other Permissions as Appropriate
myfile.cmd
create accgroup EMCSP; add host accid 12345678-87654321-ABCDEFAB name storproc to accgroup EMCSP; grant access=BASE to accgroup EMCSP for ALL devs; grant access=OPTIMIZER to accgroup EMCSP for ALL devs; grant access=ADMINRD to accgroup EMCSP for ALL devs;
The Symmetrix Service Processor also execute Solutions Enabler commands and therefore must be configured with the appropriate access controls.
ACL Database
y When querying Access Control Database, you will only see objects associated with the access group that the host belongs
Unless host has ADMIN or ADMINRD (Read-only) permissions
C:\>symacl C:\>symacl list list -acl -acl Symmetrix Symmetrix ID: ID: 000190102254 000190102254 Group Group Name Name ----------------------------------------------AdminGrp AdminGrp AdminGrp AdminGrp UnknwGrp UnknwGrp UnknwGrp UnknwGrp OraClstr OraClstr OraCLstr OraCLstr OraClstr OraClstr OraClstr OraClstr . . . . . .
Pool Pool Name Name --------------------------------------------ALL_DEVS ALL_DEVS ALL_DEVS ALL_DEVS ALL_DEVS ALL_DEVS !INPOOLS !INPOOLS ALL_DEVS ALL_DEVS OraPool OraPool OraPool OraPool ALL_DEVS ALL_DEVS
Access Access Type Type --------------------ADMIN ADMIN ALL ALL BASE BASE ALL ALL VLOGIX VLOGIX SDR SDR BCV BCV BASE BASE
Unless the host that you are running the symacl list command from has ADMIN permissions, you will only see a subset of the access control database.
ACL Database Details -symacl show

C:\>symacl C:\>symacl show show accpool accpool MilfPool MilfPool -acl -acl Access Access Pool: Pool: MilfPool MilfPool Symmetrix : Symmetrix ID ID : 000190102254 000190102254 Number Number of of Access Access Control Control Entries: Entries: 2 2 Number : Number of of Member Member Devices Devices : 5 5 Access Control Entries (2): Access Control Entries (2): { { ----------------------------------------------------------------------Group Type Group Type --------------------------------------------------------------------------MILFORD SDR MILFORD SDR BCV BCV } } Member Member Devices Devices (5): (5): { { Device Device Device Name Name Device ----------------------------------------------------- --------------------------------------------------------------------------Cap Cap Sym Config Attribute Sts (MB) Sym Physical Physical Config Attribute Sts (MB) ------------------------------------------------------- --------------------------------------------------------------------------0030 Not Visible 2-Way N/Grp'd RW 8438 0030 Not Visible 2-Way Mir Mir N/Grp'd RW 8438 0031 2-Way N/Grp'd RW 8438 0031 Not Not Visible Visible 2-Way Mir Mir N/Grp'd RW 8438 0032 Not Visible 2-Way Mir N/Grp'd RW 8438 0032 Not Visible 2-Way Mir N/Grp'd RW 8438 0033 Not Visible 2-Way Mir N/Grp'd RW 8438 0033 Not Visible 2-Way Mir N/Grp'd RW 8438 0034 2-Way N/Grp'd RW 8438 0034 Not Not Visible Visible 2-Way Mir Mir N/Grp'd RW 8438
} }
Access Control Database Backup

y Best Practice is to backup access control database prior to making any configuration changes
Backup file is saved on local administration host Host must have ADMIN permissions File contains encrypted access IDs
y Example: symacl sid 123 backup file mybackup y Note: You will be able to view the backup files, however, Access IDs are encrypted
Like any other critical configuration database, it is recommended that it be backed up before any configuration change.
Other Considerations
y Once Access Control Management have been implemented, additional steps must be added to customers Change Management Process
Provisioning operations require access control changes
y After making access control changes for a host, update the hosts SYMAPI database
symcfg discover
y Processing a command file with a Prepare and Commit action invokes a session
Locks ensure only a single ACL change at a time If a host files and a session is locked, the lock can be released
symacl list v symacl release
After making access control changes for a host, update the hosts SYMAPI database. Because the discover simply updates the database with changes, a better approach might be to delete the database file altogether and recreate it using the command. symcfg discover
User Based Access Controls

y Users, based on usernames, are mapped to specific roles which define permitted operations
Uses the symauth command
y Independent facility
Can be used with Symmetrix based Access Controls By itself
An alternative to host-based access control is user-based. With user-based access control, a role is associated with a user name. Based on the role, the user will have the specified access level for all devices in the Symmetrix. Host-based and user-based access control, while completely separate and independent, can be used together to create the highest level of security.
Roles
y Predefined set of permissions that determine what operations a user can perform
Role is assigned to a user for ALL devices in the Symmetrix Roles are predefined - symauth list -roles
None
No capabilities
Monitor
Read only for all facilities except Audit log and Access Control Database
StorageAdmin
Perform all management and control functions
SecurityAdmin
Perform security functions including symaudit, symacl and symauth
Admin
Perform all management and control plus security
Auditor
Able to view but not modify all security setting including symaudit, symacl and symauth
Unlike host based authentication, use roles are for all devices in the Symmetrix.
Users
y Users are defined using the following format:
Type:Qualifier\Name
D = Domain user H = Host Domain or Realm name Hostname User Name
Examples:
D:emc.com\diamoj D:*\diamond H:oraserver1\jim H:*\diamond jim
User-based authentication requires that users be identified by Role. The user definition is in the format above: When the hostname or domain portion of a <Username> is an asterisk, the asterisk is treated as a wildcard meaning any host. Examples of this include: H:*\user, D:*\user, and *\user. In all other cases, the asterisk is treated as a regular character.
Configuring User Role-based Authentication

1. Create user-to-role mapping
Using symauth and command file Example: symauth commit file myfile.cmd
myfile.cmd
assign user H:QA\laura to role Monitor; assign user D:Eng\hardy to role Monitor; assign user D:Eng\moe to role Admin; assign user D:Eng\larry to role Admin; assign user D:Eng\curly to role StorageAdmin; assign user D:Eng\gilligan to role StorageAdmin; assign user D:Eng\skipper to role SecurityAdmin;
2. Enable User Authentication 3. Set enforcement policy
The first step is to create a user-to-role mapping using a command file.
Enabling User Role-based Authentication

2. Enable User Authentication
symauth sid 254 enable
3. Set enforcement policy

symauth set enforcement <advise|enforce>
C:\>hostname C:\>hostname 2K3-39-106 2K3-39-106 C:\> C:\> symauth symauth -sid -sid 254 254 enable enable Perform Perform an an Authorization Authorization change change on on Symmetrix Symmetrix unit unit 000190102254 000190102254 (y/[n]) (y/[n]) ? ?
C:\>symauth C:\>symauth set set enforcement enforcement enforce enforce
Perform Perform an an Authorization Authorization change change on on Symmetrix Symmetrix unit unit 000190102254 000190102254 (y/[n]) (y/[n]) ? ? y y
Be default, User Authorization is disabled. As such, any user can make changes to the authorization control data, including creating and removing user-to-role mappings. Once User Authorization is enabled for an array, only users with the Admin or SecurityAdmin role can change authorization control data. Therefore it is recommended that, prior to enabling User Authorization, you first create your user-to-role mappings and map at least one user to the Admin or SecurityAdmin role on an array. A enforcement policy of advise will generate a warning message but allow the operation to proceed. An entry is placed in the SYMAPI log on the host and in the Symmetrix Audit log. A policy of enforce will cause the operation to fail.
User Authentication Status

y Enable User level authentication
Must previously have granted security admin privileges to execute the change
symauth symauth -sid -sid 1254 1254 list list S S Y Y M M M M E E T T R R I I X X A A U U T T H H O O R R I I Z Z A A T T I I O O N N S S T T A A T T U U S S
Symmetrix Symmetrix ID: ID: 000190102254 000190102254 Authorization Authorization Control Control Time Time Enabled Enabled Time Time Disabled Disabled Time Time Updated Updated : : Enabled Enabled : : Mon Mon Aug Aug 13 13 11:52:03 11:52:03 2007 2007 : : Fri Fri Aug Aug 10 10 11:47:10 11:47:10 2007 2007 : : Mon Mon Aug Aug 13 13 11:52:03 11:52:03 2007 2007
Enforcement Enforcement Mode Mode Server Server Policy Policy
: : Advise Advise : : Trust Trust clients clients
The default is to trust user names sent from clients . For additional security when operating in client/server mode, Solutions Enabler can be set to require a secure cross-host authentication (validate_client), such as Kerberos or Windows. If validate_client is set, the server is required to validate a user's identity via a distributed authentication mechanism. To change the server policy to trust clients, enter: symauth -sid 1234 set server_policy trust_client
Listing Users and Roles

y Identify defined users and roles
symauth symauth -sid -sid 1234 1234 list list -users -users Symmetrix Symmetrix ID: ID: 000000001234 000000001234 Role Role name name --------------------------------Admin Admin Admin Admin StorageAdmin StorageAdmin StorageAdmin StorageAdmin Monitor Monitor Monitor Monitor Username Username ------------------------------------------------------------------------------D:Eng\moe D:Eng\moe D:Eng\larry D:Eng\larry D:Eng\curly D:Eng\curly D:Eng\gilligan D:Eng\gilligan D:\Eng\laura D:\Eng\laura D:\Eng|hardy D:\Eng|hardy
Monitoring
y A second part of security is monitoring activity
Who? What? When?
y On the host
The SYMAPI log tracks all syscalls that fail due to access control or user authentication activity
y On the Symmetrix
The Audit log tracks all successful and unsuccessful access attempts
symaudit command
Nearly as important as restricting access is understanding who is attempting to do what operation, from what system, when.
Symmetrix Audit Log

Host Host Host
y Data is written to the audit log during control operations initiated by host applications
Secure, persistent circular log Activity from all hosts into one file
y The symaudit command is used to retrieve audit information

Show High level details about log List - Filter the audit data to match criteria; date/time, application, host, etc Monitor - Run in the foreground polling the Symmetrix for new audit log records using Interval and Count
FA
FA
GK GK
Audit Log
Symmetrix Symmetrix File File System System
Symmetrix
Enginuity collects chronological list of host-initiated Symmetrix actions and activities. Manual activities (for example, physically removing/replacing a component) as well as automatically initiated scripts and EMCs Solutions Enabler activities (for example, TimeFinder or SRDF routines), are tracked and recorded in the SFS. This provides a means to oversee and historically recall how and when a Symmetrix device is being used.
Symmetrix Audit symaudit High Level

y Persistent y Circular log file
C:\>symaudit C:\>symaudit show show
A A U U D D I I T T
L L O O G G
D D A A T T A A
Symmetrix Symmetrix ID ID Starting Starting date date Ending Ending date date Starting Starting record record number number Ending Ending record record number number Total Total record record count count
: : : : : : : : : : : :
000190102254 000190102254 08/08/2007 08/08/2007 16:58:44 16:58:44 08/14/2007 08/14/2007 04:19:13 04:19:13 1 1 151 151 151 151
The Audit log is a circular log file that will overwrite the oldest entry. symaudit show displays a summary of the contents of the audit log.
Symmetrix Audit symaudit Verbose

y Extensive filtering
# # symaudit symaudit -sid -sid 04 04 list list -v -v -start_time -start_time 8/13:12:00 8/13:12:00 -end_time -end_time 8/14:12:00 8/14:12:00 A A U U D D I I T T L L O O G G D D A A T T A A : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 139 139 2 2 2 2 08/13/07 08/13/07 13:49:39 13:49:39 EMC EMC Corp Corp SYMMASK SYMMASK 6.4.1.0 6.4.1.0 SEK SEK V6.3.0.0 V6.3.0.0 (Edit (Edit Level: Level: 771) 771) W2K3-39-106 W2K3-39-106 WinNT WinNT 5.2.3790Se 5.2.3790Se usxxdiamoj usxxdiamoj 00001304 00001304 00001520 00001520 DevMask DevMask Add Add [ [ 0036 0036 ] ] D:CORP\diamoj D:CORP\diamoj SEa29861bf5c SEa29861bf5c
Record Record Number Number Records Records in in Seq Seq Offset Offset in in Seq Seq Time Time Vendor Vendor ID ID Application Application ID ID Application Application Version Version API API Library Library API API Version Version Host Host Name Name OS OS Name Name OS OS Revision Revision Client Client Host Host Process Process ID ID Task Task ID ID Function Function Class Class Action Action Code Code Text Text Username Username Activity Activity ID ID
Filtering allows you to specify a specific record number, a time range, application, host, user and other selection criteria.
Module Summary
Key points covered in this module: y Host based and user based Access Controls provides options for locking down and enabling secure sharing of a Symmetrix
Grant a host only the required access level on specified volumes Prevents accidental or malicious tampering Supports change control procedures and limits ad-hoc operations
y Managed using the symacl and symauth commands that processes a command file through the review, prepare, and commit phases y symacl requires initial setup including enabling the feature on the Service Processor, creating an administrator PIN and define initial groups y Successful implementation requires planning y Post implementation requires monitoring of API, Event, and Audit logs y Can also be setup and administered using SMC
Before any type of Access Control is configured, extensive planning is required to ensure the implementation is not disruptive to the customers application environment. This includes a survey of all user, host and application requirements for controlling Solutions Enabler functions, and then organizing the requirements into host access groups, device pools, and access control entries.
y PowerLink
Closing Slide

0 d9cdf

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

0 d9cdf

Caricato da

Copyright:

Formati disponibili

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

Symmetrix Monitoring and Management Using Solutions Enabler

2007 EMC Corporation. All rights reserved.

Symmetrix HW & SW Architecture - 1

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

2007 EMC Corporation. All rights reserved.

Symmetrix HW & SW Architecture - 2

Symmetrix HW & SW Architecture - 2

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

Symmetrix HW & SW Architecture - 3

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

Symmetrix Engineering Curriculum

Advanced AdvancedSymmetrix Symmetrix Internals Internals

TimeFinder TimeFinder& &SRDF SRDF Workshop Workshop

Symmetrix HW & SW Architecture - 4

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

y Understanding Symmetrix Configuration

y Configuring Symmetrix Devices

y Symmetrix Management Console

2007 EMC Corporation. All rights reserved.

Symmetrix HW & SW Architecture - 5

Symmetrix HW & SW Architecture - 5

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

y Monitoring Symmetrix Activity

2007 EMC Corporation. All rights reserved.

Symmetrix HW & SW Architecture - 6

Symmetrix HW & SW Architecture - 6

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

2007 EMC Corporation. All rights reserved.

Symmetrix HW & SW Architecture - 7

Symmetrix HW & SW Architecture - 7

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

Symmetrix Monitoring and Management Using Solutions Enabler

Module 1: Symmetrix Hardware and Software Architecture

2007 EMC Corporation. All rights reserved.

Symmetrix HW & SW Architecture - 1

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

2007 EMC Corporation. All rights reserved.

Symmetrix HW & SW Architecture - 2

Symmetrix HW & SW Architecture - 2

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

Symmetrix HW & SW Architecture - 3

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

Symmetrix HW & SW Architecture - 4

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.

Cntl Direct Direct Matrix Matrix

Cntl Direct Direct Matrix Matrix

Multi-protocol CD Cntl Direct Direct Matrix Matrix

Multi-protocol CD Cntl Direct Direct Matrix Matrix

Multi-protocol CD Cntl Direct Direct Matrix Matrix

Multi-protocol CD Cntl Direct Direct Matrix Matrix

Control and communications signals

Control and communications signals

Direct Direct Matrix Matrix Cntl FC (back-end)

Direct Direct Matrix Matrix Cntl FC (BE or FE)

Direct Direct Matrix Matrix Cntl FC (BE or FE)

Direct Direct Matrix Matrix Cntl FC (BE or FE)

Direct Direct Matrix Matrix Cntl FC (BE or FE)

Direct Direct Matrix Matrix Cntl FC (back-end)

Direct Direct Matrix Matrix Cntl FC (back-end)

Fibre Channel back-end directors

Symmetrix HW & SW Architecture - 5

Copyright 2007 EMC Corporation. Do not Copy - All Rights Reserved.